ASDF - Another System Definition Facility
Author: the ASDF group

## ASDF

#### Another System Definition Facility

- What it is
- What it is not
- Supported Implementations
- Examples
- Documentation
- Getting it
- Extensions
- Reporting Bugs
- Mailing Lists
- What is happening

### ASDF 3

ASDF is the de facto standard build facility for Common Lisp. Your Lisp implementation probably contains a copy of ASDF, which you can load using `(require "asdf")`.

ASDF 3 is the current successor to Daniel Barlow's ASDF (created on August 1st 2001) and François-René Rideau's ASDF 2 (released May 31st 2010). It was rewritten for improved portability, robustness, usability, extensibility, configurability, internal consistency, and the ability to deliver standalone executables, all while maintaining substantial backward compatibility. Its notable versions include pre-release 2.27 on February 1st 2013, first stable release 3.0.1 on May 16th 2013, major releases 3.1.2 on May 6th 2014, 3.2.0 on January 10th 2017 and 3.3.0 on October 6th 2017. The latest release is 3.3.4, published on February 14th, 2020.

### What it is

ASDF is what Common Lisp hackers use to build and load software. It is the successor of the Lisp `DEFSYSTEM` of yore. ASDF stands for Another System Definition Facility.

ASDF 3 contains two parts: `asdf/defsystem` and `uiop`.

`asdf/defsystem`

  is a tool to describe how Lisp source code is organized in systems, and how to build and load these systems. The build happens based on a plan in term of actions that depend on previous actions; the plan is computed from the structure of the systems.

  Typical actions consist in compiling a Lisp source file (unless already up to date) and loading the resulting compilation output (unless both already loaded and up to date). And you must typically compile and load files that define packages, macros, variables, before you may compile and load other files that use them.

  If you come from the C/C++ world, ASDF covers a bit of what each of `make`, `autoconf`, `dlopen` and `libc` do for C programs: it orchestrates the compilation and dependency management, handles some of the portability issues, dynamically finds and loads code, and offers some portable system access library (see `uiop` below for the latter). Except everything is different in Common Lisp, and ultimately much simpler overall, though it does require acquiring some basic concepts that do not exactly match those of the C and Unix world. Importantly, ASDF builds all software in the current Lisp image, as opposed to building software into separate processes.

  `asdf/defsystem` is the part that people usually refer to as `ASDF`, with `uiop` being only a supporting library, that happens to be distributed at the same time, by necessity.

`uiop`

  the Utilities for Implementation- and OS- Portability, formerly known as `asdf/driver`, is a Common Lisp portability library and runtime support system that helps you write Common Lisp software in a portable way.

  In addition to many general-purpose Lisp utilities, it notably provides portable abstractions to gloss over implementation quirks, support hot-upgrade of code, manipulate pathnames, create programs, use command-line arguments, access the environment, use the filesystem, call other programs and parse their output, compile Lisp code, muffle conditions, or configure Lisp software. See its README.md for an overview, and the documentation as extracted from its docstrings by Declt or HEΛP (NB: in HEΛP, you can, though it's not obvious, scroll the list of packages with a scrollbar on the right of the top-left pane, and then click on the package you're interested in to browse its symbols).

  `uiop` is distributed as part of `ASDF`: its source code is transcluded in the single-file `asdf.lisp` being distributed and the precompiled fasls provided by Lisp implementations. `ASDF` relies heavily on it for its portability layer and runtime support, particularly so as to handle pathnames and filesystem access. `uiop` is useful on its own and can also be compiled and distributed separately.

### What it is not

ASDF will not download missing software components for you. For that, you want `Quicklisp`, that builds upon ASDF, and is great for pulling and installing tarballs of packages you may depend upon. We also recommend `clbuild`, that now builds upon Quicklisp, as a great tool for pulling from version control packages you need to modify or want to contribute to. We recommend you should not use `asdf-install` anymore, as it is an older similar piece of software that is both unmaintained and obsolete.

ASDF is also not a tool to build or run Common Lisp software from the Unix command-line. For that, you want cl-launch, buildapp, or roswell.

If you're unsatisfied with ASDF, beside helping with our TODO list, you might be interested in other build systems for Common-Lisp:

- Google's deterministic and scalable build system Bazel, for which Lisp support is available: `bazelisp`.
- Alastair Bridgewater's small and simple one-package-per-file `quick-build` (also reimplemented as the ASDF extension `asdf-package-system`, now part of ASDF 3; similar to `faslpath` below).
- François-René Rideau's `XCVB` (building object and image files deterministically and in parallel, but not actively maintained and bitrotten since 2012; a hypothetical ASDF 4 could conceivably be evolved to support these features thanks to the groundwork laid by ASDF 3; or you could use Bazel).
- Drew McDermott's `YTools` (the polar opposite of XCVB, trying to maintain coherence of the current Lisp image at a fine grain).
- Dmitriy Ivanov's `ASDlite` (a somewhat improved incompatible variant of ASDF 1, much less featureful, robust or portable than ASDF 3),
- Mark Kantrowitz's `mk-defsystem` (free software successor of the old proprietary DEFSYSTEM's and predecessor of ASDF, obsolete),
- Sean Ross's `mudballs` (an attempt at making things cleaner than in ASDF 2, aborted),
- Peter von Etter's `faslpath` (a much simpler system establishing a mapping between packages and files, abandoned but see `asdf-package-system` and `quick-build` above),
- Alexander Kahl's `evol` (a reimplementation in Lisp of the GNU autotools stack, abandoned).

There are probably more. However, none of these systems seems to ever have had the traction of ASDF, probably because none was technically superior and/or portable enough (if at all) to compensate for the first mover advantage.

### Supported Implementations

ASDF 3 now supports all CL implementations that seem to have any current user base, and then some. But ASDF does not magically turn broken implementations into working ones, and some ASDF or UIOP features may not work on less-maintained implementations that do not support them (see below).

Most implementations provide ASDF as a module, and you can simply `(require "asdf")`. (All of them but CLISP also accept `:asdf`, `"ASDF"` or `'asdf` as an argument.) All these implementations provide ASDF 3.1 or later in their latest version, but your software distribution might have an older version.

As for remaining implementations, they are obsolete and/or mostly unmaintained; ASDF was made to run with each of them at some point, but some hacking is probably required to make the latest ASDF work well with the latest release of these implementations:

- CLISP is generally well-tested with ASDF, though it has a few minor bugs. The implementation has received only minimal maintenance since 2010, and the last official release doesn't ship with ASDF, or often has antique version of ASDF tucked on it. However, ASDF has been recently updated in its source control system, and there is hope of a new release some day soon. If needs be, you can replace CLISP's provided ASDF with a newer one, or use an upgrade on top of that.
- CormanLisp was recently open-sourced. Its bundled ASDF has not yet been updated from 1.x; ASDF 3 should work fine with it, but several features were disabled because CormanLisp was insufficiently compliant with the CLHS, and requires some work in this regard. Ask the new CormanLisp maintainers for updates.
- ECL is generally well-tested with ASDF, but its bytecode compiler doesn't support the bundle operations, and support for Windows seems to be less stable. For instance, as of January 2017, `cl:require` and `ext:system` (and thus `uiop:run-program`) seem not to be working correctly on Windows.
- GCL is somewhat maintained but its maintainer doesn't seem to care about ASDF, and hasn't replied for years to requests for bug fixes or for providing ASDF via `(require "asdf")`
- Genera was never open-source and never bundled ASDF, but should otherwise just work with a recent ASDF 3, if you somehow have a license and a working version. There are rumors of people doing active development with it and having minor patches to improve ASDF on it.
- MCL similarly was open-sourced, but never bundled ASDF; ASDF 3 should work just fine with it, but Rosetta is not supported in the latest versions of MacOS X, so a lot of work is required to make something out of it — at which point, you might just use CCL.
- Mocl has its own heavily modified variant of ASDF2, and is the only implementation not currently supported by ASDF3. To make it work with ASDF 3 would require ASDF to be taught about natively supporting cross-compilation.
- SCL was seemingly abandoned and never open-sourced. It never bundled ASDF, but otherwise should just work with a recent ASDF 3, if you somehow have a license and a working version.
- XCL is now an abandoned experiment. It provides some old ASDF 2; but you can replace it with ASDF 3, which works well with it, inasmuch as anything works at all with XCL.

| Provide ASDF 3.1 or later | No ASDF | Unmaintained
--- | --- | --- | ---
Free | ABCL, CCL, Clasp, CLISP, CMUCL, ECL, Mezzano, MKCL, SBCL | CormanLisp, GCL, MCL | XCL
Proprietary | Allegro, LispWorks |  | Genera, mocl, SCL

To deal with an implementation that does not yet provide ASDF 3.1 or later, we provide a script that will install the ASDF from your git checkout to where your implementation goes looking for it when you `(require "asdf")`.

Note that upgrading from an old version of ASDF 2 or earlier is possible, but quite complex to do right in a robust way, and we do not recommend it. Also note that `mocl` only supports a heavily modified variant of ASDF 2, and will require robust cross-compilation support to be added to ASDF before it is actually supported.

If there is an old or new implementation that we are missing, it shouldn't be hard to adapt ASDF to support it. Ask us!

### Examples

Download any of the many packages available through Quicklisp to see as many examples.

### Documentation

You can read our manual:

- ASDF

- as one HTML file
- split into one HTML file per section
- as a PDF document
- as texinfo source

- UIOP

- as one HTML file
- as a PDF document

The first few sections, Loading ASDF, Configuring ASDF and Using ASDF will get you started as a simple user.

If you want to define your own systems, further read the section Defining systems with defsystem.

There is now also a Best Practices document explaining recommended usage patterns when writing `.asd` files, and detailing common pitfalls or deprecated practices to avoid.

About the latest developments in ASDF 3.2 and ASDF 3.3, see our demo Delivering Common Lisp Applications with ASDF 3.3 (2-page PDF, slides, git). Regarding the internal design of ASDF in general, and the work we did on ASDF 3, see the extended version (26 pages) of our paper ASDF 3, or Why Lisp is Now an Acceptable Scripting Language (PDF, git). The shorter version (8 pages), presented at ELS 2014, focuses on ASDF 3 and misses historical and technical information (PDF, HTML). Regarding ASDF 3, see also the slides of the ASDF 3 tutorial presented at ELS 2013, and for an introduction to the source code, this video: ASDF 3.1 walkthrough. For details about our previous work on ASDF 2, see our paper presented at ILC 2010, Evolving ASDF: More Cooperation, Less Coordination (git).

Finally, while the manual covers all the basics, some advanced or new features remain underdocumented. Please contact our mailing-list (see below) regarding any feature that isn't well-documented enough.

Until we write more documentation on the further innovations of ASDF, the documentation strings, the source code, the changelog and the git log are unfortunately your best chances for discovering the available functionality.

### Getting it

Though they may lag behind the version here, ASDF comes bundled with most Lisps. To get the greatest and latest, you can:

- download just the latest release source for asdf.lisp;
- download the latest release tarball to get all bells and whistles;
- pull the latest development tree from our git repository

      git clone https://gitlab.common-lisp.net/asdf/asdf.git

  (note that our "master" branch is for current development; get our "release" branch for the latest stable release. Run `make` to create `build/asdf.lisp`);
- browse the latest development tree from our git repository: gitlab
- download the tarball of a past release: archives

### Extensions

Known extensions to ASDF include:

- `asdf-dependency-grovel`, to compute the actual dependencies in a big ASDF system.
- `asdf-encodings`, to compile Lisp source files with character encodings other than UTF-8.
- `asdf-finalizers`, to allow macros to include code to be evaluated at the end of a file being compiled.
- `asdf-flv`, to bind file-local variables around the compilation of some files.
- `asdf-system-connections`, lets you specify systems that are automatically loaded when two other systems are loaded, to connect them.
- `cffi`, lets you interface between functions and datastructures written in C and functions written in Lisp, including support for automatically detecting constants from C macros, linking to dynamic (and now also static) libraries, and writing your own wrapper code in C.
- `poiu`, to compile a system in parallel on a multiprocessor machine.

Former extensions, now superseded, include:

- `asdf-binary-locations` used to allow one to redirect where ASDF 1 created its output files, so they don't clash between implementations and don't pollute source directories. It is superseded by `asdf/defsystem`'s builtin `asdf-output-translations` mechanism; a limited compatibility mode is available to easily convert your former ABL configuration into an AOT configuration. `common-lisp-controller` and `cl-launch` used to provide similar mechanisms, and have also been superseded by `asdf-output-translations` (built into ASDF 2 and later).
- `asdf-bundle`, née `asdf-ecl`, allowed you to create a single-file bundle out of a system, for easier delivery. It is now a builtin part of `asdf/defsystem`, and allows users to deliver a single FASL for a system, a standalone executable program (on supported implementations), or an image containing your system precompiled.
- `asdf-condition-control`, initially part of XCVB's `xcvb-driver`, allowed you to muffle uninteresting conditions during compilation. Is now superseded by equivalent functionality in `uiop`.
- `asdf-contrib`, an empty package that used to collect dependencies on other systems in the list above and below.
- `asdf-package-system`, to compile Lisp source files with one package per file that also determines dependencies, in the style of `quick-build` or `faslpath` (this functionality is built into recent versions of ASDF 3.1 and later, but this package exists for backward compatibility with earlier versions of ASDF 3; search the manual for `package-inferred-system`).
- `asdf-utils` was a collection of utilities that originated with ASDF. It is now superseded by `uiop`, aka `asdf/driver`, which is part of ASDF, and exports its functionality in its own package `uiop`.

### Contributing

Join our mailing list, check the code out from git, send questions, ideas and patches!

### Reporting Bugs

To report bugs, you can use our common-lisp.net project. If you're unsure about the bug or want to discuss how to fix it, you can send email to the project mailing-list below.

Note that the most valuable thing you can send this way are test cases, if possible as `.script` files readily runnable by our test system. If you're courageous, send us merge requests on gitlab.

While bug fixes are useful, they are not usually as valuable as test cases: small easy fixes will be obvious from the test case, and large fixes written by someone who isn't either a maintainer or working tightly with one will probably not be correct and not fit the codebase: any modification at one point is likely to have repercussions at other unobvious places in the codebase, for the code to be correct in a wider variety of scenarios than casual developers usually think about. Now, if you're willing to become a maintainer, you're welcome to join the team!

NB: Previously, we had done bug-tracking on launchpad.net, but we are now consolidating project management on common-lisp.net. If you are interested in hacking on ASDF, you may look at the bugs on launchpad for more work to do (if only to migrate the bugs to gitlab).

### Mailing Lists

- asdf-devel A list for questions, suggestions, bug reports, patches, and so on. It's for everyone and everything. Please join the conversation! asdf-devel mailman site to subscribe
- asdf-announce A low-volume mailing-list for announcements only, mostly regarding new releases. Posting is restricted to project administrators and to important notices. Please subscribe to it if you're a Lisp implementation or distribution vendor, who needs to know when to upgrade the ASDF you distribute, but are otherwise not interested in day to day design and development. asdf-announce mailman site to subscribe

### Contributing

Join our mailing list, check the code out from git, send questions, ideas and patches!

### What is happening

For a detailed description of changes see our Changelog

February 2020
  Release of 3.3.4: fourth bugfix release for the 3.3 release series.
March 2019
  Release of 3.3.3: third bugfix release for the 3.3 release series.
May 2018
  Release of 3.3.2: second bugfix release for the 3.3 release series.
November 2017
  Release of 3.3.1: bugfix release for 3.3.0 addressing a backwards-incompatibility issue with timestamp computations, and other minor breakage.
October 2017
  Release of 3.3.0, with substantial revision to the build plan to correct handle multiple phases of loading, due to e.g. `DEFSYSTEM-DEPENDS-ON` dependencies.
April 2017
  Release of 3.2.1, with many small bugfixes and cleanups. Presentation at ELS 2017 about ASDF 3.3.
January 2017
  Release of 3.2.0, a release containing many notable improvements, such as a new portable `uiop:launch-program` facility for spawning asynchronous subprocesses (with many thanks to Elias Pipping), a `uiop:with-deprecation` facility to handle progressive deprecation of functions, a cleanup and tightening of the internal dependency model (you are now required to use `make-operation` to instantiate an operation class; also you'll be WARNed if your `.asd` file contains improperly named secondary systems), a systematic pass of adding documentation to all functions, many fixes to small bugs and portability issues across all underlying platforms and operating systems, an improved test suite, and the removal of some long deprecated functionality.
March 2016
  Release of 3.1.7, another bug fix release for the 3.1.x series.
October 2015
  Although we had hoped that ASDF 3.1.5 would be the final release in the ASDF 3.1 series, a number of bug reports led us to prepare release 3.1.6. Support for Windows continues to improve, and we wished to release a number of bug fixes, and support the recent Allegro Common Lisp 10.0 release.
July 2015
  An extensive bout of bug-fixing, notably on Windows, leads to release of ASDF 3.1.5 on 21 July 2015. XDG handling has been improved to be more compliant with the standard. Preliminary support for immutable systems has been added.
May 2015
  With the LispWorks 7.0 release, all actively maintained CL implementations are now providing ASDF 3.0 or later, and support for older variants is now officially dropped.
October 2014
  More bug fixing leads to release of 3.1.4 on 10 October 2014. There should be no incompatibilities. See the Changelog for more details.
August 2014
  The ASDF mailing lists have been reestablished, in particular asdf-announce, which should allow CL implementers better access to only the information they want about ASDF development.
May 2014 to July 2014
  ASDF bug fixing from 3.1.2 leads to release of 3.1.3, a major bug fix release. We strongly urge implementors that have shipped with 3.1.2 to upgrade to 3.1.3. There should be no incompatibilities, and some very important bug fixes are provided. See the Changelog for more details.
July 2013 to May 2014
  François-René Rideau has resigned as maintainer but remained an active developer. Robert P. Goldman is interim maintainer until someone more gifted, charming, dedicated, and better-looking can be secured to fill the role. ASDF 3.0.2 was released in July 2013, 3.0.3 in October 2013, and 3.1.2 in May 2014. In addition to significant improvements and bug fixes, notably better Windows support, ASDF 3.1.2 notably sports the `package-inferred-system` extension.
November 2012 to June 2013
  François-René Rideau completely rewrites ASDF and publishes ASDF 3, pre-released as 2.27 in February 2013, and released as 3.0.1 in May 2013. It now includes both the traditional `asdf/defsystem` and a formalized portability library `uiop` (née `asdf/driver`). `asdf/defsystem` is a backward-compatible reimplementation of ASDF with correct timestamp propagation based on a consistent dependency model, and featuring support for bundle output, deferred warnings check, and more. `uiop` provides many abstractions to write portable Common Lisp programs. Last version: 3.0.1.
December 2009 to October 2012
  François-René Rideau is de facto maintainer, with notable contributions from Robert P. Goldman, but also Juanjo Garcia-Ripoll and James Anderson. ASDF 2.000 is released in May 2010 with many clean-ups, better configurability, some new features, and updated documentation. The ASDF 2 series culminates with ASDF 2.26 in October 2012, which in addition to many bug fixes and small features includes support for file encodings, around-compile and compile-check hooks. Last version: 2.26.
May 2006 to November 2009
  Gary King is de facto maintainer, with notable contributions from Robert P. Goldman, Nikodemus Siivola, Christophe Rhodes, Daniel Herring. Many small features and bug fixes, making the project more maintainable, moving to using git and common-lisp.net. Last version: 1.369.
May 2004 to April 2006
  Christophe Rhodes is de facto maintainer, with notable contributions from Nikodemus Siivola, Peter Van Eynde, Edi Weitz, Kevin Rosenberg. The system made slightly more robust, a few more features. Last version: 1.97.
August 2001 to May 2004
  Created then developed by Daniel Barlow, with notable contributions from Christophe Rhodes, Kevin Rosenberg, Edi Weitz, Rahul Jain. Last version: 1.85.