Common Lisp the Language, 2nd Edition

Next: Numbers Up: Packages Previous: Modules

# 11.9. An Example

Most users will want to load and use packages but will never need to build one. Often a user will load a number of packages into the `user` package whenever using Common Lisp. Typically an implementation might provide some sort of initialization file mechanism to make such setup automatic when the Lisp starts up. Table 11-1 shows such an initialization file, one that simply causes other facilities to be loaded.

X3J13 voted in March 1989 (LISP-PACKAGE-NAME) to specify that the forthcoming ANSI Common Lisp will use the package name `common-lisp-user` instead of `user`.

    ----------------------------------------------------------------
    Table 11-1: An Initialization File

    ;;;; Lisp init file for I. Newton.

    ;;; Set up the USER package the way I like it.

    (require 'calculus)             ;I use CALCULUS a lot; load it.
    (use-package 'calculus)         ;Get easy access to its
                                    ; exported symbols.

    (require 'newtonian-mechanics)  ;Same thing for NEWTONIAN-MECHANICS
    (use-package 'newtonian-mechanics)

    ;;; I just want a few things from RELATIVITY,
    ;;; and other things conflict.
    ;;; Import only what I need into the USER package.

    (require 'relativity)
    (import '(relativity:speed-of-light
              relativity:ignore-small-errors))

    ;;; These are worth loading, but I will use qualified names,
    ;;; such as PHLOGISTON:MAKE-FIRE-BOTTLE, to get at any symbols
    ;;; I might need from these packages.

    (require 'phlogiston)
    (require 'alchemy)

    ;;; End of Lisp init file for I. Newton.

    ----------------------------------------------------------------

When each of two files uses some symbols from the other, the author of those files must be careful to arrange the contents of the file in the proper order. Typically each file contains a single package that is a complete module. The contents of such a file should include the following items, in order:

1. A call to `provide` that announces the module name.

2. A call to `in-package` that establishes the package.

3. A call to `shadow` that establishes any local symbols that will shadow symbols that would otherwise be inherited from packages that this package will use.

4. A call to `export` that establishes all of this package's external symbols.

5. Any number of calls to `require` to load other modules that the contents of this file might want to use or refer to. (Because the calls to `require` follow the calls to `in-package`, `shadow`, and `export`, it is possible for the packages that may be loaded to refer to external symbols in this package.)

6. Any number of calls to `use-package`, to make external symbols from other packages accessible in this package.

7. Any number of calls to `import`, to make symbols from other packages present in this package.

8. Finally, the definitions making up the contents of this package/module.

The following mnemonic sentence may be helpful in remembering the proper order of these calls:

> Put in seven extremely random user interface commands.

Each word of the sentence corresponds to one item in the above ordering:

               Put             Provide
               IN              IN-package
               Seven           Shadow
               EXtremely       EXport
               Random          Require
               USEr            USE-package
               Interface       Import
               COmmands        COntents of package/module

The sentence says what it helps you to do.

The most distressing aspect of the X3J13 vote to eliminate `provide` and `require` (REQUIRE-PATHNAME-DEFAULTS) is of course that it completely ruins the mnemonic sentence.

Now, suppose for the sake of example that the `phlogiston` and `alchemy` packages are single-file, single-package modules as described above. The `phlogiston` package needs to use the `alchemy` package, and the `alchemy` package needs to use several external symbols from the `phlogiston` package. The definitions in the `alchemy` and `phlogiston` files (see tables 11-2 and 11-3) allow a user to specify `require` statements for either of these modules, or for both of them in either order, and all relevant information will be loaded automatically and in the correct order.

    ----------------------------------------------------------------
    Table 11-2: File alchemy

    ;;;; Alchemy functions, written and maintained by Merlin, Inc.

    (provide 'alchemy)                  ;The module is named ALCHEMY.
    (in-package 'alchemy)               ;So is the package.

    ;;; There is nothing to shadow.

    ;;; Here is the external interface.

    (export '(lead-to-gold gold-to-lead
              antimony-to-zinc elixir-of-life))

    ;;; This package/module needs a function from
    ;;; the PHLOGISTON package/module.

    (require 'phlogiston)

    ;;; We don't frequently need most of the external symbols from
    ;;; PHLOGISTON, so it's not worth doing a USE-PACKAGE on it.
    ;;; We'll just use qualified names as needed.  But we use
    ;;; one function, MAKE-FIRE-BOTTLE, a lot, so import it.
    ;;; It's external in PHLOGISTON and so can be referred to
    ;;; here using ":" qualified-name syntax.

    (import '(phlogiston:make-fire-bottle))

    ;;; Now for the real contents of this file.

    (defun lead-to-gold (x)
      "Takes a quantity of lead and returns gold."
      (when (> (phlogiston:heat-flow 5 x x)  ;Using a qualified symbol
               3)
        (make-fire-bottle x))                ;Using an imported symbol
      (gild x))

    ;;; And so on ...

    ----------------------------------------------------------------

    ----------------------------------------------------------------
    Table 11-3: File phlogiston

    ;;;; Phlogiston functions, by Thermofluidics, Ltd.

    (provide 'phlogiston)             ;The module is named PHLOGISTON.
    (in-package 'phlogiston)          ;So is the package.

    ;;; There is nothing to shadow.

    ;;; Here is the external interface.

    (export '(heat-flow cold-flow mix-fluids separate-fluids
              burn make-fire-bottle))

    ;;; This file uses functions from the ALCHEMY package/module.

    (require 'alchemy)

    ;;; We use alchemy functions a lot, so use the package.
    ;;; This will allow symbols exported from the ALCHEMY package
    ;;; to be referred to here without the need for qualified names.

    (use-package 'alchemy)

    ;;; No calls to IMPORT are needed here.

    ;;; The real contents of this package/module.

    (defvar *feeling-weak* nil)

    (defun heat-flow (amount x y)
      "Make some amount of heat flow from x to y."
      (when *feeling-weak*
        (quaff (elixir-of-life)))     ;No qualifier is needed.
      (push-heat amount x y))

    ;;; And so on ...

    ----------------------------------------------------------------

For very large modules whose contents are spread over several files (the `lisp` package is an example), it is recommended that the user create the package and declare all of the shadows and external symbols in a separate file, so that this can be loaded before anything that might use symbols from this package.

Indeed, the `defpackage` macro approved by X3J13 in January 1989 (DEFPACKAGE) encourages the use of such a separate file. (By the way, X3J13 voted in March 1989 (LISP-PACKAGE-NAME) to specify that the forthcoming ANSI Common Lisp will use the package name `common-lisp` instead of `lisp`.) Let's take a look at a revision of I. Newton's files using `defpackage`.

The new version of the initialization file avoids using `require`; instead, we assume that `load` will do the job (see table 11-4).

    ----------------------------------------------------------------
    Table 11-4: An Initialization File When defpackage Is Used

    ;;;; Lisp init file for I. Newton.

    ;;; Set up the USER package the way I like it.

    (load "calculus")               ;I use CALCULUS a lot; load it.
    (use-package 'calculus)         ;Get easy access to its
                                    ; exported symbols.

    (load "newtonian-mechanics")    ;Ditto for NEWTONIAN-MECHANICS
    (use-package 'newtonian-mechanics)

    ;;; I just want a few things from RELATIVITY,
    ;;; and other things conflict.
    ;;; Import only what I need into the USER package.

    (load "relativity")
    (import '(relativity:speed-of-light
              relativity:ignore-small-errors))

    ;;; These are worth loading, but I will use qualified names,
    ;;; such as PHLOGISTON:MAKE-FIRE-BOTTLE, to get at any symbols
    ;;; I might need from these packages.

    (load "phlogiston")
    (load "alchemy")

    ;;; End of Lisp init file for I. Newton.

    ----------------------------------------------------------------

The other files have each been split into two parts, one that establishes the package and one that defines the contents. This example uses a simple convention that for any file named, say, ```foo`'' the file named ```foo-package`'' contains the necessary `defpackage` and/or other package-establishing code. The idiom

    (unless (find-package "FOO")
      (load "foo-package"))

is conventionally used to load a package definition but only if the package has not already been defined. (This is a bit clumsy, and there are other ways to arrange things so that a package is defined no more than once.)

The file `alchemy-package` is shown in table 11-5. The tricky point here is that the `alchemy` and `phlogiston` packages contain mutual references (each imports from the other), and so `defpackage` alone cannot do the job. Therefore the `phlogiston` package is not mentioned in a `:use` option in the `defpackage` for the `alchemy` package. Instead, the function `use-package` is called explicitly, after the package definition for `phlogiston` has been loaded. Note that this file has been coded with excruciating care so as to operate correctly even if the package current when the file is loaded does not inherit from the `common-lisp` package. In particular, the standard load-package-definition idiom has been peppered with package qualifiers:

    (cl:unless (cl:find-package "PHLOGISTON")
      (cl:load "phlogiston-package"))

Note the use of the nickname `cl` for the `common-lisp` package.

The `alchemy` file, shown in table 11-6, simply loads the `alchemy` package definition, makes that package current, and then defines the ``real contents'' of the package.

    ----------------------------------------------------------------
    Table 11-5: File alchemy-package Using defpackage

    ;;;; Alchemy package, written and maintained by Merlin, Inc.

    (cl:defpackage "ALCHEMY"
      (:export "LEAD-TO-GOLD" "GOLD-TO-LEAD"
               "ANTIMONY-TO-ZINC" "ELIXIR-OF-LIFE")
      )

    ;;; This package needs a function from the PHLOGISTON package.
    ;;; Load the definition of the PHLOGISTON package if necessary.

    (cl:unless (cl:find-package "PHLOGISTON")
      (cl:load "phlogiston-package"))

    ;;; We don't frequently need most of the external symbols from
    ;;; PHLOGISTON, so it's not worth doing a USE-PACKAGE on it.
    ;;; We'll just use qualified names as needed.  But we use
    ;;; one function, MAKE-FIRE-BOTTLE, a lot, so import it.
    ;;; It's external in PHLOGISTON and so can be referred to
    ;;; here using ":" qualified-name syntax.

    (cl:import '(phlogiston:make-fire-bottle))

    ----------------------------------------------------------------

    ----------------------------------------------------------------
    Table 11-6: File alchemy Using defpackage

    ;;;; Alchemy functions, written and maintained by Merlin, Inc.

    (unless (find-package "ALCHEMY")
      (load "alchemy-package"))

    (in-package 'alchemy)

    (defun lead-to-gold (x)
      "Takes a quantity of lead and returns gold."
      (when (> (phlogiston:heat-flow 5 x x)  ;Using a qualified symbol
               3)
        (make-fire-bottle x))                ;Using an imported symbol
      (gild x))

    ;;; And so on ...

    ----------------------------------------------------------------

The file `phlogiston-package` is shown in table 11-7. This one is a little more straightforward than the file `alchemy-package`, because the latter bears the responsibility for breaking the circular package references. This file simply makes sure that the `alchemy` package is defined and then performs a `defpackage` for the `phlogiston` package.

    ----------------------------------------------------------------
    Table 11-7: File phlogiston-package Using defpackage

    ;;;; Phlogiston package definition, by Thermofluidics, Ltd.

    ;;; This package uses functions from the ALCHEMY package.

    (cl:unless (cl:find-package "ALCHEMY")
      (cl:load "alchemy-package"))

    (cl:defpackage "PHLOGISTON"
      (:use "COMMON-LISP" "ALCHEMY")
      (:export "HEAT-FLOW"
               "COLD-FLOW"
               "MIX-FLUIDS"
               "SEPARATE-FLUIDS"
               "BURN"
               "MAKE-FIRE-BOTTLE")
      )

    ----------------------------------------------------------------

The `phlogiston` file, shown in table 11-8, simply loads the `phlogiston` package definition, makes that package current, and then defines the ``real contents'' of the package.

    ----------------------------------------------------------------
    Table 11-8: File phlogiston Using defpackage

    ;;;; Phlogiston functions, by Thermofluidics, Ltd.

    (unless (find-package "PHLOGISTON")
      (load "phlogiston-package"))

    (in-package 'phlogiston)

    (defvar *feeling-weak* nil)

    (defun heat-flow (amount x y)
      "Make some amount of heat flow from x to y."
      (when *feeling-weak*
        (quaff (elixir-of-life)))     ;No qualifier is needed.
      (push-heat amount x y))

    ;;; And so on ...

    ----------------------------------------------------------------

Let's look at the question of package circularity in this example a little more closely. Suppose that the file `alchemy-package` is loaded first. It defines the `alchemy` package and then loads file `phlogiston-package`. That file in turn finds that the package `alchemy` has already been defined and therefore does not attempt to load file `alchemy-package` again; it merely defines package `phlogiston`. The file `alchemy-package` then has a chance to import `phlogiston:make-fire-bottle` and everything is fine.

On the other hand, suppose that the file `phlogiston-package` is loaded first. It finds that the package `alchemy` has not already been defined, and therefore it immediately loads file `alchemy-package`. That file in turn defines the `alchemy` package; then it finds that package `phlogiston` is not yet defined and so loads file `phlogiston-package` again (indeed, in nested fashion). This time file `phlogiston-package` does find that the package `alchemy` has already been defined, so it simply defines package `phlogiston` and terminates. The file `alchemy-package` then imports `phlogiston:make-fire-bottle` and terminates. Finally, the outer loading of file `phlogiston-package` re-defines package `phlogiston`. Oh, dear. Fortunately the two definitions of package `phlogiston` agree in every detail, so everything ought to be all right. Still, it looks a bit dicey; I certainly don't have the same warm, fuzzy feeling that I would if no package were defined more than once.

Conclusion: `defpackage` goes a long way, but it certainly doesn't solve all the possible problems of package and file management. Neither did `require` and `provide`. Perhaps further experimentation will yield facilities appropriate for future standardization.

Next: Numbers Up: Packages Previous: Modules