Common Lisp the Language, 2nd Edition

Next: Function Invocation Up: Control Structure Previous: Assignment

# 7.2. Generalized Variables

In Lisp, a variable can remember one piece of data, that is, one Lisp object. The main operations on a variable are to recover that object and to alter the variable to remember a new object; these operations are often called access and update operations. The concept of variables named by symbols can be generalized to any storage location that can remember one piece of data, no matter how that location is named. Examples of such storage locations are the car and cdr of a cons, elements of an array, and components of a structure.

For each kind of generalized variable, typically there are two functions that implement the conceptual access and update operations. For a variable, merely mentioning the name of the variable accesses it, while the `setq` special form can be used to update it. The function `car` accesses the car of a cons, and the function `rplaca` updates it. The function `symbol-value` accesses the dynamic value of a variable named by a given symbol, and the function `set` updates it.

Rather than thinking about two distinct functions that respectively access and update a storage location somehow deduced from their arguments, we can instead simply think of a call to the access function with given arguments as a name for the storage location. Thus, just as `x` may be considered a name for a storage location (a variable), so `(car x)` is a name for the car of some cons (which is in turn named by `x`). Now, rather than having to remember two functions for each kind of generalized variable (having to remember, for example, that `rplaca` corresponds to `car`), we adopt a uniform syntax for updating storage locations named in this way, using the `setf` macro. This is analogous to the way we use the `setq` special form to convert the name of a variable (which is also a form that accesses it) into a form that updates it. The uniformity of this approach is illustrated in the following table.

    Access Function     Update Function      Update Using setf
    ----------------------------------------------------------------------
    x                   (setq x datum)       (setf x datum)
    (car x)             (rplaca x datum)     (setf (car x) datum)
    (symbol-value x)    (set x datum)        (setf (symbol-value x) datum)
    ----------------------------------------------------------------------

`setf` is actually a macro that examines an access form and produces a call to the corresponding update function.

Given the existence of `setf` in Common Lisp, it is not necessary to have `setq`, `rplaca`, and `set`; they are redundant. They are retained in Common Lisp because of their historical importance in Lisp. However, most other update functions (such as `putprop`, the update function for `get`) have been eliminated from Common Lisp in the expectation that `setf` will be uniformly used in their place.

[Macro]
`setf` `{place` `newvalue}*`

`(setf place newvalue)` takes a form place that when evaluated accesses a data object in some location and ``inverts'' it to produce a corresponding form to update the location. A call to the `setf` macro therefore expands into an update form that stores the result of evaluating the form newvalue into the place referred to by the access form.

If more than one place-newvalue pair is specified, the pairs are processed sequentially; that is,

    (setf place1 newvalue1
          place2 newvalue2)
          ...
          placen newvaluen)

is precisely equivalent to

    (progn (setf place1 newvalue1)
           (setf place2 newvalue2)
           ...
           (setf placen newvaluen))

For consistency, it is legal to write `(setf)`, which simply returns `nil`.

The form place may be any one of the following:

- The name of a variable (either lexical or dynamic).

- A function call form whose first element is the name of any one of the following functions:

      aref        car        svref
      nth         cdr        get
      elt         caar       getf             symbol-value
      rest        cadr       gethash          symbol-function
      first       cdar       documentation    symbol-plist
      second      cddr       fill-pointer     macro-function
      third       caaar      caaaar           cdaaar
      fourth      caadr      caaadr           cdaadr
      fifth       cadar      caadar           cdadar
      sixth       caddr      caaddr           cdaddr
      seventh     cdaar      cadaar           cddaar
      eighth      cdadr      cadadr           cddadr
      ninth       cddar      caddar           cdddar
      tenth       cdddr      cadddr           cddddr

X3J13 voted in March 1988 (AREF-1D) to add `row-major-aref` to this list.

X3J13 voted in June 1989 (DEFINE-COMPILER-MACRO) to add `compiler-macro-function` to this list.

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

- A function call form whose first element is the name of a selector function constructed by `defstruct`.

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

- A function call form whose first element is the name of any one of the following functions, provided that the new value is of the specified type so that it can be used to replace the specified ``location'' (which is in each of these cases not truly a generalized variable):

      Function Name	Required Type
      --------------------------------
      char	string-char
      schar	string-char
      bit	bit
      sbit	bit
      subseq	sequence
      --------------------------------

X3J13 voted in March 1989 (CHARACTER-PROPOSAL) to eliminate the type `string-char` and to redefine `string` to be the union of one or more specialized vector types, the types of whose elements are subtypes of the type `character`. In the preceding table, the type `string-char` should be replaced by some such phrase as ``the element-type of the argument vector.''

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

In the case of `subseq`, the replacement value must be a sequence whose elements may be contained by the sequence argument to `subseq`. (Note that this is not so stringent as to require that the replacement value be a sequence of the same type as the sequence of which the subsequence is specified.) If the length of the replacement value does not equal the length of the subsequence to be replaced, then the shorter length determines the number of elements to be stored, as for the function `replace`.

- A function call form whose first element is the name of any one of the following functions, provided that the specified argument to that function is in turn a place form; in this case the new place has stored back into it the result of applying the specified ``update'' function (which is in each of these cases not a true update function):

      Function Name   Argument That Is a place    Update Function Used
      ----------------------------------------------------------------
      char-bit        first                       set-char-bit
      ldb             second                      dpb
      mask-field      second                      deposit-field
      ----------------------------------------------------------------

X3J13 voted in March 1989 (CHARACTER-PROPOSAL) to eliminate `char-bit` and `set-char-bit`.

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name refers to a global function definition and not to a locally defined function or macro.

- A `the` type declaration form, in which case the declaration is transferred to the newvalue form, and the resulting `setf` form is analyzed. For example,

      (setf (the integer (cadr x)) (+ y 3))

  is processed as if it were

      (setf (cadr x) (the integer (+ y 3)))

- A call to `apply` where the first argument form is of the form `#'name`, that is, `(function name)`, where name is the name of a function, calls to which are recognized as places by `setf`. Suppose that the use of `setf` with `apply` looks like this:

      (setf (apply #'name x1 x2 ... xn rest) x0)

  The `setf` method for the function name must be such that

      (setf (name z1 z2 ... zm) z0)

  expands into a store form

      (storefn zi zi ... zi zm)

  That is, it must expand into a function call such that all arguments but the last may be any permutation or subset of the new value z0 and the arguments of the access form, but the last argument of the storing call must be the same as the last argument of the access call. See `define-setf-method` for more details on accessing and storing forms.

  Given this, the `setf`-of-`apply` form shown above expands into

      (apply #'storefn xi xi ... xi rest)

  As an example, suppose that the variable `indexes` contains a list of subscripts for a multidimensional array foo whose rank is not known until run time. One may access the indicated element of the array by writing

      (apply #'aref foo indexes)

  and one may alter the value of the indicated element to that of `newvalue` by writing

      (setf (apply #'aref foo indexes) newvalue)

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name `apply` refers to the global function definition and not to a locally defined function or macro named `apply`.

- A macro call, in which case `setf` expands the macro call and then analyzes the resulting form.

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this step uses `macroexpand-1`, not `macroexpand`. This allows the chance to apply any of the rules preceding this one to any of the intermediate expansions.

- Any form for which a `defsetf` or `define-setf-method` declaration has been made.

X3J13 voted in March 1989 (FUNCTION-NAME) to clarify that this rule applies only when the function name in the form refers to a global function definition and not to a locally defined function or macro.

X3J13 voted in March 1989 (FUNCTION-NAME) to add one more rule to the preceding list, coming after all those listed above:

- Any other list whose first element is a symbol (call it f). In this case, the call to `setf` expands into a call to the function named by the list `(setf f)` (see section 7.1). The first argument is the new value and the remaining arguments are the values of the remaining elements of place. This expansion occurs regardless of whether either f or `(setf f)` is defined as a function locally, globally, or not at all. For example,

      (setf (f arg1 arg2 ...) newvalue)

  expands into a form with the same effect and value as

      (let ((#:temp1 arg1)     ;Force correct order of evaluation
            (#:temp2 arg2)
            ...
            (#:temp0 newvalue))
        (funcall (function (setf f))
                 #:temp0
                 #:temp1
                 #:temp2 ...))

  By convention, any function named `(setf f)` should return its first argument as its only value, in order to preserve the specification that `setf` returns its newvalue.

X3J13 voted in March 1989 (SYMBOL-MACROLET-SEMANTICS) to add this case as well:

- A variable reference that refers to a symbol macro definition made by `symbol-macrolet`, in which case `setf` expands the reference and then analyzes the resulting form.

`setf` carefully arranges to preserve the usual left-to-right order in which the various subforms are evaluated. On the other hand, the exact expansion for any particular form is not guaranteed and may even be implementation-dependent; all that is guaranteed is that the expansion of a `setf` form will be an update form that works for that particular implementation, and that the left-to-right evaluation of subforms is preserved.

The ultimate result of evaluating a `setf` form is the value of newvalue. Therefore `(setf (car x) y)` does not expand into precisely `(rplaca x y)`, but into something more like

    (let ((G1 x) (G2 y)) (rplaca G1 G2) G2)

the precise expansion being implementation-dependent.

The user can define new `setf` expansions by using `defsetf`.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES) to extend the specification of `setf` to allow a place whose `setf` method has more than one store variable (see `define-setf-method`). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to `nil`, as is usual in situations involving multiple values.

A proposal was submitted to X3J13 in September 1989 to add a `setf` method for `values` so that one could in fact write, for example,

    (setf (values quotient remainder)
          (truncate linewidth tabstop))

but unless this proposal is accepted users will have to define a `setf` method for `values` themselves (not a difficult task).

[Macro]
`psetf` `{place` `newvalue}*`

`psetf` is like `setf` except that if more than one place-newvalue pair is specified, then the assignments of new values to places are done in parallel. More precisely, all subforms that are to be evaluated are evaluated from left to right; after all evaluations have been performed, all of the assignments are performed in an unpredictable order. (The unpredictability matters only if more than one place form refers to the same place.) `psetf` always returns `nil`.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES) to extend the specification of `psetf` to allow a place whose `setf` method has more than one store variable (see `define-setf-method`). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to `nil`, as is usual in situations involving multiple values.

[Macro]
`shiftf` `{place}+` `newvalue`

Each place form may be any form acceptable as a generalized variable to `setf`. In the form `(shiftf place1 place2 ... placen newvalue)`, the values in place1 through placen are accessed and saved, and newvalue is evaluated, for a total of n+1 values in all. Values 2 through n+1 are then stored into place1 through placen, and value 1 (the original value of place1) is returned. It is as if all the places form a shift register; the newvalue is shifted in from the right, all values shift over to the left one place, and the value shifted out of place1 is returned. For example:

    (setq x (list 'a 'b 'c)) => (a b c)

    (shiftf (cadr x) 'z) => b
       and now x => (a z c)

    (shiftf (cadr x) (cddr x) 'q) => z
       and now x => (a (c) . q)

The effect of `(shiftf place1 place2 ... placen newvalue)` is equivalent to

    (let ((var1 place1)
          (var2 place2)
          ...
          (varn placen))
      (setf place1 var2)
      (setf place2 var3)
      ...
      (setf placen newvalue)
      var1)

except that the latter would evaluate any subforms of each place twice, whereas `shiftf` takes care to evaluate them only once. For example:

    (setq n 0)
    (setq x '(a b c d))
    (shiftf (nth (setq n (+ n 1)) x) 'z) => b
       and now x => (a z c d)

    but

    (setq n 0)
    (setq x '(a b c d))
    (prog1 (nth (setq n (+ n 1)) x)
           (setf (nth (setq n (+ n 1)) x) 'z)) => b
       and now x => (a b z d)

Moreover, for certain place forms `shiftf` may be significantly more efficient than the `prog1` version.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES) to extend the specification of `shiftf` to allow a place whose `setf` method has more than one store variable (see `define-setf-method`). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to `nil`, as is usual in situations involving multiple values.

Rationale: `shiftf` and `rotatef` have been included in Common Lisp as generalizations of two-argument versions formerly called `swapf` and `exchf`. The two-argument versions have been found to be very useful, but the names were easily confused. The generalization to many argument forms and the change of names were both inspired by the work of Suzuki [47], which indicates that use of these primitives can make certain complex pointer-manipulation programs clearer and easier to prove correct.

[Macro]
`rotatef` `{place}*`

Each place form may be any form acceptable as a generalized variable to `setf`. In the form `(rotatef place1 place2 ... placen)`, the values in place1 through placen are accessed and saved. Values 2 through n and value 1 are then stored into place1 through placen. It is as if all the places form an end-around shift register that is rotated one place to the left, with the value of place1 being shifted around the end to placen. Note that `(rotatef place1 place2)` exchanges the contents of place1 and place2.

The effect of `(rotatef place1 place2 ... placen)` is roughly equivalent to

    (psetf place1 place2
           place2 place3
           ...
           placen place1)

except that the latter would evaluate any subforms of each place twice, whereas `rotatef` takes care to evaluate them only once. Moreover, for certain place forms `rotatef` may be significantly more efficient.

`rotatef` always returns `nil`.

X3J13 voted in June 1989 (SETF-MULTIPLE-STORE-VARIABLES) to extend the specification of `rotatef` to allow a place whose `setf` method has more than one store variable (see `define-setf-method`). In such a case as many values are accepted from the newvalue form as there are store variables; extra values are ignored and missing values default to `nil`, as is usual in situations involving multiple values.

Other macros that manipulate generalized variables include `getf`, `remf`, `incf`, `decf`, `push`, `pop`, `assert`, `ctypecase`, and `ccase`.

Macros that manipulate generalized variables must guarantee the ``obvious'' semantics: subforms of generalized-variable references are evaluated exactly as many times as they appear in the source program, and they are evaluated in exactly the same order as they appear in the source program.

In generalized-variable references such as `shiftf`, `incf`, `push`, and `setf` of `ldb`, the generalized variables are both read and written in the same reference. Preserving the source program order of evaluation and the number of evaluations is particularly important.

As an example of these semantic rules, in the generalized-variable reference `(setf reference value)` the value form must be evaluated after all the subforms of the reference because the value form appears to the right of them.

The expansion of these macros must consist of code that follows these rules or has the same effect as such code. This is accomplished by introducing temporary variables bound to the subforms of the reference. As an optimization in the implementation, temporary variables may be eliminated whenever it can be proved that removing them has no effect on the semantics of the program. For example, a constant need never be saved in a temporary variable. A variable, or for that matter any form that does not have side effects, need not be saved in a temporary variable if it can be proved that its value will not change within the scope of the generalized-variable reference.

Common Lisp provides built-in facilities to take care of these semantic complications and optimizations. Since the required semantics can be guaranteed by these facilities, the user does not have to worry about writing correct code for them, especially in complex cases. Even experts can become confused and make mistakes while writing this sort of code.

X3J13 voted in March 1988 (PUSH-EVALUATION-ORDER) to clarify the preceding discussion about the order of evaluation of subforms in calls to `setf` and related macros. The general intent is clear: evaluation proceeds from left to right whenever possible. However, the left-to-right rule does not remove the obligation on writers of macros and `define-setf-method` to work to ensure left-to-right order of evaluation.

Let it be emphasized that, in the following discussion, a form is something whose syntactic use is such that it will be evaluated. A subform means a form that is nested inside another form, not merely any Lisp object nested inside a form regardless of syntactic context.

The evaluation ordering of subforms within a generalized variable reference is determined by the order specified by the second value returned by `get-setf-method`. For all predefined generalized variable references (`getf`, `ldb`), this order of evaluation is exactly left-to-right. When a generalized variable reference is derived from a macro expansion, this rule is applied after the macro is expanded to find the appropriate generalized variable reference.

This is intended to make it clear that if the user writes a `defmacro` or `define-setf-method` macro that doesn't preserve left-to-right evaluation order, the order specified in the user's code holds. For example, given

    (defmacro wrong-order (x y) `(getf ,y ,x))

then

    (push value (wrong-order place1 place2))

will evaluate place2 first and then place1 because that is the order they are evaluated in the macro expansion.

For the macros that manipulate generalized variables (`push`, `pushnew`, `getf`, `remf`, `incf`, `decf`, `shiftf`, `rotatef`, `psetf`, `setf`, `pop`, and those defined with `define-modify-macro`) the subforms of the macro call are evaluated exactly once in left-to-right order, with the subforms of the generalized variable references evaluated in the order specified above.

Each of `push`, `pushnew`, `getf`, `remf`, `incf`, `decf`, `shiftf`, `rotatef`, `psetf`, and `pop` evaluates all subforms before modifying any of the generalized variable locations. Moreover, `setf` itself, in the case when a call on it has more than two arguments, performs its operation on each pair in sequence. That is, in

    (setf place1 value1 place2 value2 ...)

the subforms of place1 and value1 are evaluated, the location specified by place1 is modified to contain the value returned by value1, and then the rest of the `setf` form is processed in a like manner.

For the macros `check-type`, `ctypecase`, and `ccase`, subforms of the generalized variable reference are evaluated once per test of a generalized variable, but they may be evaluated again if the type check fails (in the case of `check-type`) or if none of the cases holds (in `ctypecase` or `ccase`).

For the macro `assert`, the order of evaluation of the generalized variable references is not specified.

Another reason for building in these functions is that the appropriate optimizations will differ from implementation to implementation. In some implementations most of the optimization is performed by the compiler, while in others a simpler compiler is used and most of the optimization is performed in the macros. The cost of binding a temporary variable relative to the cost of other Lisp operations may differ greatly between one implementation and another, and some implementations may find it best never to remove temporary variables except in the simplest cases.

A good example of the issues involved can be seen in the following generalized-variable reference:

    (incf (ldb byte-field variable))

This ought to expand into something like

    (setq variable
          (dpb (1+ (ldb byte-field variable))
               byte-field
               variable))

In this expansion example we have ignored the further complexity of returning the correct value, which is the incremented byte, not the new value of `variable`. Note that the variable `byte-field` is evaluated twice, and the variable `variable` is referred to three times: once as the location in which to store a value, and twice during the computation of that value.

Now consider this expression:

    (incf (ldb (aref byte-fields (incf i))
               (aref (determine-words-array) i)))

It ought to expand into something like this:

    (let ((temp1 (aref byte-fields (incf i)))
          (temp2 (determine-words-array)))
      (setf (aref temp2 i)
            (dpb (1+ (ldb temp1 (aref temp2 i)))
                 temp1
                 (aref temp2 i))))

Again we have ignored the complexity of returning the correct value. What is important here is that the expressions `(incf i)` and `(determine-words-array)` must not be duplicated because each may have a side effect or be affected by side effects.

X3J13 voted in January 1989 (SETF-SUB-METHODS) to specify more precisely the order of evaluation of subforms when `setf` is used with an access function that itself takes a place as an argument, for example, `ldb`, `mask-field`, and `getf`. (The vote also discussed the function `char-bit`, but another vote (CHARACTER-PROPOSAL) removed that function from the language.) The `setf` methods for such accessors produce expansions that effectively require explicit calls to `get-setf-method`.

The code produced as the macro expansion of a `setf` form that itself admits a generalized variable as an argument must essentially do the following major steps:

- It evaluates the value-producing subforms, in left-to-right order, and binds the temporary variables to them; this is called binding the temporaries.

- It reads the value from the generalized variable, using the supplied accessing form, to get the old value; this is called doing the access. Note that this is done after all the evaluations of the preceding step, including any side effects they may have.

- It binds the store variable to a new value, and then installs this new value into the generalized variable using the supplied storing form; this is called doing the store.

Doing the access for a generalized variable reference is not part of the series of evaluations that must be done in left-to-right order.

The place-specifier forms `ldb`, `mask-field`, and `getf` admit (other) place specifiers as arguments. During the `setf` expansion of these forms, it is necessary to call `get-setf-method` to determine how the inner, nested generalized variable must be treated.

In a form such as

    (setf (ldb byte-spec place-form) newvalue-form)

the place referred to by the place-form must always be both accessed and updated; note that the update is to the generalized variable specified by place-form, not to any object of type `integer`.

Thus this call to `setf` should generate code to do the following:

- Evaluate byte-spec and bind into a temporary
- Bind the temporaries for place-form
- Evaluate newvalue-form and bind into the store variable
- Do the access to place-form
- Do the store into place-form with the given bit-field of the accessed integer replaced with the value in the store variable

If the evaluation of newvalue-form alters what is found in the given place-such as setting a different bit-field of the integer-then the change of the bit-field denoted by byte-spec will be to that altered integer, because the access step must be done after the newvalue-form evaluation. Nevertheless, the evaluations required for binding the temporaries are done before the evaluation of the newvalue-form, thereby preserving the required left-to-right evaluation order.

The treatment of `mask-field` is similar to that of `ldb`.

In a form such as:

    (setf (getf place-form ind-form) newvalue-form)

the place referred to by the place-form must always be both accessed and updated; note that the update is to the generalized variable specified by place-form, not necessarily to the particular list which is the property list in question.

Thus this call to `setf` should generate code to do the following:

- Bind the temporaries for place-form
- Evaluate ind-form and bind into a temporary
- Evaluate the newvalue-form and bind into the store variable
- Do the access to place-form
- Do the store into place-form with a possibly new property list obtained by combining the results of the evaluations and the access

If the evaluation of newvalue-form alters what is found in the given place-such as setting a different named property in the list-then the change of the property denoted by ind-form will be to that altered list, because the access step is done after the newvalue-form evaluation. Nevertheless, the evaluations required for binding the temporaries are done before the evaluation of the newvalue-form, thereby preserving the required left-to-right evaluation order.

Note that the phrase ``possibly new property list'' treats the implementation of property lists as a ``black box''; it can mean that the former property list is somehow destructively re-used, or it can mean partial or full copying of it. A side effect may or may not occur; therefore `setf` must proceed as if the resultant property list were a different copy needing to be stored back into the generalized variable.

The Common Lisp facilities provided to deal with these semantic issues include:

- Built-in macros such as `setf` and `push` that follow the semantic rules.

- The `define-modify-macro` macro, which allows new generalized-variable manipulating macros (of a certain restricted kind) to be defined easily. It takes care of the semantic rules automatically.

- The `defsetf` macro, which allows new types of generalized-variable references to be defined easily. It takes care of the semantic rules automatically.

- The `define-setf-method` macro and the `get-setf-method` function, which provide access to the internal mechanisms when it is necessary to define a complicated new type of generalized-variable reference or generalized-variable-manipulating macro.

Also important are the changes that allow lexical environments to be used in appropriate ways in `setf` methods.

[Macro]
`define-modify-macro` `name lambda-list function
[doc-string]`

This macro defines a read-modify-write macro named name. An example of such a macro is `incf`. The first subform of the macro will be a generalized-variable reference. The function is literally the function to apply to the old contents of the generalized-variable to get the new contents; it is not evaluated. lambda-list describes the remaining arguments for the function; these arguments come from the remaining subforms of the macro after the generalized-variable reference. lambda-list may contain `&optional` and `&rest` markers. (The `&key` marker is not permitted here; `&rest` suffices for the purposes of `define-modify-macro`.) doc-string is documentation for the macro name being defined.

The expansion of a `define-modify-macro` is equivalent to the following, except that it generates code that follows the semantic rules outlined above.

    (defmacro name (reference . lambda-list)
      doc-string
      `(setf ,reference
             (function ,reference ,arg1 ,arg2 ...)))

where arg1, arg2, ..., are the parameters appearing in lambda-list; appropriate provision is made for a `&rest` parameter.

As an example, `incf` could have been defined by:

    (define-modify-macro incf (&optional (delta 1)) +)

An example of a possibly useful macro not predefined in Common Lisp is

    (define-modify-macro unionf (other-set &rest keywords) union)

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT) to specify that `define-modify-macro` creates macros that take `&environment` arguments and perform the equivalent of correctly passing such lexical environments to `get-setf-method` in order to correctly maintain lexical references.

[Macro]

    defsetf access-fn {update-fn [doc-string] |

                       lambda-list (store-variable)

                       [[{declaration}* | doc-string]] {form}*}

This defines how to `setf` a generalized-variable reference of the form
`(access-fn ...)`. The value of a generalized-variable reference can always be obtained simply by evaluating it, so access-fn should be the name of a function or a macro.

The user of `defsetf` provides a description of how to store into the generalized-variable reference and return the value that was stored (because `setf` is defined to return this value). The implementation of `defsetf` takes care of ensuring that subforms of the reference are evaluated exactly once and in the proper left-to-right order. In order to do this, `defsetf` requires that access-fn be a function or a macro that evaluates its arguments, behaving like a function. Furthermore, a `setf` of a call on access-fn will also evaluate all of access-fn's arguments; it cannot treat any of them specially. This means that `defsetf` cannot be used to describe how to store into a generalized variable that is a byte, such as `(ldb field reference)`. To handle situations that do not fit the restrictions imposed by `defsetf`, use `define-setf-method`, which gives the user additional control at the cost of increased complexity.

A `defsetf` declaration may take one of two forms. The simple form is

    (defsetf access-fn update-fn doc-string)

The update-fn must name a function (or macro) that takes one more argument than access-fn takes. When `setf` is given a place that is a call on access-fn, it expands into a call on update-fn that is given all the arguments to access-fn and also, as its last argument, the new value (which must be returned by update-fn as its value). For example, the effect of

    (defsetf symbol-value set)

is built into the Common Lisp system. This causes the expansion

    (setf (symbol-value foo) fu) -> (set foo fu)

for example. Note that

    (defsetf car rplaca)

would be incorrect because `rplaca` does not return its last argument.

The complex form of `defsetf` looks like

    (defsetf access-fn lambda-list (store-variable) . body)

and resembles `defmacro`. The body must compute the expansion of a `setf` of a call on access-fn.

The lambda-list describes the arguments of access-fn. `&optional`, `&rest`, and `&key` markers are permitted in lambda-list. Optional arguments may have defaults and ``supplied-p'' flags. The store-variable describes the value to be stored into the generalized-variable reference.

Rationale: The store-variable is enclosed in parentheses to provide for an extension to multiple store variables that would receive multiple values from the second subform of `setf`. The rules given below for coding `setf` methods discuss the proper handling of multiple store variables to allow for the possibility that this extension may be incorporated into Common Lisp in the future.

The body forms can be written as if the variables in the lambda-list were bound to subforms of the call on access-fn and the store-variable were bound to the second subform of `setf`. However, this is not actually the case. During the evaluation of the body forms, these variables are bound to names of temporary variables, generated as if by `gensym` or `gentemp`, that will be bound by the expansion of `setf` to the values of those subforms. This binding permits the body forms to be written without regard for order-of-evaluation issues. `defsetf` arranges for the temporary variables to be optimized out of the final result in cases where that is possible. In other words, an attempt is made by `defsetf` to generate the best code possible in a particular implementation.

Note that the code generated by the body forms must include provision for returning the correct value (the value of store-variable). This is handled by the body forms rather than by `defsetf` because in many cases this value can be returned at no extra cost, by calling a function that simultaneously stores into the generalized variable and returns the correct value.

An example of the use of the complex form of `defsetf`:

    (defsetf subseq (sequence start &optional end) (new-sequence)
      `(progn (replace ,sequence ,new-sequence
                       :start1 ,start :end1 ,end)
              ,new-sequence))

X3J13 voted in March 1988 (FLET-IMPLICIT-BLOCK) to specify that the body of the expander function defined by the complex form of `defsetf` is implicitly enclosed in a `block` construct whose name is the same as the name of the access-fn. Therefore `return-from` may be used to exit from the function.

X3J13 voted in March 1989 (DEFINING-MACROS-NON-TOP-LEVEL) to clarify that, while defining forms normally appear at top level, it is meaningful to place them in non-top-level contexts; the complex form of `defsetf` must define the expander function within the enclosing lexical environment, not within the global environment.

The underlying theory by which `setf` and related macros arrange to conform to the semantic rules given above is that from any generalized-variable reference one may derive its ```setf` method,'' which describes how to store into that reference and which subforms of it are evaluated.

Compatibility note: To avoid confusion, it should be noted that the use of the word ``method'' here in connection with `setf` has nothing to do with its use in Lisp Machine Lisp in connection with message-passing and the Lisp Machine Lisp ``flavor system.''
And of course it also has nothing to do with the methods in the Common Lisp Object System (CLOS) .

Given knowledge of the subforms of the reference, it is possible to avoid evaluating them multiple times or in the wrong order. A `setf` method for a given access form can be expressed as five values:

- A list of temporary variables

- A list of value forms (subforms of the given form) to whose values the temporary variables are to be bound

- A second list of temporary variables, called store variables

- A storing form

- An accessing form

The temporary variables will be bound to the values of the value forms as if by `let*`; that is, the value forms will be evaluated in the order given and may refer to the values of earlier value forms by using the corresponding variables.

The store variables are to be bound to the values of the newvalue form, that is, the values to be stored into the generalized variable. In almost all cases only a single value is to be stored, and there is only one store variable.

The storing form and the accessing form may contain references to the temporary variables (and also, in the case of the storing form, to the store variables). The accessing form returns the value of the generalized variable. The storing form modifies the value of the generalized variable and guarantees to return the values of the store variables as its values; these are the correct values for `setf` to return. (Again, in most cases there is a single store variable and thus a single value to be returned.) The value returned by the accessing form is, of course, affected by execution of the storing form, but either of these forms may be evaluated any number of times and therefore should be free of side effects (other than the storing action of the storing form).

The temporary variables and the store variables are generated names, as if by `gensym` or `gentemp`, so that there is never any problem of name clashes among them, or between them and other variables in the program. This is necessary to make the special forms that do more than one `setf` in parallel work properly; these are `psetf`, `shiftf`, and `rotatef`. Computation of the `setf` method must always create new variable names; it may not return the same ones every time.

Some examples of `setf` methods for particular forms:

- For a variable `x`:

      ()
      ()
      (g0001)
      (setq x g0001)
      x

- For `(car exp)`:

      (g0002)
      (exp)
      (g0003)
      (progn (rplaca g0002 g0003) g0003)
      (car g0002)

- For `(subseq seq s e)`:

      (g0004 g0005 g0006)
      (seq s e)
      (g0007)
      (progn (replace g0004 g0007 :start1 g0005 :end1 g0006)
             g0007)
      (subseq g0004 g0005 g0006)

[Macro]

    define-setf-method access-fn lambda-list
                       [[ {declaration}* | doc-string ]]  {form}*

This defines how to `setf` a generalized-variable reference that is of the form `(access-fn...)`. The value of a generalized-variable reference can always be obtained simply by evaluating it, so access-fn should be the name of a function or a macro.

The lambda-list describes the subforms of the generalized-variable reference, as with `defmacro`. The result of evaluating the forms in the body must be five values representing the `setf` method, as described above. Note that `define-setf-method` differs from the complex form of `defsetf` in that while the body is being executed the variables in lambda-list are bound to parts of the generalized-variable reference, not to temporary variables that will be bound to the values of such parts. In addition, `define-setf-method` does not have `defsetf`'s restriction that access-fn must be a function or a function-like macro; an arbitrary `defmacro` destructuring pattern is permitted in lambda-list.

By definition there are no good small examples of `define-setf-method` because the easy cases can all be handled by `defsetf`. A typical use is to define the `setf` method for `ldb`:

    ;;; SETF method for the form (LDB bytespec int).
    ;;; Recall that the int form must itself be suitable for SETF.
    (define-setf-method ldb (bytespec int)
      (multiple-value-bind (temps vals stores
                            store-form access-form)
          (get-setf-method int)         ;Get SETF method for int
        (let ((btemp (gensym))          ;Temp var for byte specifier
              (store (gensym))          ;Temp var for byte to store
              (stemp (first stores)))   ;Temp var for int to store
          ;; Return the SETF method for LDB as five values.
          (values (cons btemp temps)    ;Temporary variables
                  (cons bytespec vals)  ;Value forms
                  (list store)          ;Store variables
                  `(let ((,stemp (dpb ,store ,btemp ,access-form)))
                     ,store-form
                     ,store)                     ;Storing form
                  `(ldb ,btemp ,access-form)     ;Accessing form
                  ))))

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT) to specify that the `&environment` lambda-list keyword may appear in the lambda-list in the same manner as for `defmacro` in order to obtain the lexical environment of the call to the `setf` macro. The preceding example should be modified to take advantage of this new feature. The `setf` method must accept an `&environment` parameter, which will receive the lexical environment of the call to `setf`; this environment must then be given to `get-setf-method` in order that it may correctly use any locally bound `setf` method that might be applicable to the place form that appears as the second argument to `ldb` in the call to `setf`.

    ;;; SETF method for the form (LDB bytespec int).
    ;;; Recall that the int form must itself be suitable for SETF.
    ;;; Note the use of an &environment parameter to receive the
    ;;; lexical environment of the call for use with GET-SETF-METHOD.
    (define-setf-method ldb (bytespec int &environment env)
      (multiple-value-bind (temps vals stores
                            store-form access-form)
          (get-setf-method int env)     ;Get SETF method for int
        (let ((btemp (gensym))          ;Temp var for byte specifier
              (store (gensym))          ;Temp var for byte to store
              (stemp (first stores)))   ;Temp var for int to store
          ;; Return the SETF method for LDB as five values.
          (values (cons btemp temps)    ;Temporary variables
                  (cons bytespec vals)  ;Value forms
                  (list store)          ;Store variables
                  `(let ((,stemp (dpb ,store ,btemp ,access-form)))
                     ,store-form
                     ,store)                     ;Storing form
                  `(ldb ,btemp ,access-form)     ;Accessing form
                  ))))

X3J13 voted in March 1988 (FLET-IMPLICIT-BLOCK) to specify that the body of the expander function defined by `define-setf-method` is implicitly enclosed in a `block` construct whose name is the same as the name of the access-fn. Therefore `return-from` may be used to exit from the function.

X3J13 voted in March 1989 (DEFINING-MACROS-NON-TOP-LEVEL) to clarify that, while defining forms normally appear at top level, it is meaningful to place them in non-top-level contexts; `define-setf-method` must define the expander function within the enclosing lexical environment, not within the global environment.

[Function]
`get-setf-method` `form`

`get-setf-method` returns five values constituting the `setf` method for form. The form must be a generalized-variable reference. `get-setf-method` takes care of error-checking and macro expansion and guarantees to return exactly one store variable.

As an example, an extremely simplified version of `setf`, allowing no more and no fewer than two subforms, containing no optimization to remove unnecessary variables, and not allowing storing of multiple values, could be defined by:

    (defmacro setf (reference value)
      (multiple-value-bind (vars vals stores store-form access-form)
          (get-setf-method reference)
        (declare (ignore access-form))
        `(let* ,(mapcar #'list
                        (append vars stores)
                        (append vals (list value)))
           ,store-form)))

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT) to add an optional environment argument to `get-setf-method`. The revised definition and example are as follows.

[Function]
`get-setf-method` `form` `&optional` `env`

`get-setf-method` returns five values constituting the `setf` method for form. The form must be a generalized-variable reference. The env must be an environment of the sort obtained through the `&environment` lambda-list keyword; if env is `nil` or omitted, the null lexical environment is assumed. `get-setf-method` takes care of error checking and macro expansion and guarantees to return exactly one store variable.

As an example, an extremely simplified version of `setf`, allowing no more and no fewer than two subforms, containing no optimization to remove unnecessary variables, and not allowing storing of multiple values, could be defined by:

    (defmacro setf (reference value &environment env)
      (multiple-value-bind (vars vals stores store-form access-form)
          (get-setf-method reference env)     ;Note use of environment
        (declare (ignore access-form))
        `(let* ,(mapcar #'list
                        (append vars stores)
                        (append vals (list value)))
           ,store-form)))

[Function]
`get-setf-method-multiple-value` `form`

`get-setf-method-multiple-value` returns five values constituting the `setf` method for form. The form must be a generalized-variable reference. This is the same as `get-setf-method` except that it does not check the number of store variables; use this in cases that allow storing multiple values into a generalized variable. There are no such cases in standard Common Lisp, but this function is provided to allow for possible extensions.

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT) to add an optional environment argument to `get-setf-method`. The revised definition is as follows.

[Function]
`get-setf-method-multiple-value` `form` `&optional` `env`

`get-setf-method-multiple-value` returns five values constituting the `setf` method for form. The form must be a generalized-variable reference. The env must be an environment of the sort obtained through the `&environment` lambda-list keyword; if env is `nil` or omitted, the null lexical environment is assumed.

This is the same as `get-setf-method` except that it does not check the number of store variables; use this in cases that allow storing multiple values into a generalized variable. There are no such cases in standard Common Lisp, but this function is provided to allow for possible extensions.

X3J13 voted in March 1988 (GET-SETF-METHOD-ENVIRONMENT) to clarify that a `setf` method for a functional name is applicable only when the global binding of that name is lexically visible. If such a name has a local binding introduced by `flet`, `labels`, or `macrolet`, then global definitions of `setf` methods for that name do not apply and are not visible. All of the standard Common Lisp macros that modify a `setf` place (for example, `incf`, `decf`, `pop`, and `rotatef`) obey this convention.

Next: Function Invocation Up: Control Structure Previous: Assignment