Thursday, June 6, 2013

On the misuse of dynamic extent

I've been away from the Lisp family of languages for a while, and am in the process of getting myself re-acquainted. I've started playing with Clojure again. And while there are lots of cool ideas I've found in the Clojure camp, today I found an outright error. The problem with this article is that it sets up a poor straw-man use of dynamic extent, and then trashes it. I have never seen a (modern) Common Lisp program written in this manner, and I hope this isn't common practice with Clojure.

To re-iterate, here's the bad form:

1
2
3
4
5
(defmacro with-resource [src & body]
  `(binding [*resource* (acquire src)]
     (try ~@body
       (finally
         (dispose *resource*)))))

There are two problems with this code:
  1. It fixes the variable that will be bound.
  2. It assumes that dynamic extent is the right manner for binding this variable.
In Common Lisp, this code would typically be written something like this, and makes neither assumption:

1
2
3
4
(defmacro with-resource ((var src-form) &body body)
  `(let ((,var (acquire ,src-form)))
    (unwind-protect (progn ,@body)
      (dispose ,var))))

The key difference here is in the argument list. Good programming practice dictates that a with- macro in Common Lisp should always take the name of the variable that it's going to bind. Also relevant is that Common Lisp allows binding both lexical and dynamic scope variables using let. So the same macro can be used for both effects. The following two uses of with-resource are both legal:

1
2
3
4
5
6
7
(defvar *dynamic*)

(with-resource (*dynamic* <some-src>)
  <do-something>)

(with-resource (lexical <some-src>)
  <do-something>)

Finally, Common Lisp also allows you to introduce a new lexical binding using a let form:

1
2
3
(let ((*new-dynamic* ...))
  (declare (dynamic-extent *new-dynamic*))
  ...)

If with-resource were a library function, you would almost certainly want to analyze the body argument for declarations, and re-arrange the macro-expansion accordingly. I'll leave that as an exercise for the reader.

But Common Lisp isn't Clojure. I don't know yet if this translates to Clojure well, as Clojure has distinct let and binding forms for lexical and dynamic variables. If Clojure has sufficient introspection to distinguish a lexical from a dynamic variable, a macro could choose the appropriate form, and produce an effect similar to Common Lisp. If not, I hope such introspection is added in due course.

Now, this is no panacea for the other problems the author had pointed out. You would never want to use a stack bound (or thread local) value for communicating state between threads. I would argue however that each language tends to have gotchas that programmers in that language must learn. In C/C++ one has to learn a lot about safe use of pointers. In garbage collected languages programmers must know something about how the particular garbage collector works in order to write performant, and even safe, code. In languages that provide dynamic scoping, an incredibly powerful feature that is open to misuse, one has to understand how that feature is implemented. This is emphatically not a reason to take away a language feature, as the article's author has suggested.