Sunday, July 7, 2013

To clabango or not to clabango

Pedestal's lack of documentation and confusing client/server development model caused me to switch to luminus, which comes bundled with clabango. It's been a while since I've done template based page generation, so maybe this is par for the course. So far clabango has left me a bit underwhelmed. It might prove to be sufficient for my needs, however, so I'm trying to find ways to work with the package.

The first thing that struck me is its utter un-lisp-ness. That seems to be a positive for some. More troubling to me is how clabango seems to encourage breaking data abstractions. I have a protocol Identifier that I would like to render in a page. I have two options when I do so: pull out the value from the record implementing the protocol ahead of time, or access the record's raw fields through the map API. There doesn't appear to be a reasonable way to render through the protocol methods on the page.

;; Define an identifier protocol and a widget-id record
user=> (defprotocol identifier
         (value [this]))
user=> (defrecord widget-id [id-value]
         (value [id]
           (.id-value id)))
user=> (require 'clabango.parser)
user=> (alias 'parser 'clabango.parser)
;; This is obviously wrong user=> (parser/render "The ID is {{ id }}." {:id (->widget-id "abc")}) "The ID is user.widget-id@caada592."
;; Can't use the protocol abstraction user=> (parser/render "The ID is {{ id.value }}." {:id (->widget-id "abc")}) "The ID is ."
;; Works if we don't use the abstraction,
;; which defeats the purpose of having one at all
user=> (parser/render "The ID is {{ }}." {:id (->widget-id "abc")}) "The ID is abc."
;; Works if we pass in the raw value,
;; which doesn't propagate the abstraction through the whole program.
;; Once again it defeats the purpose of the abstraction.
user=> (parser/render "The ID is {{ id-value }}." {:id-value (value (->widget-id "abc"))}) "The ID is abc."

My initial reaction was that clabango should just allow arbitrary clojure code in its templates. This is a reasonable strategy if the templates are all part of the application. I can't think of why you'd want to have the templates be user defined anyway, and the code generation abilities of Lisp-like languages will make turning templates into compiled code fairly simple. But clabango is what it is, and such a radical departure would yield something other than clabango.

So instead I implemented a simple filter that executes a one argument function:

(require 'clabango.filters)
(alias 'filters 'clabango.filters)

(require '
(alias 'edn '

(defn dequote [quoted-string]
  (if (and (> (count quoted-string) 2)
           (= (first quoted-string) \")
           (= (last quoted-string) \"))
    (.substring quoted-string 1 (- (count quoted-string) 1))
    (throw (IllegalArgumentException. ""))))

(defn find-function [ns-function-str]
  (when (string? ns-function-str)
      @(resolve (edn/read-string (dequote ns-function-str)))
      (catch Exception e

(filters/deftemplatefilter "apply" [node body arg]
  (when body
    (when-let [function (find-function arg)]
      (function body))))

Which allows me to render an ID thus:

user> (parser/render "The ID is {{ id|apply:\"user/value\" }}."
                     {:id (->widget-id "abc")})
"The ID is abc."

I am deeply disappointed with clabango. My initial hesitation was that moving from clabango to something else would effectively mean tracking changes in luminus on an ongoing basis, which is too high an overhead for a one-person hobby project. But luminus doesn't seem to have any libraries of its own, just some templates that make starting a project straightforward.

Whatever I choose to do, I will write up in a follow-on blog post.

Saturday, July 6, 2013

Clojure: Symbols, Reading, and Execution

I wanted to read a function from a string, and apply an argument to it. This seemingly simple task is surprisingly  tricky in Lisp-like languages. The basic steps are straightforward: read the contents of a string, parse them as a function name, and apply the corresponding function to the desired arguments. Lisp-based languages are homoiconic, and also tend to have support for read-time evaluation. The combination makes safe parsing of data structures both powerful and dangerous.

I tried to rely on my prior Common Lisp knowledge, but that turned out to be information I needed to unlearn. There might still be errors in my understanding. And, while my writeup is accurate as of Clojure 1.5.1, the language is still fast evolving. Take the information here with a grain of salt.

Of Symbols and Namespaces

While reading, one has to deal with symbols, as they are invariably part of the input stream. In Common Lisp (CL) a symbol always belongs to a package, except when it doesn't. Every package symbol is automatically interned. So two symbols with the same name are always identical. The exception is that you can create uninterned symbols that don't belong to any package. And these symbols aren't equal to each other even when they have the same name.

;; By default intern into the current package
? (intern "FOO")
;; A quoted symbol is interned
? (eq (intern "FOO") 'foo)
;; Two quoted symbols are the same
? (eq 'foo 'foo)
;; This is an uninterned symbol
? '#:foo
;; Two uninterned symbols aren't the same
? (eq '#:foo '#:foo)
;; This is how you create a new uninterned symbol
? (make-symbol "FOO")
;; Two created uninterned symbols still aren't the same
? (eq (make-symbol "FOO") (make-symbol "FOO"))
;; But two unintnerned symbols are, as expected, identical
? (let ((uninterned '#:foo)) (eq uninterned uninterned))

Clojure is different. Instead of packages we have namespaces, and quoted symbols aren't automatically interned into namespaces. You can ask ns-interns for a map of all the interned symbols. Quoting will give the uninterned symbol, even if there's an interned symbol with the same name. There's some more information and examples in this google groups post. The CL transcript above can be approximated in clojure thus:

;; By default intern into the current package
user> (intern 'user 'abc)
;; Hash-quoted returns an interned symbol, but only if previously interned
user> #'foo
CompilerException java.lang.RuntimeException: Unable to resolve var: foo in this context, compiling:(NO_SOURCE_PATH:1:602) 
user> #'abc
;; Two hash-quoted symbols are the same
user> (identical? #'abc #'abc)
;; This is an uninterned symbol
user> 'abc
;; Two uninterned symbols aren't the same
user> (identical? 'abc 'abc)
;; This is how you create a new uninterned symbol
user> (symbol "abc")
;; Two created uninterned symbols still aren't the same
user> (identical? (symbol "abc") (symbol "abc"))
;; But two unintnerned symbols are, as expected, identical
user> (let [uninterned (symbol "abc")] (identical? uninterned uninterned))

The keyword namespace in clojure is different, and symbols in that namespace have a distinct appearance. This is also the case in CL, and clojure's behavior more closely resembles that of CL.

  1. All keyword symbols begin with a :.
  2. Keywords are automatically interned. One needn't explicitly create an interned keyword.
  3. A corollary is that two keywords with the same printed representation will always be identical.

Keywords have a special role in clojure, in their ability to access content in data structures. These properties are essential in enabling that role.

One of the read-time pitfalls possible in CL is interning a very large number of symbols into a package, which is effectively a memory leak. Symbols are garbage collected only when they're uninterned. And the automatic interning of symbols is a subtle bug that can creep into many programs. This concern isn't realized in clojure for two reasons:

  1. Symbols aren't automatically interned, except keywords.
  2. Interned symbols are based on interned strings, so can be garbage collected when there are no references to them.

The Pitfalls of read-string

Another type of problem occurs during read-time evaluation. The reader can execute code as its reading an s-expression from a stream. Here's a far better writeup on the matter than I could have done.

The Solution?

Add to your project.clj. Then the following works:

user=> (require '
user=> ( "clojure.core/list")
user=> (resolve ( "clojure.core/list"))
user=> @(resolve ( "clojure.core/list"))
#< clojure.lang.PersistentList$1@37f6b4df>
user=> (@(resolve ( "clojure.core/list")) 1 2)
(1 2)