Sunday, June 30, 2013

Changing namespace aliases in Clojure

For unit testing my clojure code, I decided to try a little trick which I wasn't sure would work. I have a model namespace that defines the application state, which could be a rather large object that's computationally expensive to create and maintain. Rather than use with-redefs everywhere, I attempted to mock out the entire model namespace:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
(ns myapp.routes.home-test
  (:require [clojure.test :refer :all]
            [myapp.routes.home :as routes]))

(ns myapp.routes.home-test.model-fake)

(def ^{:dynamic true :doc "Count of the number of ID generation calls"}
  *ids-generated* nil)

(defmacro with-test-context [& body]
  `(binding [*ids-generated* 0]
     ~@body))

(defn generate-id []
  (var-set *ids-generated* (+ *ids-generated* 1)))

(ns myapp.routes.home-test)

(alias 'fake 'myapp.routes.home-test.model-fake)

(use-fixtures :once
  (fn [f]
    (ns myapp.routes.home)
    (ns-unalias 'myapp.routes.home 'model)
    (alias 'model 'myapp.routes.home-test.model-fake)
    (try (f)
         (finally
           (ns myapp.routes.home)
           (ns-unalias 'myapp.routes.home 'model)
           (alias 'model 'myapp.model)))))

The idea was pretty straightforward:

  1. The source file name dictates the primary namespace, so the first form declares the namespace of our test cases.
  2. Next, we define a namespace that will serve as the model fake. We define a macro that defines a standardized test context, and then a set of functions that will be used to mock out the real namespace functions.
  3. Now, back in the main namespace, we create a fixture that switches to the namespace being tested, renames the model alias to point to the fake, runs the tests, and then switches the alias back to the correct namespace.

Sounds great in theory, but (fortunately) this doesn't work. I have verified that the namespace is indeed switched. However, clojure seems to compile in the namespace resolved symbol at the time of compilation. So while with-redefs works, switching namespace reference isn't sufficient.

Tuesday, June 11, 2013

Point Reyes, with the California Academy of Sciences

In late April I had visited Point Reyes on a trip organized through the California Academy of Sciences. It was a photography focused trip, covering key aspects of conservation efforts in Marin and the Point Reyes seashore .As such, I got a lot more out of it than I had expected. One of the big bonuses was Calumet helping organize the trip. They brought along cameras and glass. I had a 70-200 F2.8 LII IS lens, with a 2x extender, without which I wouldn't have gotten very many decent photographs on this trip. I have been on a longer trip to the Eastern Sierras through the Cal Academy in the past, this trip was every bit as good. See the whole album here.

Even with a 35mm equivalent focal length of 640mm (200mm with a 2x extender on a sensor with 1.6 crop factor) some of the situations were extremely difficult to photograph. We visited an egret nesting site, and the viewing platform was, photographically speaking, far from ideal. The day was very bright, and the lighting harsh, necessitating quite a bit of post-processing to reduce the contrast. I didn't realize it at the time, but the 2x extender is as bad as all the reviews claim. It added a lot of grain to the photographs. I would probably have been better off working with a 1.4x extender.
It was spring. There were a lot of flowers. I thought we were doing well before we got to Point Reyes.
Point Reyes, well, was blooming. You couldn't point your camera many places without flowers getting in the frame. Which is great. I like photographing flowers.
One of the stops we made was at a farm in the Point Reyes area. We learned a lot about the history of Point Reyes, and the role ranchers played in its development. All the farms around Point Reyes are small family operations. Ranchers initially had no desire to have the NPS as a neighbor, and fought the establishment of Point Reyes. That no longer seems to be the case. The farms are all on the northern end, the southern end is now relatively untouched.
Some more flowers.
There are tule elk at Point Reyes. They were keeping their distance that day. This herd was all female. The males stay away except around mating season. The elk have no natural predators, and introducing wolves or grizzlies wasn't really an option. Population control is through intravenous contraception, administered through a dart gun.
One last flower. This was with my 100mm macro, rather than the 70-200 with extender. Especially when zoomed in, this looks a lot more pleasing. No grain, not quite as much contrast, and really nice and smooth bokeh. I could have spent the entire day there. But there wasn't much time left, and I didn't get any really good shots with the macro.

Sunday, June 9, 2013

Getting Started with Pedestal

I want to write a web app in clojure. I'll begin with picking a framework. I had played a little with Noir the last time I had picked up clojure. And now it has morphed into lib-noir. Other than that, there are also Luminus and Joodo. I picked Pedestal though, which caught my attention for its unique approach to clients. Pedestal seems to be a relatively immature framework, which also means I need to spend a lot more time and effort figuring out how to get going with it.

Implementing a service is easy enough, and well enough documented on the pedestal site.

mkdir ~/tmp
cd ~/tmp
lein new pedestal-service helloworld
cd helloworld

The documentation however doesn't cover how to get going with the client. I had to peek into clojars to get the equivalent:

mkdir ~/tmp
cd ~/tmp
lein new pedestal-app helloworld-app
cd helloworld

That this basic bit isn't documented should give you a sense for how far the documentation still has to go. The pedestal samples aren't very well documented, so it is rather hard to get a sense for how they're working. Here's what I have been able to figure out, in no particular order:
  • Like most web frameworks, the pedestal service receives a request, and produces a response to it, using an appropriate registered handler.
  • Unlike most web frameworks, a thread isn't tied to a request (and vice versa), which makes pedestal potentially much more flexible and efficient.
  • The client specification is written in the form of a dataflow, where an event is translated to an application state change, which in turn triggers a UI change.
  • The entire application state is captured by one (hidden) state object.
  • A client application created as above will give you a simple dev environment. You can easily interact with this dev environment. This is partially documented in the README for the helloworld sample. More documentation is available once you start the dev environment for a new project and go to http://localhost:3000/index.html
  • There doesn't seem to be a simple example on having a pedestal client communicate with a pedestal service. The chat client/server example is fairly complex, and will take a little time to boil down to the minimal messaging you can do between client and server.
  • Client and server applications are implemented and executed entirely independently of one another. For instance the chat sample requires you to link (or copy) the build result of the chat client into the server's resources.
  • I don't see why pedestal separates client and server lein templates. The client template after all includes some JVM-targeted clojure (as opposed to ClojureScript) to run a dev server. Why not have that dev server be a pedestal instance? This might help a great deal in getting started with building a coherent client/server application.
  • I don't know browser javascript APIs that well. And pedestal isn't going to help me with that lack of knowledge. This is entirely a personal problem that I didn't really expect pedestal to help with.
Given all these shortcomings, perhaps I shouldn't bother with pedestal. But I'm doing a hobby project to learn clojure better. And if the learning also brings along a different paradigm for thinking about web application programming, all the better. Pedestal to me is an experiment that I feel I would do well to learn from.

Thursday, June 6, 2013

On the misuse of dynamic extent

I've been away from the Lisp family of languages for a while, and am in the process of getting myself re-acquainted. I've started playing with Clojure again. And while there are lots of cool ideas I've found in the Clojure camp, today I found an outright error. The problem with this article is that it sets up a poor straw-man use of dynamic extent, and then trashes it. I have never seen a (modern) Common Lisp program written in this manner, and I hope this isn't common practice with Clojure.

To re-iterate, here's the bad form:

1
2
3
4
5
(defmacro with-resource [src & body]
  `(binding [*resource* (acquire src)]
     (try ~@body
       (finally
         (dispose *resource*)))))

There are two problems with this code:
  1. It fixes the variable that will be bound.
  2. It assumes that dynamic extent is the right manner for binding this variable.
In Common Lisp, this code would typically be written something like this, and makes neither assumption:

1
2
3
4
(defmacro with-resource ((var src-form) &body body)
  `(let ((,var (acquire ,src-form)))
    (unwind-protect (progn ,@body)
      (dispose ,var))))

The key difference here is in the argument list. Good programming practice dictates that a with- macro in Common Lisp should always take the name of the variable that it's going to bind. Also relevant is that Common Lisp allows binding both lexical and dynamic scope variables using let. So the same macro can be used for both effects. The following two uses of with-resource are both legal:

1
2
3
4
5
6
7
(defvar *dynamic*)

(with-resource (*dynamic* <some-src>)
  <do-something>)

(with-resource (lexical <some-src>)
  <do-something>)

Finally, Common Lisp also allows you to introduce a new lexical binding using a let form:

1
2
3
(let ((*new-dynamic* ...))
  (declare (dynamic-extent *new-dynamic*))
  ...)

If with-resource were a library function, you would almost certainly want to analyze the body argument for declarations, and re-arrange the macro-expansion accordingly. I'll leave that as an exercise for the reader.

But Common Lisp isn't Clojure. I don't know yet if this translates to Clojure well, as Clojure has distinct let and binding forms for lexical and dynamic variables. If Clojure has sufficient introspection to distinguish a lexical from a dynamic variable, a macro could choose the appropriate form, and produce an effect similar to Common Lisp. If not, I hope such introspection is added in due course.

Now, this is no panacea for the other problems the author had pointed out. You would never want to use a stack bound (or thread local) value for communicating state between threads. I would argue however that each language tends to have gotchas that programmers in that language must learn. In C/C++ one has to learn a lot about safe use of pointers. In garbage collected languages programmers must know something about how the particular garbage collector works in order to write performant, and even safe, code. In languages that provide dynamic scoping, an incredibly powerful feature that is open to misuse, one has to understand how that feature is implemented. This is emphatically not a reason to take away a language feature, as the article's author has suggested.