A Year Of Clojure

Bloody hell, more than a year evaporated.

What happened in that time? Mostly, a lot of Clojure, at least with respect to home stuff. It finally clicked something like nine months ago, in that I suddenly felt much happier writing in this language than pretty much any other. It was also marked by the crushing realisation that I didn't really know how to elegantly structure a program the size of the game I was making, in the new-ish language I was making it in. I still don't having restarted in a new and exciting language, but starting from scratch means the pressure has been relieved. Temporarily.

I've dithered with the old Scala project a bit, but to be honest it has too many ugly bits and not enough game to be worth salvaging. I'll likely rip out the tech that worked and give it a whirl in the new language du jour; the deferred rendering was fun, if simple, and the transparency hack surprisingly effective.

2011 apparently having been the year of Clojure and non-blogging, figured I'd start the renewed attempt at updates with a few notes and statements concerning the language.

---

Records don't behave like functions of keys, unlike maps which they otherwise resemble.

I can see why this is the case, sorta. I'm sure you don't want records to implement too many interfaces by default. But I like to be able to get the value of a key using both (map :key) and (:key map) syntax, in addition to (get map :key) etc. It's also useful to be able to pass a record/map to higher order functions at times.

To enable the above, you just need to make your record type implement clojure.lang.IFn and a trivial invoke method:

(defrecord record-thing [id biscuit]
  clojure.lang.IFn
  (invoke [this k]
    (get this k)))

Now instances of record-thing will support both (:biscuit a-record-thing) and (a-record-thing :biscuit) syntax. Easy.


Handles, handles for everyone!

This is more of a general functional thing. Any language which encourages the use of immutable data structures obviously invalidates a common method by which objects refer to each other, namely by holding a pointer or reference to the other object(s). Take for example a simple case where a monster wants to keep a short list of relevant enemies, for use as input state to some AI functions. Our monster is stupidly simple, and simply stupid:

(def fred-the-goblin
  {:type :goblin
   :health 10
   :position [0 0]
   :targets []})

What goes in the :targets list? Clearly it can't be equally simple representations of target entities. They'd have to be updated as the world evolves, and may refer to Fred in their own target lists and thereby create recursive doom. In clojure, we could use atoms or refs everywhere, but that's hideous and rather belies our intent to be pure. What I've been tending to do is stick an :id field on everything, using this as a convenient handle for any object that should be uniquely identifiable (within a collection of similar objects at any rate, I make no attempt at global uniqueness. Quite possibly this will redound to my detriment).

Of course, in order to actually get at the entity referred to by the handle, we really want some nice associative structure.

(defn auto-map [entities] (into {} (map vector (map :id entities) entities)))

This seems like the most reasonable approach. So far it feels like spatial organisation structures, turn ordering, other such ways of organising the set of active entities seem more naturally expressed as collections of smaller chunks of relevant data with an ID attached.

I'm still not convinced this is the best approach though. It seems somewhat brittle. How does a more determinedly pure language handle things?


I don't read enough documentation

I had a semi-rant here about the lack of a promise-delivered? function, equivalent to future-done?... but whilst writing it, I discovered that this function exists. It's just called realized? (and it also works for futures, delays, and lazy sequences). I can't believe I missed this for so long, and it will help simplify some code that currently combines the two.


Promises are nice

Continuing the above: don't get me wrong, I love futures. They're handy little things. But there's a great reason to use promises for games: controlling the thread upon which you do work. One very common case when dealing with things around the level of OpenGL 2.0/DX9 is rendering resource creation, which generally takes place on the rendering thread. In my case, I have a queue of pending jobs on the rendering thread in the form of promise/closure pairs. Each frame, the rendering thread pops a number of these jobs off the queue, executes the closures and delivers the result to the associated promise. Other threads can block or skip executing code that depends on those GPU resources/tasks, as they choose.


Dealing with the (lack of) speed

No real getting around it: Clojure is pretty slow. Not agonisingly slow, but if you're used to decently optimised C++ it can be striking. Now, many things act against this, not least the fact that it's so very joyful to write in the language (In my opinion, YMMV etc). Concurrency is orders of magnitude more tractable. Concision, dynamic typing and a functional style frequently makes iteration on algorithmic optimisations much easier than it would be for a huge lump of C++. Transient collections speed up some common operations without sacrificing referential transparency, and so on.

But.

Still slow. Especially when writing somewhat idiomatic, pure code, which is what I want to be doing.

Thankfully GPUs are fast, and much as with the Scala project my aim is to use relatively few draw calls per frame and make each as expensive as possible to get the results I want. State from the CPU should be small and cuddly. This just leaves physics, AI, sound, IO and general game logic to be worried about when it comes to performance. For a turn based game the remaining problems are greatly reduced in potential for menace, but I'm sure I'll have fun with them when I do something real-time and action-y.


Vectors as vectors

I've lost track of the number of little linear algebra libraries I've written and used. For my Clojure experiments, I wrote yet another, but kept it deliberately stupid.

I have minimal requirements from such a library. In fact after reducing my initial bulletpoints a bit, I have but one: Be like HLSL/GLSL.

That means adding a scalar to a vector is legal. Multiplying two vectors together, also legal (it does this componentwise, dot and cross products are handled as separate ops). Preferably, don't do the GLSL thing and conflate transforming a vector using a matrix with your vector-vector and vector-scalar multiplications.

These are very much not based around sensible mathematics, but instead represent the kind of things you actually do with vector quantities on a regular basis.

Oh, and I like having decent quaternion support, because they're approximately infinitely nicer than trying to keep rotation matrices/Euler angles sane. Swizzling is optional, as it doesn't come up that often in practice.

The end result is that I use Clojure's vector collection type for all vector-like objects. I has nice literal syntax in the form [x y z] and supports unpacking directly: (let [[x y z] v] ...). Components are not named members of a structure. All the basic operations are based around a version of map which accepts a mixture of scalars and collections in all positions.


Because this needs more attention, however negligible the increment may be

A discussion of the concept of "wat" in software.

It's just a few things that Ruby and JavaScript do which... may be surprising. I'm sure there are more than a few of those lurking closer to home, possibly only unremarked upon due to my overlong exposure to the madness.



In closing...

I shouldn't have let this blog rot. I should get some pretty screenshots for next time!