Sweden, Ho!

I moved again! Now living in Uppsala, working in Stockholm. The past few months have been pretty exciting and we just moved to a new flat, but things are settling down. I've had a little time to poke at my personal projects, although I'd always like more.

My recent obsession has been with managing state. For many things, I seem to be tending towards a pipeline approach:

(defn foo [state x y z]
  (let [{:keys [a b c]} state]
    (assoc state 
           :a (bar x a)
           :b (baz b z y)
           :c (inc c))))

(defn quux [state]
   (-> state
       (foo "x" [] 0)
       (foo "y" [:r :s] 42)))

(defn outer-loop [state] (recur (quux state))

And so on and so forth. Passing state around explicitly as maps, writing functions that take the current state as their first argument and return that state modified with assoc, dissoc, assoc-in, update-in, merge etc.

Honestly, I'm not sure that this approach is the best way to do what I want. I have spent a little time looking at Clojure's state monad, but I suspect this is overkill (and despite being simple in practice, the whys and wherefores of all monads ever continue to befuddle me). The piped-stateblob approach translates nicely into ClojureScript with WebGL too; this is another thing I have been investigating of late.

Other advantages include very quick inspection or serialisation of a system's state (pprint to the rescue), and a single obvious point to recurse around in a thread's main loop. The stateblobs can also be useful carriers for communicating between threads.

On the other hand there's nothing particularly elegant or nice about carrying around the entire world with you wherever you go. And it works well enough that I'm starting to feel it is my hammer with which to treat every problem as a nail.

Using stateblobs everwhere: an ugly turn-based for prototype graphical RL #734.

Clojure-flavoured Rendering Framework: State

My rendering framework, such as it is, sits on clojars.org and does... not very much. That should change. I can't guess how useful it'd be to anyone else, but it could certainly be a lot more friendly.

With that in mind, I started taking a critical look at a few emergent antipatterns in the very core of the framework: the OpenGL wrapper.

This example, I am happy with. Mostly.

Last time, I mentioned how the rendering functions have state very explicitly threaded through them in the form of an argument. A bunch of my wrapper functions do very simple things to mutate GL state. Some of them reset it afterwards, or wrap those that don't in a friendly way. For example, matrix stuff is done something like this:

(ogl/with-matrix
  (ogl/translate 1 0 0)
  (ogl/scale 5.0)
  (ogl/triangles
    (ogl/vertex 0 0 0)
    (ogl/vertex 1 0 0)
    (ogl/vertex 1 1 0)))

Hopefully that's fairly obvious to someone familiar with OpenGL. The with-matrix wraps a glPush/PopMatrix pair of calls around all the expresions within it. Likewise triangles calls glBegin/glEnd with the appropriate primitive type. I debated not including this ancient method of drawing stuff, but it's so handy for debugging and quick tests... anyway, the point is that even though translate and scale mutate some OpenGL state, they're intended to be used within this kind of protected block. Maybe they should all have exclaimation marks, I'm a little bit hazy of when it's polite to start exclaiming when you're not mutating your arguments.

Obviously the triangles call and all the stuff therein is side effecting like crazy, but I'm not sure how much value attaches to trying to make those calls look different. If anyone has an opinion on the aesthetics of such things, please yell.

A lot of the rendering stuff behaves similarly to with-matrix. I have equivalent calls to wrap GL attribute fiddling, wireframe rendering (mostly debug friendliness), capturing the results of rendering expressions to render targets, assigning vertex buffers, and binding shaders. Some of these are could do with a refactor to make them more consistent, but they generally behave.


But then there's this...

So that's all well and good. Then I came across the with-matrix-mode macro.

Ick.

This one looks so similar to the with-matrix example, but instead of pushing and popping from the current matrix stack to allow safe manipulation of a transformation, it changes which matrix stack is currently active. OpenGL has a few of these, but the only ones that are really relevant are your "world" transform (GL_MODELVIEW) and your view/projection transform (GL_PROJECTION). The former moves vertices around, the latter manipulates your notional camera, as it were.

The reason this makes me twitch is that this doesn't in any way revert the changes you make within it. For example, here's a little function that sets up an orthographic view in place of the conventional camera-like perspective view:

(defn orthographic
  "Loads an orthographic projection matrix to the top
of the projection matrix stack."
  ([left right bottom top near far]
     (with-matrix-mode :projection
       (GL11/glLoadIdentity)
       (GL11/glOrtho left right bottom top near far)))
  ([left right bottom top]
     (orthographic left right bottom top -1.0 1.0)))

Sure, at the end of the function the matrix mode has been restored to whatever it was at the start, but there's no attempt made to preserve the state of the projection matrix. That's rather the point, in fact - it gets murdered and replaced. Everything after this will be rendered using the orthographic projection.

I suppose this function is named in a misleading fashion. Anyone assuming the state of the GL will be unchanged after executing it will be disappointed, whereas elsewhere I'm trying to leave things as I found them. On the other hand, changing matrix mode is rather rare compared to rendering stuff, you generally do it once to set up your camera and then leave it. This kind of mutate-and-forget approach makes it relatively easy to split up the rest of your rendering into small chunks, as long as you keep things in the correct order.

Elsewhere similar compromises might sensibly be made with the excuse of efficiency; there's no sense constantly setting and then resetting a bit of state if the next expression simply sets it again, or worse, if it sets it back to the same value. All told, I probably just want to find a consistent way to signal that some functions do not behave with respect to render state, whilst the default remains that all functions clean up their toys after playing with them.

Tweaks

Minor tweaks of late.

Restructured the rendering quite a bit. Keeping things very, very simple still: the rendering thread owns a list of functions that execute each frame, in the order in which they were added. Dumb as a pile of bricks, and assumes that each such render-function will actually do a bunch of stuff including culling.

Originally the rendering functions were all nullary and their results discarded. I swapped this over to each taking a single state argument, a hash map of arbitrary rendering guff, and each returns a (potentially modified) state blob. So the core of the rendering loop is a nice, cuddly-ish^*
(reduce (fn [s f] (f s)) initial-state render-funcs).

Right now the uses for this blob are limited. I'm just bunging a few things which were previously updated from different threads in there, such as camera data (ad-hoc derefing of atoms and such gives inconsistent rendering and general ickyness). So it's more like a way to carry around an immutable copy of the game state as of the start of the frame than anything truy useful. Suspect it'll be handy later to mirror the GL state and avoid heavy state-changing operations where they're not necessary, do pre- and post-invariant checks and other contract-y things, accumulate frame stats such as objects rendered... stuff.

Image is just some randomly coloured lights scattered around, this time with a bit less gloom. Still quite gloomy. I find myself wondering if it might be better to work with entirely solid hexagonal prisms as a starting point, rather than more elaborate geometry. Hmm.

---

* Cuddly-ish, but not very cuddly. I would really rather use something like (flip apply) instead of the embedded anonymous function, but cannot quite seem to conjure the right way to do this without doing something slightly horrible. Will look at it later with fresh eyes and doubtless slap my forehead at my own stupidity.

Lights!

Decided it was time to fling some deferred rendering at the project, because who doesn't like taking huge chunks of VRAM to do simple things? Thanks to Jotaf's comments last week, I have a much easier method for making and rendering walls, so it only took a few minutes of Blender-torture to generate new ones.

The question of style is one I haven't resolved yet, alas. Making tech is far, far easier than trying to hang a scene together as a coherent whole. I know the cultural roots the game world will draw from, but not whether to go for a cell-shaded or painterly or somewhat realistic approach. The latter is probably most tedious in terms of rendering tech and modelling, but is almost certainly the easier to at least rough out textures based on photographic sources.

2D Is Easier

Annoyed with my inability to find a nice way to create 3D tiles, I made a little tiny tile-editor for the 2D version.

Everything's so much easier in 2D. Ah well.

Edit:

I added some water and grass tiles, but not enough for a new post. Trying to do something in the spirit of pixel art but that is amenable to rotation around 60 degree increments (and requiring less in the way of raw talent). I think I might need to try flat colour polygons and a finer subidivision, but it's an interesting clean look.

Holes

Not an awful lot to discuss this weekend. The dungeon is now a multi-level affair; you can just see the hole in the ceiling in the ugly screenshot.

At the moment, you can fall down a hole by walking into it, but climbing up requires a few conditions to be met. You must be directly under the hole, facing a wall (against which I shall eventually rest a ladder, I suspect), and the hex on the upper floor directly above the bit of wall you're facing must also be clear of obstacles. These conditions being met, you scramble on to the upper floor.

I'm considering shrinking the vertical distance somewhat. To keep things simple, the hexes have a diameter of 1 unit, the player has a notional eye-height of 1 unit above the floor, and ceilings are 2 units off the floor. Whilst this mostly works, it means that even with a relatively wide vertical FoV (I'm using ~100 degrees in recent screenshots), it's hard to clearly see a hole you're standing underneath. I'm considering just halving everything in the vertical direction and seeing what it looks like.

I also finally knuckled down and properly categorised wall-tiles. Given a tile that contains a wall and that isn't completely surrounded by walls, you either have:

• Zero neighbouring walls - a pillar.
• One neighbouring wall, a length of wall comes to a stop here.
• Two neighbouring walls - with the walls in ortho, meta, or para position (adjacent, with a one-hex gap, or opposite each other).
• Three neighbouring walls - four possibilities here, either clumped together, in a Y shape, or two different chiralities of "two-blocks-together plus one block on its own".
• Four neighbouring walls - equivalent to swapping the two-wall case for two-spaces.
• And lastly five neighoubring walls, giving a little nook.

Someone smart would doubtless have done something clever involving symmetries; I just assigned each wall tile a binary number by starting at the hex due north and proceeding in counterclockwise fashion around neighbours, setting a 1 bit for walls and 0 for empty spaces. I created rough meshes for the thirteen interesting cases in Blender and named them according to this scheme (wall_00, wall_09 etc). Then it's just a case of extracting this categorisation number from the map data at runtime, rotating the bits to get the minimum number out of the given arrangement of zeroes and ones, and rotating the numbered mesh the same number of times before rendering.

So that worked quite well, apart from the tiny fact that it's looking like a real pain in the arse to author interesting meshes without leaving sparkly gaps. I'm considering welding the mesh as a post-process, either the whole dungeon or in chunks depending on poly budget, but by this point it almost feels more sensible to hand-craft a dungeon level as a chunk of unique geometry and then go around setting up the logical grid to match.

Alternatively, it should be easy to do something like calculate a large, concave polygon for all dungeon geometry and then extrude it, but this doesn't help make the walls interesting to look at.

None of these approaches save hand-crafting or something involving marching cubes/tets suggest anything for a more natural environment such as caves, either.

Think I'm going to try to get the 2D debug viewer working and maintain that alongside the project, so I have somewhere to test out things whilst wrestling with the demons of content creation.

Dense Bugs

"The reasons go back to perhaps the most important empirical result in software engineering, one we've cited before: the defect density of code, bugs per hundred lines, tends to be a constant independent of implementation language. More lines of code means more bugs, and debugging is the most expensive and time-consuming part of development." — Eric Steven Raymond, The Art of Unix Programming.

I rather dislike verbosity in code, so I would love the above quotation to be true. Unfortunately at the moment I'm running on received wisdom and gut feeling; I have yet to find any concrete studies on the subject. It certainly wouldn't be easy to perform such a study, given the moving target of popular languages, frequently poor distribution of languages in a given domain, differing fluency of programmers requiring large sample sizes, choice of how one enumerates bugs and when one chooses to do so... I'd hope that a language that makes high-coverage automated testing easier would have a somewhat sharper decline in extant bugs over time, but maybe that's simply wishful thinking.

There are limits to how far this pursuit of brevity can be taken, of course. The more conceptually dense the code the more wetware involved in expanding it to a form that can be understood, and humans are hardly infallible when it comes to such transformations. Misunderstanding an obfuscated chunk of code will lead to the introduction of bugs, and the point at which pleasantly concise turns into impenetrably terse will be different for each reader. Meaningful naming of intermediate values helps me unpack a series of functional transformations of a collection, so I'll fairly regularly expand a series of maps, filters and folds within a let expression, and if necessary interject comments (with small functional chunks of code producing values bound to meaningful names, it is rarely necessary). So far so standard.

On the flipside, various kinds of boilerplate that add nothing meaningful beside additional LOC are all too common, even when avoidable. To take a small example, as the quite lovely Effective Scala notes, there is little benefit to "syntactical ceremony" (I love that phrase) by way of superfluous braces on simple functions or control structures. Yet many languages do not allow for such details to be elided, and even if they do we have a curious tendency to enforce maximal syntactic salt. I have never yet worked at a game developer whose coding standards allowed the quite standard C++ convention of skipping the braces on single-line conditionals and loops.

I call bullshit on this kind of ruling. In fact, I am not at all convinced by coding standards beyond a way to record naming conventions, formatting of comments to generate valid automatic documentation, and very general guidelines for style where the choice might otherwise be non-obvious (e.g. avoid using interfaces for small data-structure objects used in an inner loop, those vtable jumps add up quickly). When it boils down to the dictation of salting levels, especially favouring more rather than less, such documents cease to be useful.

In my opinion, anyway. I'm sure someone disagrees, because the bloody things never cease to be a pain in the neck.

Long story short, death to large codebases and coding standards that promote them, no matter how small the increment.

---

Game progress ticks along, I'm currently investigating dual quaternion skinning and fiddling with the relationship between the game logic loop and the renderable representation of logical entities. In particular, it was deeply confusing having multiple monsters performing multiple actions all at once, so I've fallen back on the model used by Wizardry 8 (among other games): wait for animations to complete before letting the next logical entity take a turn. I am worried that this will become painful for large groups of monsters, so this will doubtless require iteration.

I'm also going to have to check how to better push animation state onto renderable entities. At the moment it's just inferring that a "hurt" animation needs to play because the entity has less health than it had last turn, and so on. This is simple and makes sense in some cases, but does preclude having a "stagger backwards from explosion" animation that is unique from a "walk backwards carefully" animation, for example.