RE: Steve Yegge: Code's Worst Enemy

Hello Reader,

First, read Steve's article Code's Worst Enemy,which makes a pretty sensible claim that his half million line of codebehemoth of a game is too big, and that it is partially Java's fault.Right at the end of the article, he dismisses Lisp as the new languageof choice, with no real argument, except that he would be laughed at.He also indicates that he wants to continue using the JVM, because hisgame depends very heavily on the JVM's threading model. I hear variousforms of this argument often, claiming that a JIT compiler is betterthan native code. What they probably mean to say is that the dynamiccompilation abilities of a JIT compiler are better than a staticallycompiled C++ application. You can make a C++ application dynamic, butonly by performing DLL voodoo, or using managed C++, which is even uglier than ordinary C++.

A quick walkthrough the Evolution of Game Development

Because of concerns about performance, the mainstream video gameindustry has always been paranoid, and is years behind the frontof the curve for new languages. Assembly, C, C++, and now a little bitof managed C++ for the application, and a score of scripting languagesfor the parts of the game that absolutely require dynamic languages,albeit with lots of distrust for framerate hitches by the JIT compileror garbage collector. The version of a game that ships to consumers istypically called "Release" or "Shipping" build, and has almost alldynamic C++ features disabled, including RTTI and exceptions.

The indy/hobbyist game development world is currently going nutsover XNA. Write once, deploy on PC and XBox 360. C# is better than C++,and is a reflective language. C# comes with a performance hit, but thathit is less than the percentage of productivity gained by a developerwho has avoided C++. So XNA has turned out to be a pretty big deal. Andeveryone agrees that the CLR's support of C# is essential, because C#is so much more dynamic than C++. 

Thevideo game industry is waking up to the idea that multicore programmingis hard, particularly hard in C++, and that C++ is not efficient forrapid prototyping in terms of programmer effort. This is occuring atthe same time that Microsoft and Sun are beating their drums andpromoting two VM based languages that are better than C++, where betteris defined as easier to program in the host language with only a smallperformance loss. Both C# and Java are subsets of C++ that are cleaner,and with additional features to make it easier to write code that isself inspecting.

But why use a JIT compiler or the JVM/CLR at all?

So here's my beef. Why use a JIT compiler or heavyweight runtime atall? What if I could use a very dynamic/reflective language that couldbe compiled into a native code .exe? What if I could use the samelanguage that was used for the game engine as a scripting language, anddynamically compile and link it into the native code application? Thisis precisely why I am using Corman Common Lisp in my game.

Before you dismiss my game project as a toy, let mebuild a little credibility with you. I am using Gamebryo, which is apretty popular graphics solution for AAA games and Korean MMOs,Chipmunk 2D physics, Flash 9 for my UI, and fmod for sound. All thesepackages are wrapped and exposed to my Lisp environment. When the gameloads, it dynamically compiles all the game specific code, which ishenceforce known as "compiled" code. The garbage collector nevercauses framerate hitches, and I can replace compiled code with othercompiled code in a live running application instance. The only overheadis the same overhead that exists for DLL calling applications, which isthat instead of a JMP instruction to some function, the Lisp runtimeprovides a hash table to map function calls to the current codedefinition for that function. 

Is that extra complexity worth it? Here are some of the things that Lisp has allowed my game to do:

• Offload all game specific code to scripting language without a significant performance loss.
• Retain scripting language benefits like executing commands from in game console or hot reload of code.
• Use macros and special variables to define a compact DSL (domain specific language) that hides complexity from user.
• Even that DSL is compiled!

How can macros be helpful?

 (upon-collision (collectible vehicle)
    (play-sound "../data/munch.ogg")
    (remove-game-object collectible)
    NIL)

That's one example ^. I wanted to define what happens when twodifferent objects collide, so if I exand the upon-collision macro, Iget this:

(LET ((COLLISION-TYPE1 (GET-GAME-OBJECT-COLLISION-TYPE 'COLLECTIBLE)) 
      (COLLISION-TYPE2 (GET-GAME-OBJECT-COLLISION-TYPE 'VEHICLE))) 
    (SET-COLLISION-CALLBACK COLLISION-TYPE1 COLLISION-TYPE2 
        (LAMBDA (A B CONTACTS NUMCONTACTS DATA) 
            (LET ((COLLECTIBLE (GET-GAME-OBJECT-FROM-SHAPE A)) 
                  (VEHICLE (GET-GAME-OBJECT-FROM-SHAPE B))) 
                (PLAY-SOUND "../data/munch.ogg") 
                (REMOVE-GAME-OBJECT COLLECTIBLE) 
                NIL))))

That chunk of code is looking up a value in a hash table andcreating a new value if there isn't one, for each game object type thatis represented here. A dynamically defined function body is being setas a callback which is given back to the C++ based physics engine. Idare you to show me another language that could make it this easy toembed a pre-filled hash table in compiled code! 

What about these "special variables"?

 (offset '(1000 100)
	(make-carrot))

when macroexpand-1'ed becomes:

(LET ((*OFFSET* (V+ '(1000 100) *OFFSET*))) 
    (MAKE-CARROT))

*offset* is a special variable. It is defined like this: 

(defparameter *offset* '(0 0))

You can think of *offset* as a global variable with a twist. When a let block, such as the one in the offsetmacro refers to a global variable by assigning it a new value, then thenew value is pushed onto the global variable. At the end of the let block, that new value is popped, and the old value is restored. The function make-carrot calls various lowest level chunks of code that will add the current value of *offset* to the final values that are called out to the C++ host application when physics objects are being generated.

Think about this abstraction for a second. I'm doing something thatmight seem very dirty! I'm using a global variable as a hidden extraargument to some very low level functions. Moreover, the user doesn'teven know that he is updating this variable's value with the offset macro. If the user calls those low level functions directly at the toplevel, the default value of x=0,y=0 from *offset* won'tadversely affect how those functions behave. Because I only am updatinga global variable's value within a local scope, I don't have toreimplement the save/restore pattern when modifying it, and therestoration happens automatically, so it happens reliably without humanintervention.

Another Macro example:

(per-frame
    ; animate a little circle directly
    (set-position (list (* (sin *accumulated-time*) 400.0)  0.0) *test-circle-body*)
   
    ; put camera at average vehicle location
    (let ((pos '(0 0)) (num-found 0))
        (dolist (game-object *game-objects*)
	        (when (typep game-object 'VEHICLE)
	            (incf num-found)
	            (setf pos (v+ pos (get-position (vehicle-chassis game-object))))))
	    (unless (= num-found 0)
	        (setf pos (v/ pos num-found)))
	        
	    (set-camera-position 0 (list (first pos) (+ (second pos) 2500) 5000))))

 becomes

 (SETF *UPDATE-FRAME* 
    (LAMBDA (FRAME-TIME) 
        (SETF *TIME* FRAME-TIME) 
        (SETF *ACCUMULATED-TIME* (+ *ACCUMULATED-TIME* *TIME*)) 
        (DOLIST (GAME-OBJECT *DELETE-THESE-OBJECTS*) 
            (WHEN (FIND GAME-OBJECT *GAME-OBJECTS*) 
                (ACTUALLY-REMOVE-GAME-OBJECT GAME-OBJECT))) 
        (DOLIST (GAME-OBJECT *GAME-OBJECTS*) 
            (LET ((UPDATE (GAME-OBJECT-UPDATE GAME-OBJECT))) 
                (WHEN UPDATE (FUNCALL UPDATE GAME-OBJECT)))) 
        (UPDATE-GRAPHICS) 
        (SET-POSITION (LIST (* (SIN *ACCUMULATED-TIME*) 400.0) 0.0) 
            *TEST-CIRCLE-BODY*) 
        (LET ((POS '(0 0)) 
              (NUM-FOUND 0)) 
            (DOLIST (GAME-OBJECT *GAME-OBJECTS*) 
                (WHEN (TYPEP GAME-OBJECT 'VEHICLE) 
                    (INCF NUM-FOUND) 
                    (SETF POS (V+ POS (GET-POSITION (VEHICLE-CHASSIS GAME-OBJECT)))))) 
            (UNLESS (= NUM-FOUND 0) 
                (SETF POS (V/ POS NUM-FOUND))) 
            (SET-CAMERA-POSITION 0 (LIST (FIRST POS) (+ (SECOND POS) 2500) 5000)))       
        NIL))

Again, alot of stuff is happening that the level scripting usersnever even have to see. Without macros, I would have to write aframework that calls CGameApp::UserUpdate or something. Here, I don'teven need a framework, because macros allow me to set up the state forthe user by inserting code for them. If you have any questions or wouldlike to see other examples of how I am using macros, just reply in thecomments. I hope that someone somewhere is curious now, and willconsider disregarding Stevey's post's disregarding of Lisp.