“The Art History of Games” will be a 3 day symposium on the role of games as art form
February 4-6, 2010 at the High Museum of Art, Atlanta
How do games relate to the established art world? Games have served humanity in a number of diverse ways including entertainment, education, exercise, conflict resolution, ritual and self-expression. But not until the 20th century did games and the play experiences they provide emerge as an art form as well. With nods to the past and future, and with open acknowledgement of all the awkwardness, bravado and measured successes thus far, “The Art History of Games” seeks to more clearly articulate the importance of digital games as a form of art. The symposium aims to bridge the gulf between practice and theory in a combination of participating game designers, scholars and art historians.

Confirmed speaker include John Romero, game industry pioneer and icon; Christiane Paul, Curator of New Media at the Whitney Museum of American Art; Frank Lantz, Director of the New York University Game Center; Jesper Juul, noted game scholar, author of Half-Real; and Henry Lowood, Curator of the History of Science & Technology Collections at Stanford University.
Embedded in the event will be the premiere of three commissioned artgames by Jason Rohrer, Tale of Tales and Eric Zimmerman. The designers will present their work during the conference, and will participate in the discussion.

The symposium is organized by SCAD Atlanta and the Georgia Institute of Technology. Info and registration here.

“Ninety per cent of the work is convincing people that the eyes are real”

But what makes them “real”? We talk about the dead eyes of Tom Hank’s characters in The Polar Express and the quality of way eyes are depicted often co-defines the live-ness of any animated character. Disney used the different animation techniques of “dead” versus “alive” eyes for example in Pinocchio to emphasize the difference between the wooden puppet and puppet in its alive state.

“As Walt had said, the audiences watches the eyes, and this is where the time and money must be spent if the character is to act convincingly.” (say Thomas/ Johnston)

The same logic reversed is at work in Coraline, where the dead eyes are a good example for truly scary symbols.

The classic animation technique for eyes follows the squash and stretch techniques. On the other side, there is Wall-E that follow completely different concepts of how to bring eyes to live. Wall-E himself has literally deep and multi-facetted eyes that seem to lead into a deeply romantic robot soul as opposed to the flat and brutal “other mom” in Coraline.

So what makes the eyes of Wall-E so expressive and those of the human Tom Hanks so dead? Wall-E’s eyes are abstracted and seem to work exceptionally well on three interconnected layers: design, animation, and rendering.

The basic design seems to be simple and rigid. Where the eyes in Robots include eyebrows and overall try to make “humans with metal skin”, Stanton’s approach to Wall-E was to see the main character as initially a pure robot come to live (another reference to Pinocchio). Wall-E’s eyes, thus, are functional, based on binoculars. In fact, as Wall-E demonstrates in the film, eyes can be replaced. At the same time, they are highly detailed. The great classical Disney animators gradually added more and more detail to the drawn eye: iris, pupil, light reflection, shadow effects, lashes. Pixar adds lenses, materials and mechanical devices. As Stanton mentions:

“One of the biggest epiphanies we had was giving him zoom lenses for eyes. They have inner lenses that slide back and forth and replace what a human face does with its pupils and eyebrow muscles. It shows he’s alive and thinking. I think that’s what gives him a soul.”

However mechanical the underlying design, the animation of the eyes is outstanding in the way it allows a single raised eyebrow and a change of individual eye positioning. One reason is the lack of any other facial feature. Because there is no mouth we start to completely rely on the eyes as the main expressive tools and assign them more value then, for example, in Robots. This is aided by the sound design, which does not try to hide the robot-ness of Wall-E (again see the difference to Robots) but instead supports his identity. Finally, the rendering:

Stanton went back to the look of 70mm cameras to design the look and feel for Wall-E and shape it toward a much more intimate environment. This includes extensive work on focus, lens flares, visual distortions, and the like. All of that plays into the expression of the eyes, as you can see in the shot above.

So where does Machinima stand in this regard? Is a random blinking of the eye the best it can do? And how can real-time imagery learn from this?

Sure enough, Wall-E has been re-modeled for machinima and it might serve as a first point of reference in this comparison.

This is not a cutting edge implementation. Not even for machinima standard. For example, one can control the gaze in more modern game engines, but controlling the direction of the gaze is still a long shot from the options that drive more elaborate animation.  The limitations of the result also highlight the long way ahead to create “alive” eyes in real-time. The above mentioned Wall-E model has a fixed texture. Unreal 2K4 separates eyes into iris and eyeball, then concentrating on the animation of the iris. But the button-like flatness becomes immediately obvious in a close up:

Unreal 3 its FaceFX technology obviously adds a lot more detail, materials, and animation options but I somehow see more of Polar Express in some samples (like facenet’s). If one looks at CGI tutorials on modeling an eye (like this one), their focus on the irregularities in the shapes becomes clear. When creating a realistic eye, it seems that these irregularities become more and more important to get it right.

The obsession with photorealistic appearance might once again be a limiting factor here. So let’s focus on a better example to compare to Wall-E: Stolen Life. Here, we have also robots as main characters, eyes do most of the acting – and they are remarkably simple.

The robotic eyes in Stole Life work because they are very simple. Eyes are literally cameras pointing into a given direction, often moving only along a single axis. Their emptiness mirrors the emptiness of their surroundings, which are sparse and low detail. Consequently, they perfectly fit into the film’s setting and overall universe. If one compares the opening of Stolen Life with the space sequences in Wall-E, the differences become obvious. Still, because Stolen Life stays within its territory and style, the visuals overall make sense. It is the simplicity and imagination of the design that makes Stole Life stand out as a piece of art.

To finish this much too long post up on a positive note: Where machinima might just be on par with Pixar’s wonders is in the way the animation might actually be triggered – or better: how we can interact with the animation. Because it is real-time machinima might be able to offer interesting control options like these. These might be rather useful in some real-time performative machinima piece.

So maybe we see some interesting new opportunities here. I have been wondering what a new technology like Natal might bring to gaming (and I still wonder) but it might just be the right tool for machinima and its offspring.

Now … before I start to talk about the eyes of Eve I better stop these ramblings or I’ll be completely doomed.

For somebody who lived somehow parallel to this “match”, it is exciting to hear Aarseth referencing narratology, characters, movies (!), and narrative spaces. Definitely a change of vocabulary, although he himself argues that it is not necessarily a change in his position but a “deepening.” A long version of his current position can be found here. In his short talk here at Tech he seems to focus on characterization. Another element that becomes clear is that the hot air of the old fight has evaporated and we are on much less confrontational grounds now.

Aarseth stays true to his trade: talking about structures and narratology as a matrix we can use to analyze. Fox Harrell offers a more associative idea of “phantasmata” where some part of the image remains withdrawn and mysterious. And he looks at that through the looking glass of cognitive science. The results are less linear, inspired e.g. by Calvino and Jazz music, and pointing to oral traditions instead of literary ones. They might look like this:

Janet Murray talks about “dramatic expectation-setting” and creating suspense in games. She argues that we have to “synchronize” and make sense of events to provide for a valuable story in a game. She distinguishes between “multisequential” structures that have multiple paths and limit the player to experience only one – thus always having other options of the individual moment available in the system. And “multiform” scenarios that live off the variation of a single situation – e.g. Groundhog Day. She counts Mateas’/Stern’s Facade in this second form. Both forms have their own structural elements and key factors but they both rely on dramatic compression. Like Aarseth, she basically closes with a point to future and developing richness and growing media that will show us the way in games and narrative.

Ian Bogost starts the discussion and one discussion point is that “story” might have become a topic itself that needs to be revisited in the cross media/ video game world we live in. Maybe it is not a stable reference point anymore? There are still different perspectives but all seem to embrace the narrative. The debate got a bit more heated over Wittgenstein – which indicates where the whole discussion is situated: deep in academic waters. Thinking about it, with Janet leaning on the understanding and drama and Fox concentrating on cognition, Espen somehow remains as the closest to the traditional idea of “story” as classic narratology describes it. Maybe less surprising than is sounds.

After all, my own musings about narrative in games (like this one or that one) is somewhere inbetween as I think that narrative is basically a form of understanding and we can use game worlds to shape the experience that is understood. With that I am still “parallel” to all three of them.

Second thing was to compare the pitch with the (first?) film trailer released by Disney. Before I looked at the original film trailer and the film’s site I was a bit disappointed by Mechner’s pitch reel. Given the livelong interest of Mechner in making Hollywood action movies, the trailer is a bit fragmented and to me there were more bones and beef in the game than the pitch reveals. But then again: this is not about content revelation but about evoking emotion and excitement in Jerry Bruckheimer and compared to the Disney version, I have to say that his pitch is actually not too bad. It is also twice as long – so that might be the reason I get more out of it. Still, Mechner’s version uses more (and better) music then the Disney one, which adds some breadth to the admittedly not too complicated premise presentation. It also has way more locations, which basically does the same: add value and visual variety. Especially this second point clearly plays to the strengths of machinima. We can create a new film set far easier and cheaper then some full production. Basically, this comparison highlights Hugh’s points on what machinima does well: scale, learning the trade, action, speed, and cheap production. It is also a bit of a trap, though, because maybe we expect this set variety in the future of video game movies. Maybe these strengths of machinima are not only blessings for their entry to Hollywood but set new expectations that are hard to meet by “real” film production?

Overall, the machinima version carries pretty much the same appeal as the film version, which might say something about the film project as a whole but also indicates that machinima IS really capable of swimming with the big fish here. The action is better, it has some unique camera references (no slo-mo in the Disney movie? – the Mechner version at least alludes to it)), the characters are pretty much the same overall, and the mood is presented better (mainly through music and sets).

No doubt, the live and play elements are two of the unique selling points of Machinima and there were a range of shows that experimented with it: This Spartan Life, Tra5hTa1k and the Campaign Trail shows by the ILLclan, even the Bob Block show. Strangely enough all of them are on somewhat of a hiatus and I am not aware of any other live shows at the moment but would love to hear about more of them (so please comment). All the technical conditions are there: two revolutionary animation techniques, dedicated communities behind both of them, both have qualities of live TV formats …

So why is there no machinima Muppet Show?

One point is closely related to the technically defined issues of expression that hovered of the last post. The problem with basically all Machinima engines is that they strife in one way or the other to offer photorealistic imagery (below one of the Far Cry 2 comparisons).

The puppets in The Muppet Show are not perfect. In fact, their imperfections are one reason why they are so lovable. We embrace them as individual characters because they show their “flaws” as part of their character traits. Because each puppet is unique, each unrealistic “flaw” is unique and interesting. The Muppet Show is a variety show not only of brilliant puppetry, comedy, and music – but also of continuously changing imperfections. As one example, let’s look at hand control in three different puppets:

Kermit’s rod-controlled hands flap around (giving him the excited expression whenever he announces one of his guest stars) while Rowlf’s fingers in the gloves of his puppet literally seem to play the piano (which allows him to have numbers where he literally does little more then playing a classical piece with small comedic moments). One of the piano playing puppeteers – Steve Whitmire – mentioned that

One of my favorite things to do, ever, in my career has been to do Rowlf’s hands on the piano, which is something that other people had done, but once I started doing it, I kind of have done it since… It’s the best job in the world for somebody who sort of plays the piano, but would really like to play the piano well…you can approximate what it’s supposed to be and make it look really good.

And finally there is Dr. Teeth, who takes this set up even further with extendable arms. All three characters originally were played by Jim Henson himself – all three work differently – all three are non-realistic and the differences between them make it interesting to look at them.

Compare this to the repetitive perfection of the Master Chief or the mo-cap libraries of other tools. Admitted, the more physics enter the animation system, the less repetitive the movements get, but the problem of realism remains. The holy grail in game animation still seems to be the fact that the character puts down his foot properly on a somewhat deformed surface – not the option of doing a ballet jump.

A second point might be timing

The Bob Block and ILLClan shows depend on relatively advanced live puppeteering systems. Operators control mouth, individual arms, gaze etc. to far greater detail then in closed systems (such as Halo in the case of This Spartan Life). But for some reason we rarely give our characters time to breath in these live shows. Traditional machinima movies are better (see the timing in Red vs Blue or Clear Skies) but our live show timing is usually very high paced. The ILLClan even succeeds in bringing in a comparable kind of stand-up comedy into their shows that at times equals The Muppet Show’s writing. But it is a sign of their limitations that even the ILLClan does not manage to depict calmer moments in their show – a regular feature in The Muppet Show, that moves expertly between comic mayhem and reflective calmness. Again, it is the variety of expressions that carry the change. One changes not only from one number to the next but also from one animation method to another.

This change is not available in machinima. Even Sid the Science Kid with its real-time animation control and Henson credentials uses the same system for all characters. Here they are, lip-synching and doing facial animations:

Conceptually, this mirrors the TV-specific puppeteering control of The Muppets. Henson developed that approach where the puppeteer controls the puppet while looking at a TV screen showing the camera’s view. But in contrast to the Muppets all these controls are the same for different characters. The characters in Sid are usually human kids and the controls might be just fine for that, but one still wonders whether this is not the end of variety in performance if the interface is streamlined. And that is exactly the case in video games.

Another challenge to machinima variety shows is the inclusion of real actors. It is a constant source for comedic interaction in The Muppet Show, with guest stars joking all the time about the differences between puppets and real humans – while Kermit steadfast considers his troupe “real”. This Spartan Life tries to offer the element of celebrity guests but only their voice truly remains in the show. Yes, guests might have different play styles, prefer different weapons for a shoot out, chose different Halo characters as their representation … but all this dwarfs in comparison to what the Muppets do to their guest stars on a regular basis. They actively use the difference between real and virtual, turn humans into puppets and puppets into humans. After all, Dr. Teeth was modelled after Elton John and when he did his show for the Muppets, it seems completely normal that Elton and Miss Piggy do a duet.

The question of acting in VR has been targeted in academia but how to structure a possible combination of real vs virtual remains a question.

Sure, there are a number of real-time puppeteering events that use some kind of video compositing to combine the computer with the real world image. Here at Georgia Tech we played ourselves with Augmented Reality and Improvisation theater but the virtual remains floating on top of the real (or behind, depending on your set up). The very same unique magic that allows Rowlf to shake hands with a real person or Elton John to look into Piggy’s eyes is missing. Not only is a handshake ridiculously difficult to control on the virtual side, but we obviously lack the physical contact.This kind of “meeting space” between the real and virtual is a difficult area to explore in machinima.

So why lamenting about these problems (and I am sure there are a ton more)?

Because I believe that the variety show would be one thing Machinima could do that a real-time Maya application could not do. It certainly is a difficult task that poses technical and conceptual challenges but we  see real-time stuff coming out of Autodesk now and we have to find the niches where machinima can deliver the goods and Maya cannot.

The book is a short collection of essays and interviews taken from within the demoscene with Lassi’s “A Brief History of the Demoscene” as the most substantial one. It is not necessarily academic/ critical but a helpful introduction to the demoscene and its roots.

It always helps to feed some reference materials to our (still) under-published field.

However the image below seemed to me to be maybe the most relevant change.

Now you can see the assets that available depending on the packages you bought – but you also see the assets that you have not yet acquired. Those assets are x’ed out but you still get an idea of how they look. This might end up to be a very smart marketing move and possibly encourage a lot of folks to buy more packages and thus feed MovieStorm. I am sure that Matt will post a ton of much more detailed info on the next version somewhere here; but if it works out, then this interface move might be a smart way to support the company overall and a glimpse into future marketing models in Machinima to come.

As it is in norwegian, it’s a bit tough for me to explain what exactly is going on – but it’s a sci-fi piece about breaking out of the order and finding out who you are.

The overall production time for the piece was 4 months with a very, very low budget. Creative director was Catherine Kahn, who also wrote the script together with Trond Morten Kristensen Vernaasen. You can check out the full credits on the poster.

I was in charge of the Art and Technical department, and had help from a couple of illustrators (Chris, Mika, Hannah) and Chris Sugrue as an additional programmer who developed some super-useful tools for animation and ported videotextures to moviesandbox using some openFrameworks functionality (original code by Theo Watson).

I would like to post some sort of post-mortem on the way things went in this production, if people are interested.

I do not yet know what to think of it. In my humble opinion, some names and films should be added to their first outline. But what I found really funny is that is seems they use Italo Calvino’s ‘Invisible Cities‘ as a reference for their machinima production (e.g. Argia here). It is an amazing book and very inspirational.

And this post is not so much about the overall artistic value of 6 Days but watching the film another thought came up – that of the limitations of the engine and what to do about them. In short, I think one can use them, negate them, or bend them and each of the three approaches leads to a different outcome.

Originally machinima was touted as game-base film-making. Looking at 6 Days it becomes apparent that the game engine can be brutally limiting in the range of expressions available. This is not really a new discovery. The game’s interactive design is like a rigid filter for how the engine can be used. When you are a hammer, the whole world seems to be full of nails. When you are only capable of shooting in Call of Duty 4 (the engine used in 6 Days), the whole world seems to be at war. It is about the limitations of the underlying interactive system and the play that evolves from it.

To some extent 6 Days (as well as a lot of DeLappe’s other work, such as dead-in-iraq or his Gandhi performances in Second Life) is exactly taking these restrictions as a jump off point for the art. Dead-in-Iraq might be the most obvious approach. Without a single change to the engine, he manages to build his own statement on top of the ongoing game. As he types in the names of soldiers killed in Iraq into the chat channel of an America’s Army game session, his performance depends on the game running in its usual way and the new connection he creates through his word performance. Truly machinima in spirit and just one step away from pure gameplay movies.

Where DeLappe leaves the game intact but changes the play – others like Kirschner or Condon take the game apart to express something that very often does not refer anymore to the originally intended play. Karma Physics Elvis by Condon or Person 2184 by Kirschner might run in real-time and display the UT2K4 (and UT2K3) game technology in various ways, but they certainly are not about the original game play. DeLappe’s work is. Between these two ideas of either completely utilizing or disregarding one could find all kinds of gray zones.

Some time ago a few of my students (Thomas Barnwell, Paul Clifton, Betsy Gooch, Justin Smith) made a project that rests somewhere in between the two poles. I taught the course as a graduate class and the final project was supposed to be a piece in Unreal that would address the question of the American Myth. Students were asked to develop an interactive piece that would express some element of the American Dream. I liked all the final group projects, but this one might be the easiest to describe.

Barnwell/Clifton/ Gooch/ Smith built on the idea of action painting by Jackson Pollock and combined this with the simplified idea that anybody can “make it” and succeed in their path in America. Their own description reads:

The project is a reframing of the idea that each of us creates our own unique American Dream as we traverse through space over the course of our lifetimes. Trails of changing color are left as you traverse across the virtual space. These trails represent the choices you make during your life, and the changing colors represent the factors that are beyond your control. Once the sculpture is complete, you have the ability to view it as a whole, or print out a physical copy which represents your own unique American Dream.

In practice: the player steers a character over a huge empty map of the USA and leaves a trail of colored particles on the way. One paints an action painting as one lives their virtual life. Here you are traversing the virtual Rocky Mountains:

Once you are finished, you can receive a top down perspective of the picture you have created:

To me, this seems to be a third way of game-based machinima – or whatever one might call this kind of work. Neither does it reject the underlying limitations of the engine by completely modding them away; nor does it comment on the engine itself. Instead, it is a form of re-modeling. There are still important elements of the game present (e.g. that they did not switch off the HUD nor the weapon – which certainly gives a weird context reminding me of completely different US-traits) but it is a bending of the play and the game rules themselves, not a negating (as in Condon and Kirschner’s case) or a rigid re-framing (as in DeLappe’s and Diltz’s case).