Monthly Archives: March 2013

Cel-Shaded Graphics: Hardware and Art

It should be pretty uncontroversial to say that software and hardware have something to do with one-another. This idea is very strange, however, given just how opposite the individual natures of software and hardware are. Computer hardware is the physical, tangible component of computer science: if you took an electron microscope and applied it to a computer, you would be able to see the doped silicon lattices in its semiconductors. Computer software, however, in its ideal form, is conceptual. Software is something that is not necessarily physical. Software possesses the ideas and ‘thoughts’ contained in its programming, and this is something that can’t be manifested physically. If we tried to look at software in its physical form–say, on a hard-drive or on an EEPROM–all we would be able to see or sense is the patterns in which a program’s atoms and electromagnetic charges have been arranged.

The Hogfather: Your Ideal Swine

The character Death in a Terry Pratchett work, Hogfather, puts the dilemma rather well. Obviously here Death is talking about morals, but the argument is analogous to concepts or ideas:

…then take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.

Admittedly, it’s pretty well understood that software is written for hardware–computer instructions are specifically written for physical machines to interpret and process. And, it is in fact true that the line between software and hardware to be somewhat blurry; take, for instance, firmware. Firmware exhibits the features of both software and hardware by being (a) machine-interpretable instructions, but (b) static and physically manifested. But, if software was hardware, how have we managed to construct complex and flashy edifices like Modern Warfare 3 on top of incomprehensible patterns of transistors? How can we copy and mimic the world so convincingly if we were not, at any point, permitted to employ the use of ideas, thoughts, morals &c? In philosophical terms, we have a real dilemma here: are pieces of software just physical parts of the universe, like hardware, or are they something more? Something… metaphysical?

The direction in which I’m now heading obviously seems perilously similar to the mind-body dualism dilemma that has been so numerously and irritatingly rehashed in philosophical writing, so it should appear welcome to the reader when I say this is not what I’m doing to discuss, but that I would like to take a look at what the alleged divide between software and hardware means to video games as an art-form.

The supposed software/hardware dualism mentioned above has everything and anything to do with video games. Video games feature both hardware and software in their constitution, making them an incredibly complex beast to understand when it comes to dealing with them aesthetically. Why? Take video game graphics/visuals as an example. Kotaku recently ran two articles about the graphical prowess of the next generation of PlayStation and Xbox consoles (Graphics Don’t Make Good Characters and Remember They Have More to Offer Than Just Graphics). Both of these articles make arguments to assert that something other than video game console processing power (if not complex and fantastic visuals, which is probably more on the mark) is also needed in order to make a good video game. This other element, character development, story-telling, plot structure/substance, is not as dependent as game visuals on hardware architecture–it is something that is idealistic; semantically speaking, this doesn’t mean the pejorative term of not being pragmatic, but something that is not physically tangible. A character or a story is an idea, a kind of ghost that inhabits the physical shell of an actor or a 3D hardware-rendered model.

Thus far one could say that we seem to have identified another famous dialectic here–albeit one that is potentially illusory, as physicalists (proponents of the ‘software-is-hardware’ philosophy, as I’ll call it) are wont to interject: hardware determined visuals versus almost entirely software-inhabiting story and character development. This kind of opposition seems to have existed since time immemorial–one can easily think of the way paperback novels transformed the way books were read. The same thing must go for video games in the league of Battlefield 3. The kind of artistic malaise in games that critics like Yahtzee on The Escapist identify is evidently not something new, and will probably never cease.

Indeed this fantastic article on the Gameological Society‘s website performs an assessment of the likely future of the mainstream video game industry within a similar framework to that being pursued here. Speaking in light of the mysterious launch of Sony’s PlayStation 4, the critical component of the article comes right at the end:

Creativity thrives under limitations. People who love games understand this implicitly, since the best players find the most creative ways to succeed within the confines of the rules. The Great Train Robbery is a masterpiece not in spite of its limitations but because of them. So if David Cage doesn’t think he can produce an emotional work of art with a PlayStation 3 and an eight-figure budget, maybe he shouldn’t be in the art-making business.

Expanding the technological capabilities of our game machines is not inherently bad, but treating new tech as a magic bullet is a self-destructive delusion (if a familiar one). The reason that so many games suck is not because the technology is too modest. The reason that so many games suck is because so many games suck. Making art is hard. No microchip changes that.

The really important thing about this quote (and the whole article, really) is not that it reinforces the idea developed above that artistic ideas are independent of their physical medium. The important thing mentioned in the quote is that it brings up the idea that artistic ideas can be constrained by their physical medium. It asserts that the medium through which art forms its expression sets ‘rules’ that artists need to find creative ways in which to work. Here we have a slightly more nuanced interpretation of the software-idealism position, which mixes in a little bit of physicalism: hardware determines the boundaries of software’s concepts/ideas/thoughts/substance. One could almost call this ‘soft software determinism’ because a more radical version can immediately be imagined: hardware outright determines the substance of software. Against software determinism you obviously have software indeterminism (or free-will &c) which asserts that hardware and software aren’t causally related.

Enter Jet Set Radio

It’s important to note that the software determinism position is related to the software physicalism question because the software-as-hardware (physicalist) position directly bears on how software and hardware are causally related. If software is just hardware, its realistic-seeming appearance is just a clever arrangement of atoms and magnetic charges. This will become important later on.

In any case, consider Jet Set Radio on the Sega Dreamcast. Its artistic direction was so influential that it inspired a virtual (in the potential and not epistemological sense) artistic movement in video games. JSR legitimised the use of cel-shaded graphics at a time when realism was a dominant driving force in the artistic value system of the video game industry. Realism obviously still remains an important way of going about and making games today, but if it wasn’t for games like JSR (or Windwaker–I wonder if Nintendo saw JSR and decided to copy its visual style?), we’d be left with an overwhelming corpus of mainstream games trying to copy the visual appearance and mechanics of the real world.

Humouring our determinism discussion, we can arguably put JSR‘s cel-shaded graphics down to the limitations of the Dreamcast. See this useful comparison of the raw processing power of the Dreamcast to the Nintendo GameCube and the original Microsoft Xbox. I find it so illuminating that I regularly go back and re-read it to make sure I haven’t missed anything that it might have said. The particularly interesting point it raises about the very modest increase in RAM and, to a lesser extent, CPU processing power relative to the other hardware features of the Dreamcast in comparison to the GC and Xbox is thought-provoking. The Dreamcast‘s unique Z-buffering technology is also interesting.

cel-shading1Cel-shading is most importantly related to the lighting in a 3D-rendered game. It replaces what would otherwise be gradients of light in shadows on models with a larger incidence of solid colours. One can easily connect this to the way in which a game engine would deal with textures (something the Dreamcast was, on paper, noticeably weaker at handling than the other two consoles), and what you have is, like John Teti said in his Gameological Society article, a creative way to succeed within the limitations of Sega’s (admittedly cool) console. Or, more intriguingly, you have the hardware design of the Dreamcast subtly dictating to Smilebit how they should make their game.

What about Gauroud shading on the N64? It was a solution to the incredible challenge posed by the cripplingly small texture cache on the platform. The N64 had its fair share of games with realistic graphics, but the fact that it struggled with complex textures meant that programmer’s favoured method of dealing with this–shading–lead to better performance with cartoon-ish visuals that dealt with simple colours.

More broadly, consider the transition from 2D to 3D graphics. Games like Tomb Raider were unthinkable in the late 80s–the hardware processing power simply didn’t exist to render such complex environments. Games are more or less forced to resort to 2D graphics when being based on more limited hardware. The kinds of experiences 2D graphics can deliver can be brilliant substantively/qualitatively, but they must be far more limited formally than those based in three dimensions: Zelda-esque RPGs, platformers, 2D RTS games. Indeed, search and you will find that 2D platformers by far outnumbered games published in any other genre (much like first-person shooters today).

Zizek and The Matrix

Slavoj Zizek’s Parallax View gives us a good schema to not only deal with the ideal-software/physical-software dialectic, but also that of the determined-software/undetermined-software simultaneously.

In an extended passage in which Zizek discusses the operation of ideology in today political environment, he performs an analysis of The Matrix series. Following his Lacanianism, he works out that the function of the matrix is based on a kind of perversion. The perversion on which the matrix is founded is two-fold:

  1. Perceivable human reality is reduced to a virtual domain whose rules can be suspended. It’s theoretically possible for someone to do or have anything in the matrix.
  2. The concealed truth of this apparent freedom is that humanity is actually the perfect kind of slave. Ultimately passive and instrumentalised, humans are farmed for their ability to generate electrical energy.

This dialectic corresponds to, and informs that of the ideal/physical and determined/undetermined ones. Humans are not only divided into ideal and physical components in the matrix, but they are also controlled and possessed by it, in exactly the same way that a piece of game software is with its corresponding hardware platform.

The perfect conception of software as an idealistic form is that which is totally independent and undetermined by its hardware base, free of any kind of limitation or constraint. The best kind of illusion imaginable, any kind of creative desire could be pursued with this form of software: characters of infinite depth, plots infinitely complex and intriguing. The complete opposite of this would be software as a ‘slave’ to a dominant and ‘parasitic’ hardware base. Instead of being a perfect dream, software would really just be a host from which for hardware to ‘feed’. This image isn’t meant to be literal, it means that hardware is just exploiting software in an attempt to ensure its survival. A good example of this would be the way iPhones are propagated by Farmville, and other kinds of ‘casual’ games (the marriage is perfect, isn’t it? A device that is always with you, always requiring your attention). Some more examples are the way operating system update schedules work, always requiring you to keep your device ‘up-to-date’, and the way console launch titles work–to make their console seem appealing.

The way The Matrix solves this dilemma (ultimate freedom/ultimate slave) is by giving Neo the ability to practice the powers he has in the matrix outside the matrix. Zizek quite rightly regards this method of synthesising The Matrix‘s virtual and real worlds as insufficient–much like revealing at the end of a detective novel that the murderer has incomprehensible magical powers. It merely conflates the original premises of the dilemma by trying to confer upon human-kind, in its slave-form, the virtual freedom it is able to possess in the matrix.

Zizek’s preferred solution to The Matrix dialectic puts us on the right track to conceiving a coherent relationship between games and their platforms: why didn’t humans try to sabotage the matrix by refusing to secrete any more energy into it? The important thing about this possible solution is that it is negative: it destroys both humanity’s slavery by the evil machines, and its ultimate freedom in the matrix. What is left is just good old, really-existing humanity–which is in fact a very complex way of arriving at the same conclusion as John Teti: software is partly constrained and determined by the limitations of its hardware, and is in reality physical.

But by performing all of this work, while we rather uninterestingly return to the position that software does have an existence that partly transcends its physical determination (the form that this transcendence takes isn’t of immediate interest), we learn that software needs that obstacle. The idea isn’t the mundane truism that software needs hardware in order to run, but that hardware presents a game designer/developer/programmer with a mould that needs to be broken out of. A good example to demonstrate this point is the amount of first-person open-world games that proliferate our present gaming culture. We’ve reached a point in our gaming hardware development that allows us to render huge worlds, but they’re not necessarily fun or interesting–it’s basically ‘filler’. As Teti points out, Jonathan Blow’s upcoming game The Witness has attempted to make its game world as compact and as carefully constructed as possible. Like Teti, I regard this as good design, and good art. Blow’s piece is evidence of getting over the obstacle of taking orders from one’s hardware base.

Think about some games you know that did their best to transcend their hardware limitations. A few that I can think of are Exhumed for the Saturn, Super Mario 64 (compare with the horribly linear Crash Bandicoot series which stole SM64‘s hub-world concept–NB that SM64 was a launch title!), id software’s Doom and Quake, and the first two Pokemon generations.

Tagged , , ,

Virtua Racing

virtuaracingmegadrive5

I’ve always wanted to square off Virtua Racing against Starwing. That’s a much bigger job than just reviewing either one of them individually, and since I lack the time to immediately that, and, having been playing a bit of Virtua Racing recently, I thought I’d forgo my original plans to review it alongside its early home console 3D-graphics competitor. In its home console form, Virtua Racing is obviously important for two reasons, the first being its conceptual import, and the second its innovativeness in terms of its use of hardware.

Previously, racing titles relied on a carefully constructed two dimensional illusion of distance and depth of environment in order to convey a plausible game scenario. While VR wasn’t the first 3D racing game to do away with these techniques, it was the first to do it well. What was definitely not lost in VR, and indeed continues to be an important concept in racing games after their transition to three-dimensional environments, is its persuasive illusion of speed.

See that dot on the horizon? Yeah. That's your Badiouian Truth-Event.

See that dot on the horizon? Yeah. That’s your Badiouian Truth-Event.

I didn’t find VR‘s Acropolis level to be particularly well-executed level in conceptual terms, but its intermediate-difficulty level, Bay Bridge, a shining example of what Sega and Yu Suzuki were trying to achieve. While VR‘s environment can be divided up binarily, interactively speaking, into obstacles and non-obstactles, Bay Bridge demonstrates the kind of raw, intuitive feeling that you can only get from a (what was then) realistic three-dimensional environment. On the level, you pass through the namesake steel bridge, which is enormous. It towers over you, and the feeling of that empty column of space stays with you the rest of the level. You pass through a mountain in a tunnel, underneath huge freeway-like concrete manifolds, and alongside imposing grey retaining walls. The level design and eye-candy work together to drive home this concept of speed and transit in a very convincing manner.

virtuaracingmegadrive3

The only time this breaks down is when you crash, or, through naive level design, become aware that–much like any 2D racing game, Hang On and Out Run being famous Sega examples–basically all of the world is proceeding towards you from a point in the middle of your TV screen. At these two moments you’ll realise just how much VR has inherited from its ancestors, as it will become patently apparent that the car onto which you’re projecting your will, and the environment around it, are not inhabiting the same plane of existence: the player’s car, at times, feels as if it is being drawn on top of everything that’s happening on the screen, like it’s not really ‘there’ on the road. This should all be regarded as forgivable, because, first, most of the time, the player’s car feels flush with the road, and secondly, this criticism is minor gripe. Seeing as the technology powering VR was very, very new, the fact that they managed to pull off a game as atmospheric and realistic as they did is a real achievement.

virtuaracingmegadrive1The second important thing about VR, its use of hardware, is also something worthy of praise. I think it speaks to the great flexibility of the cartridge format that game manufacturers were able to incorporate secondary processors into their games in order to better realise their creative ideas. Today, one is more-or-less stuck with the hardware they have when they come to possess a new game title, but that was not necessarily so with cartridge-based software. Cartridges could–and did–contain extra RAM, or special memory bank switching hardware (think the NES) in order to push the envelope in terms of sprite and background complexity. As with Nintendo and their Super FX line of chips, Sega made use of a secondary microprocessor to handle the calculations necessary to display 3D visuals on their fourth-generation console. What distinguishes VR‘s special chip, the ‘Sega Virtua Processor’ (good old-fashioned marketing spin at work here) from the Super FX is that it was not a full, multi-purpose microprocessor. It was a Digital Signal Processor (DSP).

virtuaracingmegadrive2A DSP is microprocessor that is custom-built for a very particular purpose, which is frequently a very simple and repetitive, but power-hungry task. That’s not to say that all custom-built microprocessors are DSPs, but in this particular case, the “SVP” was designed to perform two operations (multiplication and addition) for the purpose of calculating VR‘s 3D polygonal data, and transforming it into “8×8 tiles” (which one can assume to be 8×8 pixel patterns). These two-dimensional tiles are transferred, ultimately, to the Mega Drive’s video RAM via Direct Memory Access (which is only possible through the use of cartridges!).

You could be forgiven for thinking that the SVP stood in the same league of semiconductor fidelity as the RISC Super FX chip, because, short of performing some sort of hard empirical comparison between VR and Starwing in terms of their polygon-count (VR displaying somewhere between 300-500 polygons @ 15 FPS), early alternative to Z-buffering (see Quake’s Binary Space Partitioning), use of colour, and frame-rate (&c &c), the visuals that both games push are comparable. Honestly, whenever I pick up VR I forget that the SVP is only capable of rendering polygons on-screen in 16 (!!!) colours.

starfox venomThere may be something to be said for Starwing‘s better player game-environment interactivity and command of atmosphere (NB Starwing‘s final level), but, short of performing the all-out compare-and-contrast disclaimed above, Virtua Racing is an absolute stellar title. I’m asserting this despite the fact that I don’t particularly like racing games as a genre–as a game in itself, the world environment that this title manages to convey to the player is reasonably deep and tangible. While I haven’t spoken about the game’s controls at all, its scheme of interactivity is solid. In that manner, it doesn’t fail as a racing game.

The great thing about VR is that it is able to be acquired on eBay and other retro gaming shops on the internet for ten to twenty dollars. That’s a far cry from the game’s original asking price. With Mega Drive systems being correspondingly so durable and cheap, it’s worth getting both and learning a bit about the history surrounding the video game industry’s fits and starts into the 3D graphics era. Retrogarden puts it pretty well when it says that

the SVP chip brought hope to die-hard and downtrodden Sega fans; if Virtua Racing was possible, why not Virtua Fighter too?  Could 3rd party developers  take the SVP chip and use it to port Doom, or even make a Starwing-type game?

The SVP represents the highly contingent nature of the aesthetics of the Western video game industry from the mid-nineties onwards. The progression to more and more complex foundations of hardware didn’t push more complex artistic ideas with it, as disillusioned Sega fans can surely attest. While VR offered a glimmer of hope, the future wasn’t secure as the industry moved into the mid-to-late nineties, as everyone is so obviously aware.

For a brilliant in-depth hardware analysis of Virtua Racing (which Jump’n Shoot will probably exploit later) on the Mega Drive, see this blog post.

For more information about the “Sega Virtua Processor” (a Samsung SSP160x Digital Signal Processor), see this forum thread.