One of the interesting cultural by-products to have emerged from the American assassination of Osama bin Laden is the public’s sudden fascination with situation rooms. As noted by Alexia Tsotsis on TechCrunch last week, the photograph of Obama and his national security team taken during the raid has received millions of page views and inspired both serious commentary and dumb meme photoshop tomfoolery. Additionally, given the gravity of the action and the secrecy that allowed it to be executed so seamlessly, the media has revelled in celebrating every minute detail of the planning and management of the operation.
I’d been meaning to plug a project proposal by the Argentine artist Gustavo Romano that will be developed at the upcoming Visualizar’11 workshop in Madrid. Considering its mandate, it is undoubtedly the perfect moment to discuss this venture and I think it will serve as a useful point of entry into a related discussion on the visual representation of conflict and power relations.
Given the theme for this year’s edition of Visualizar is “Understanding Infrastructures”, the projects that have been selected for developmentexamine a range of supply chain, consumption and finance-related topics. The above image represents Gustavo Roman’s proposed The Psychoeconomy War Room Table (PWRT), a tangible interface for exploring global economic data. PWRT will utilize the open source computer-vision framework driving theReactable Live! musical instrument to create a collaborative workspace for exploring (quantified) international relations. Roman outlines the goals for his project as
… try[ing] to display the relationship between two or more countries in the world in terms of some specific social and economic variables. The proposal builds on the metaphor of the table of the War Room, the room where are discussed possible tactical moves in a military confrontation. Using a multitouch surface,reacTIVision and fiducials, persons using the table may choose to place different flags on stage of the global economy and they can visualize the relationships between these countries. We will use data related to flows of assets (goods and financial capital), human flows (migration and tourism), energy flows (fuel and food), information flows (corporate media and alternative media). We will collect the data from public websites like the UN, the IMF, the World Bank, the CIA, etc.
I think this is an utterly brilliant idea and look forward to seeing the proof-of-concept interface that Roman and his team prototype next month. Imagine being able to manipulate the contents of the World Bank data catalogue and browse the interdependencies between nations – it would be both engaging and illuminating. Given the project is currently only an elevator pitch, I’ll make a point of mentioning the work that emerges when the results of the Visualizar workshop are posted online later this summer. It is worth noting that PWRT is part of Roman’s larger Psychoeconomy project, an artistic platform for exploring global issues.
For those within striking distance of Madrid, you might consider applying to work on PWRT or any of the other selected projects. Medialab-Prado is accepting applications through June 12, the workshop runs over the second half of June.
The still of Dr. Strangelove in the lower right corner of Roman’s PWRT collage gets me thinking about how depictions of these strategic ‘command’ spaces have evolved over the decades. The above image is a still from Joseph L. Mankiewicz’s opulent 1963 film Cleopatra that hypothesizes what ‘real time naval imaging’ might have looked like during the Battle of Actium (31 BCE). For those unfamiliar with the downfall of Mark Antony, the military leader sailed his flotilla into a trap set by his nemesis Octavian (who had obtained vital intelligence from a defecting general). As Antony’s naval forces were decimated the strategists remotely monitoring the battle at their ‘war room table’ set the appropriate ship models aflame. The interesting thing about this fictional case study is—as it is pre-screen—it functions as a tangible interface…
…and the scale models of Cleopatra bring us back to the inescapable reference of wargaming. The cleverest of the anonymous internet situation room photo edits was a tight crop of the intensely-focused Obama wielding a Playstation controller alongside a Brigadier General hunched over a laptop; drone mishaps notwithstanding, perhaps this is our caricature of warfare for 2011? The absurd addition of a gaming controller brings to mind a 2006 sound bite by Henry Kissinger where he described the (pre-makeover) White House situation room as “uncomfortable, unaesthetic and essentially oppressive” – in this image, wargaming is pure playbour.
I still contend that the most engaging game mechanics that I’ve encountered is the visualization of empire in Sid Meier’s Civilization series. The above screen capture is the city management interface from Civilization V, where you see the players’ borders butting up and flowing around those of their neighbours as well as detailed informatics that reveal the yield of the landscape and chart out where expansion will occur. There is sufficient depth in Civ that competing nations are required to develop extremely nuanced trade and diplomatic relations to acquire needed luxury items and natural resources while forging strategic alliances. However, the game kind of falters in failing to represent these complex flows of goods, materials and capital visually – at times it can be quite difficult to determine exactly what is going on. This is why I’m so fascinated by the PWRT as it aspires to provides a handy interface for exploring the global economy as a field of vectors rather than relying on stale geographic representations of borders and trade routes. In Newsgames, Ian Bogost, Simon Ferrari and Bobby Schweizer describe this kind of open engagement as an exploratory graphic that
…shows data that is meant to be synthesized by the user independently of the creator’s expectations. Both [Edward] Tufte and Benjamin Shneiderman encourage the use of information graphics to offer multiple levels of granularity for maximum flexibility. Tools or controls allow the reader to arrange, filter, or zoom data.
Can the complexity of international relations be distilled down to a work surface or game environment? Presumably, but exploring this kind of data can only be as revelatory as the interface it is delivered in.
About this article: The Psychoeconomy War Room Table (And Other Situational Awareness Vignettes) first appeared on serialconsign.com on 2011-05-11.
About the Author: Greg J. Smith a Toronto-based designer and researcher with interests in media theory and digital culture. Extending from a background in architecture, his research considers how contemporary information paradigms affect representational and spatial systems. Greg is a designer at Mission Specialist, blogs at Serial Consign, writes a column on emerging technology for Current Intelligence and is a managing editor of the digital arts publication Vague Terrain. He currently teaches in the CCIT program (University of Toronto/Sheridan College) and at OCAD University.
- On simulation, aesthetics and play: Artifactual Playground In 1958, the American physicist William Higinbotham created what is one of the first instances of what we would today call a modern "video game". The game, named Tennis For Two, was built at the Brookhaven National Laboratory for their yearly open-house presentations of the lab's activities. The game was built using an oscilloscope and a programmable analog computer, the Donner Model 30. It simulated a simple tennis match between two players, with a sideways perspective of the net and a ball bouncing back and forth, controlled by two player-manipulated inputs. _ William Higinbotham, Tennis For Two, Brookhaven National Laboratory, 1958 Although it would take a few more years, namely 1962 and the game "Spacewar", before we could see the emergence of a true modern form of "gameplay", "Tennis for Two" nevertheless contains enough basic elements of interactive play to connect it to more contemporary descendants, for example the iconic Nintendo hit, "Wii Tennis". While there are a few missing details here and there, such as avatars, scoring and the various forms invented to interact with the machine, fundamentally there is very little that has changed since "Tennis for Two". It contains all the modern tropes of animated algorithmic representation, namely a highly kinetic visual form that emerges in real-time from within the game via its gameplay. From this perspective, it is one of the forebears for "arcade" style games. The game is fast and dynamic, and only by interacting with the system does the image emerge. But perhaps most importantly, "Tennis for Two" is significant in that it is not only a representation of playable interactive visual forms, but that these forms represent something greater than their graphical output: the game is in fact a physics simulator of a ball moving through space and interacting with objects in its path. Watch how the ball bounces against the net and then try to imagine what it would take to program such a movement, even today; then remember that Higinbotham was working back in 1958. For its time, this is a sophisticated simulator of physical interactions: "The 'brain' of Tennis for Two was a small analog computer. The computer's instruction book described how to generate various curves on the cathode-ray tube of an oscilloscope, using resistors, capacitors and relays. Among the examples given in the book were the trajectories of a bullet, missile, and bouncing ball, all of which were subject to gravity and wind resistance. While reading the instruction book, the bouncing ball reminded Higinbotham of a tennis game and the idea of Tennis for Two was born." — Brookhaven National Laboratory, The First Video Game?, p.2. In other words, Tennis for Two was not only the first "Pong" game, but also the first physics game, à la Box2D and its shameless re-branding in the infinitely more popular form, Angry Birds. And like Angry Birds' relation to Box2d, the underpinnings for the game "Tennis for Two" were already inscribed in the routines of the machine itself, the Donner Model 30. These routines were then re-contextualized using what we would today call "joysticks" and voilà: a modern arcade game. Wii Dog vs Wii Cat & Angry Birds Live, T-Mobile Given the historical context, there is nothing surprising in this idea of a computer simulating a physical phenomenon such as a bullet or a missile. In the 1950's, computers were still emerging from World War II era cybernetic formulations of "telelogical" or "self-regulating" machines, precipitated in large part by the acceleration of faster and faster flying weapons that required new techniques for shooting them out of the sky (cf. V-2 Countermeasures). The history of interactivity is traversed by this question of simulation, i.e. by the idea of adaptive mathematical and physical models that could allow machines to regulate themselves in real-time, based on constantly evolving conditions. So while it might be considered a historical curiosity that post-war cybernetic machines would produce the modern video game, it is unsurprising that such a game would be constructed out of a physical simulator of bouncing balls or flying bullets and missiles. Aesthetics, Simulation, Play The historical relationship between aesthetics and play has always been a complex one. There is much overlap and interpenetration, but they are in no way interchangeable terms. Most performative art forms, such as theatre or music, oscillate constantly between the ludic and aesthetic realms. In the work of art-game pioneer Eddo Stern — for example his work with C-Level, or his newer Wizard Takes All — we can see these two domains interact with one another in a contstant back-and-forth that suggests perhaps a more fundamental genealogy connecting the two. But despite the deeply connected roots, they are nevertheless two expressive forms that cannot be conflated, all the calls for games-as-art be damned. But whatever the relationship between aesthetics and play, it is further complicated by this introduction of the principle of simulation in play, made all the more acute in the context of video games. Simulation questions the mimetic tendencies of representation, which might explain in part the constantly recurring uproar over violence in video games (and all the ire over provocative gamer-artists that apparently "hate freedom" ;-). But no matter how small-minded the complaints, people nevertheless understand that these games are not merely mimetically presenting us with representations of violence; instead, they are directly modeling the violence itself of the scene. The resulting image flows from the model; it is a "rendering" of the underlying scene. This is the specificity of simulation: the ability to represent the dynamics of a situation as itself a form of representation. The representation needs to be played in order to take form. This is the historical twist of simulation: the image has shifted from a predominantly mimetic function of re-presentation to that of rendering complex interactions visible through playability. In fact, simulations can take place through other mediums and channels of perception. The American far-west simulator, The Oregon Trail (1971), for example, was a simulator that originally used only textual communication to represent the state of the game. Although modern variants of The Oregon Trail, such as Red Dead Redemption now use sophisticated graphics to represent the game state, the game is nevertheless animated by a simulation engine that cannot be be reduced merely to the artifacts displayed on-screen. The Oregon Trail (Apple II edition), 1971/1984 & Red Dead Redemption, 2010 A Poor Man’s Simulator The quality of the simulated movements of the Higinbotham/Model-30 ball and its interactions with the net are impressive, especially when compared to the clunky, almost weightless movements of Pong, designed some fifteen years later. If there were so many games about space in the 70s and 80s, it might be because earthbound physical simulations are hard to design and certainly hard to calculate in real-time, especially when you've moved from analog computers to digital ones. Physics are a mostly logarithmic, analog realm, and are hard, or long, to calculate using digital circuitry. Although many games with bouncing balls and gravity would appear throughout the next few decades of digital gaming, it would truly take Erin Catto's Box2D and accelerometer-based controllers like the Wiimote and the iPhone for the form to emerge as a fundamental gameplay mechanic. Why so early then our first variant on what would later become Angry Birds? The prophetic nature of Tennis for Two can somewhat be explained by context: Higinbotham was a physicist, whereas Pong’s inventors — Ralph Baer (Magnavox) and Allan Alcorn (Atari) — were engineers. Higinbotham was working with scientific instrumentation that did not adhere to the economic constraints or objectives of Baer who was for his part trying to design mass-producible circuitry that could be plugged into to millions of customers’ televisions. But it is precisely this poor-man's quality of video game's simulators that helped emerge the ludic qualities of gaming. Tennis for Two is frankly a little boring next to Pong, whereas Pong remains one of the best-designed games of all time, giving birth to an infinitely expanding field of variants all the way from Breakout to Bit.Trip Beat. Ralph Baer and Bill Harrison Play Ping-Pong Video Game, 1969 & Bit.Trip Beat, Gaijin Games, 2009 One of the ironies of video game history relates to this desire to simulate infinitely complex interactions, but with access to only the most mediocre means of calculation. This contradiction has led to what might in some senses be considered an historical anomaly: an in-between period in which computer games’ desire for "realism" would have to wait for the technological means to catch up. A Poor Man's Renderer This anomaly relates not only to the simulation itself, but also to the manner in which it is rendered to the screen. In this in-between period of video game design, situated somewhere between the late 1960s and Box2D (circ. 2006), a cornucopia of visual forms emerged from video games that have given games their distinctive identity as an aesthetic form. We now identify video games as much by their visual artifacts, as by their particular form of gameplay. A truly innovative game will in fact design a specific form of visual artifact, in order to better match the gameplay, outside of any criteria of realism. This approach will often go on to trump the simulation itself and become the central mechanism of gameplay. It is precisely because of the technological limitations of early gaming technology that gaming eventually found its singular language of representation where the graphical artifacts would themselves become the playable form. Artistic Playgrounds This playable visual language has even circled back around to influence various forms of visual communication, in order to make them more "playful". And artists for their part have used this visual language of computer game artifacts to transform less electronic contexts into playable forms. The list could go on almost forever of artists working in this space: Mary Flanagan, Aram Bartholl, Damien Aspe, etc. In the well-known work of French artist Invader, the city landscape becomes a platformer to be traversed literally, leaving behind physical pixels: Invader Sneakers & Space Invader in Shoreditch, London In the aforementioned Eddo Stern’s "portal" sculptures, gaming logics of representation and interaction are re-projected back onto traditional spaces of representation (gallery, public square, etc) in the form of sculpture: Eddo Stern, Fake Portal, 2012 While neither of these examples are even playable as games, they communicate nevertheless with the video game medium through this imperfect, unrealistic video game form of visual rendering. They look and feel like classical electronic forms of play. The artifactual visual language of video games is sometimes constructed out of a patchwork of various historical forms that have been redefined through the filter of gaming. Sometimes video games skeuomorphically imitate previous technologies and mediums, for example by flashing television-style signal noise to signify a weak connection, or imitating hand-written messages and drawings strewn about a 3d world (cf. Myst, Resident Evil). But video games have also introduced their own domain of visual logic based on the specific contours of the technological limitations that animate them. Often a closer reading is required in order to reveal the nature of these contours. Raster-Scan A strange by-product of the historical anomaly can be seen in the role of the pixel in video games. Originally, as was the case with Tennis For Two, games were built with vectors, as were many related visual technologies such as Ivan Sutherland's Sketchpad. In fact, Tennis for Two used vectors for both the simulated phenomena (force, velocity, etc), as well as the physical image constructed within the oscilloscope. This is completely logical if you're looking to construct a physics simulator. This vector-based approach is also the case today, where games are often built out of polygons which — assembled together — construct the playable scene. But somewhere in between Tennis for Two and our modern-day graphics pipeline, came the pixel. And this anomaly, the pixel, continues to this day to influence profoundly the manner in which even vector-based images are rendered to our eyes. Alan Kay, The Early History of Smalltalk, 1993 Like many of the computing concepts we take for granted today, the pixel concept was perfected in the late 60's and early 70's somewhere between Douglas Engelbart's Stanford Research Institute and the Xerox PARC in neighboring Palo Alto: "The TX-2 display that Ivan Sutherland used for Sketchpad [...] would project a single bright spot on a dark screen and then electronically move that spot around to trace out a circle, say, or the letter A. By tracing and retracing the pattern very, very fast, [it] could create the illusion of a solid outline. [...] The problem was that the more complicated the drawing, the faster you had to wiggle that spot. [...] Then there were the "raster-scan" displays that Bill English had developed for the "PARC Online Office System", POLOS. [...] The POLOS displays used digital electronics that were better suited to the binary world of computing: in effect, they would divide their screens into a fine grid of "pixels" and then make a picture by turning each pixel either on or off, as appropriate, with no shades in between. [...] The programmers would have a much easier time devising graphics software to generate those images, because all they had to do was define a chunk of computer memory to be a map of the screen, one bit per pixel, and then drop the appropriate bit into each memory location: 1 for white and 0 for black. [...] Unfortunately, that use of the computer's memory was also the major difficulty with bit-mapped graphics: memory was very, very expensive in those days." — The Dream Machine. J.C.R. Licklider and the Revolution that Made Computing Personal, W. Mitchell Waldrop, Penguin Books, 2001, p.366. In many ways, "bit-map" graphics are simply a historical hack used to generate text and images dynamically on a screen. In the case of the heavily text-centric Xerox PARC machines, one would assume that a more vector-based image generator would make more sense: typography is essentially a history of shapes built out of lines, with a visual language heavily influenced by the traits of handwritten letterforms. In fact, it took some thirty-odd years, led by Apple's "retina" ultra hi-definition screens, for bitmapped text to match the quality of the printed page. So it could probably be argued that the "bit-mapped" approach was historically the wrong one, even if it is now somewhat catching up. Douglas Engelbart, Workstation With Mouse, Agumentation Research Center, cir. 1964-1966 & Maze War, Xerox Alto, 1974 But from a purely technological, engineer's perspective, bit-map images make all the sense in the world. In the above quote we need only retain that "the programmers would have a much easier time..." in order to understand why the pixel approach won out. Computers are "discrete" machines, capable of switching parts of itself off and on independently. This logic gives us random-access memory which in turn gives us databases, which in turn gives us things such as hyperlinks. Machine architecture influences use and to assume that this would not influence the resulting aesthetics is naïve. The infinitely re-configurable and re-contextualizing nature of the machine is the whole point of why we use these damn things. So an image construction method that would closely match this discrete logic, down to the very 0s and 1s of the machine's ABCs, was an important step in creating a "plastic" image, capable of reconfiguring itself multiple times per second. It is out of just such a type of image that video games as a medium emerge. Raster-scan vs. Vector-scan Let's compare two images from two iconic video games from 1980, Battlezone and Pacman. Battlezone is a vector-based game, and originally used a vector-scan method for displaying shapes on-screen. This created razor-sharp images, albeit in black-and-white, or actually black-and-green. The use of vectors also allowed Battlezone to be one of the first mass-market games to effectively represent a three-dimensional scene, using the first-person perspective of a tank commander to navigate the game space. It would be many years before a pixel-based computer system could anywhere approach the visual elegance of early 1980s 3D games such as Battlezone, Star Wars or Tempest. One of the great iconic raster-based 3D games, Castle Wolfenstein, wasn't even in 3D at its introduction in 1981; and even when it became Castle Wolfenstein 3-D in 1992, that visual representation was made up of large blocky pixel shapes, far inferior to Atari's 1980's graphical representations. But Battlezone's vector-scan technique also created some curious visual anomalies: for example objects on screen were fully transparent, defined solely by their outlines without any possibility for image "textures" to fill in the gaps. This created the odd situation where an enemy tank could be seen transparently on the other side of an obstacle, but could not be shot at. In a sense, this improved the gameplay and created part of the strategy of playing Battlezone — no matter what level of realism it achieved as a simulation. Ultimately, it was a game made for fun, for play, but even so it would eventually be used by real tank commanders as a training simulator for their soldiers. The simulation was good enough so as to be a functional form of training in the real world manipulation of tanks. Visually, Pacman (a.k.a. Puckman) is a very different animal. Contrary to Battlezone, or even the more-colorful Tempest, Pacman is practically drenched in color. Ghosts are brightly-colored with different hues based on character traits, allowing players to read their individual algorithmic behavior within the game. The player's character, Pacman, is a completely opaque bright yellow animated blob, full of visual charm. Like the ghosts, he is full of personality. Color is even used as a gameplay element, allowing players to distinguish between dangerous ghosts (multi-colored) and edible ones (blue). Everything about Pacman screams "bit-map" techniques: the maze is a series of bit-mapped 0s and 1s, turned on or off to represent a wall or a navigable open space. And the dots or crumbs that we eat are also represented as a bit-map, i.e. a scattering of pixels that we have to turn off by running our character over them. In Pacman, the gameplay, in fact the whole game algorithm, is directly controlled by the graphical representation, as opposed to Battlezone where the graphical representation is often in contradiction with the physical simulation of interaction with physical objects. Pacman is a collection of pixels, he lives to eat other pixels, and the level is over when there are no more pixels to be eaten. Pacman essentially spends his time running around a memory map until he has effectively manipulated all the memory registers by setting them all to 0. The internal circuitry of the machine is visually exposed to the player who is then asked to navigate into this memory register map and manipulate the digital switches via an on-screen representation. Cellular Automata While it is not technically a video game, and was in fact designed as a scientific simulation experiment, John Conway's Game of Life is nevertheless one of the best examples of one of these immanent pixel-plane spaces from which a "playable" image emerges. The "game" is played entirely by comparing one pixel to the pixels that surround it: too many surrounding pixels, the pixel dies from overcrowding; too few, it dies from lack of resources; and from just the right number of pixels, a new pixel is born (if none) or survives (if already alive). The visual representation of the life "game" is exactly the same map of values as the memory registers that control it. There is no representation of the simulation outside of the frame of the grid. Based on this immanent principle, a complex interaction of forms emerges, hence the term "game of life". Conway's Game of Life, 1970 & Runxt, R-Life for iOS One of the best known games of all time, Sim City, was directly inspired by this Conway thought-experiment: "[John Conway's work] is so extraordinary, because the rules behind it are so simple. It's like the game Go. [...] They can arise from fairly simple rules and interactions, and that became a major design approach for all the games: "How can I put together a simple little thing that's going to interact and give rise to this great and unexpected complex behavior?" So that was a huge inspiration for me." — The Replay Interviews: Will Wright, Gammasutra, 23 May 2011. In Conway's Game of Life as well as Wright's Sim City, the immanent pixel grid is the space itself of the "game", conflating both the pictorial representation and the simulated one. It is the "map" upon which the simulation of SimCity, an architectural construction if there ever was one, would be built. Animation Another significant trait found in pixel-based games such as Pacman, far more absent in vector-based games, is the narrative dimension. Pacman tells a story, and even introduced comedic interludes every few levels, telling little Keaton-esque sketches of Pacman being chased by ghosts and then turning the tables to chase the ghosts in turn. Pacman cutscenes, arcade edition 1980 & Atari 800 edition, 1983 Many interactive characters were built out of these basic, often extremely limited, collection of "bit-map" pixels: the whole Pacman family (Pacman, Ms. Pacman, Pacman Jr., etc), Mappy, Dig Dug, Mr. Do, Mario, et cætera. Even known animated characters — such as Popeye —, found their way into the heavily pixellated game screens of the 1980s. There is nothing arbitrary about this use of cinema-animation logic aesthetics to animate the characters of early video games. For animation had already solved this problem of opening up cinematic figuration by eschewing realism and embracing the artificial nature of the image. Gerty the Dinosaur, Betty Boop and Felix the Cat, all the way up to La Linea and Don Hertzfeldt's pencil-drawn absurdities: these are all forms of reduction down to the visual interaction of a few basic visual forms. So too in video games: the key to their success in adding expressive characteristics came not from the militaristic, cybernetic-inspired scientific simulation instrumentation. Instead, it came precisely from embracing the abstract, graphical, nature of their primitive cousins and in accepting the artifactual, visually limited detail of the early digital machines. In accepting this fate, video games tapped into a deep tradition of expressive visual tapestries that had been explored throughout the 20th century in cinema through the work of experimental film-makers and animators such as Len Lye or Norman McLaren, using simple abstract shapes such as lines, scratches, and blobs of color to great expressive effect. Vanishing Points Although the term is a bit dubious, we are exploring here the problem of realism, or perhaps more specifically that of mimesis, i.e. the art of imitation. A significant historical component to this debate on art and realism relates to the introduction of a very specific form of pictorial representation: geometric perspective of the sort demonstrated by Brunelleschi in the early 1400s. In our parallel history of video games — notably as it traverses its naive period of representation —, we as well can see some interesting effects of perspective as it relates to how images are constructed on-screen. Due to the purely arbitrary nature of the discrete pixel grid where any section can be turned on or off at will, a strange form of mixed perspective becomes possible with multiple forms of perspective not only co-existing on screen but even interacting with one another. Pacman and the ghosts within the maze are completely devoid of principles of foreshortening and vanishing points, and are in fact a mixture of top-down vertical perspective (of the maze), and side-view perspective (of the characters) reminiscent of early forms of perspective emerging in the work of Giotto where, to take an observation from Deleuze & Guattari in Mille Plateaux (p.219), Christ alternates between divine receiver, enduring the stigmata, and kite-machine, commanding the angels and heavens via kite strings. The emerging nature of the Brunelleschian-style of geometric perspective is not fully developed at the time of Giotto, hence the optical oscillations for a modern eye between flatness and depth, foreground and back, and so on. Jesus is at once commanding Saint Francis, and simultaneously being flown by him like a kite. It is only through narrative cues, understood by semiotically reading the painting, that we are able to reconstruct these spatial relationships between the various figures. Like many paintings from the middle ages to the early renaissance, perspective in early video games contain multiple points of view and often chooses its perspectival representation based on contextual narrative needs. These are naïve and/or mixed perspectival geometries (cf. Tapper, Zoo Keeper, et al.) that have recently been exploited to brilliant effect in Polytron's visual delight, Fez. Tapper, Bally Midway, 1983 & Fez, Polytron, 2012 We could also mention Game Yarouza's Echochrome where the gameplay takes place somewhere in between the OpenGL pipeline where vector data is rasterized into pixel data and itself becomes a gameplay mechanic as players exploit visual absurdities and try to line them up. Echochrome, Game Yarouze, Japan Studio, 2008 Such hybrid forms of perspective would have been much harder to acheive had gaming stuck with purely vectorial and mathematical forms of representation. Visual abstractions It might be temping, based on such an art-historical exposition, to start comparing video games to the history of art and graphical design. For example, it would be fairly easy to visually juxtapose the paintings of Piet Mondrian/De Stijl, with Taito's 1981 arcade classic, Qix: Piet Mondrian, Composition 10, 1939–1942 & Piet Mondrian, Composition II in Red, Blue and Yellow, 1930 & Taito, Qix, 1981 Obviously, on some level there is a visual inheritance taking place, either explicitly, culturally or unconsciously, even though such causalities are either impossible to prove or even, if true, merely anecdotal. Another juxtaposition might be to look at the Russian avant garde, starting with El_Lissitzky, and compare his visual language with the shapes and forms of more abstract forms of video games, including early 3D games that had not yet perfected their perspectival rendering engines: A Prounen, El_Lissitzky, c.1925 (cf. Prouns) & Sixty Second Shooter, Happion Laboratories, 2012 Blaster, Williams 1983 & Ballblazer, Lucas Arts 1984 Rez, Tetsuya Mizuguchi, 2001 The problem, ultimately, with all these approaches is that these are merely visual cues and not aesthetic ones. The problem with just such a visualist reading is that it assumes that both De Stijl and Taito constructed their representations purely as visual tableaux — in other words as just a bunch of pretty pictures —, instead of looking at the material, conceptual and historical visual languages and logics that might have led them there. In the case of Qix, it would probably be far more instructive to compare its geometric abstractions to early MacPaint software, and Bill Atkinson's visual algorithms that made it possible, especially since these routines would go on to influence gaming history via Bill Budge's Pinball Construction Set. To begin with, both Qix and MacPaint were built as profoundly raster images, and both use similar algorithms for "painting" in their geometric forms. But more importantly, much of Atkinson's work, like that of Qix, was not only an attempt to find an algorithmic method for interactively constructing visual output, but to do so within the constraints of a Motorola 68000 microprocessor using 128kb of memory. MacPaint, Macintosh, 1984 & Pinball Construction Set, Bill Budge, 1983 And again, we can see even in these early days of MacPaint, that in order to construct the computer image in a visually compelling way, Apple's marketing machine opted to look back to previous techniques of image construction, here the japanese wood cut, and not to that of the photograph. Pixel Clouds One of the most beautiful games to emerge in the last few years is Proteus, a love-letter to this naive period of highly pixellated gaming. Only here, the game is rendered with a modern vector-based graphics pipeline. This creates a strange oscillation between the utterly fluid 3D navigation, and the giant blocky pixellated landscape. Trees, shrubbery, waves, raindrops, animals: everything has been reduced down to a limited grouping of pixel blocks. In Proteus, we walk around the simulation of an island world and explore its aesthetic qualities: sound, color and shape all interact in an elegant generative landscape. There is no real "goal" to the game, although season-shifts can be provoked in a pleasant transition that eventually leads the player to new forms of gaming experience. The whole experience suggests that perhaps some new media form — of an entirely new quality — could be afoot in what we call gaming, although I cringe to qualify such a future as "just over the horizon" because gaming has been promising such an unattainable land for the past several decades. Still, the hope here is that this emerging form is less about Holodecks and more about the raw interactive audiovisual experience of this new media form. The ultimate goal of Proteus, I suppose, is that of aesthetikos, i.e. sensation, or perhaps more accurately the experience itself of human sensing. In other words, we are talking about aesthetics in the Kantian sense of a search for beauty — via the senses — that eventually discovers itself in the limits of its search (cf. Sublime). For, the overall effects turns out to be indeed highly romantic, something akin to a multidimensional interactive 8-bit rendition of a Turner-esque tone poem. While playing Proteus recently, I found myself in a curious situation. I was high up atop one of the hilly peaks of the island, watching as night began to fall and rainclouds emerged below. As I descended down from the hill and onto the rain-soaked plains, I suddenly found myself awash in a pure sea of color that originally felt like a visual glitch: while I could still move somewhat, it seemed that any direction just led me to more colored polygons rendered as flat shapes. For a few moments, I even imagined that the game engine had crashed and I started to reach for the ESC button to get myself back in control of the machine. But then, slowly, I began to realize that I had merely descended down into the level of the clouds themselves and was swimming in the middle of their visually depth-less space. Anyone who has flown in a plane knows this de-spatialized zone while traversing the clouds: there is no focal point or point of reference and everything feels atemporal and ethereal. Essentially this is what happened to me looking through the little portal of my computer screen, the same logic taking place on a purely representational level of pixels that refused to figure the depth contours of the objects in space. Finally, I just leaned back and watched as abstract geometric shapes of treetops re-emerged only to be submerged again in swaths of color as waves of clouds chased ever more waves of clouds. It was a profoundly pleasureful oscillation between recognition and disorientation, one of the key ingredients to many successful works of at. Eventually the cloud formation began to recede from my point of view, and the three dimensional perspective of the landscape re-emerged, re-aligning the simulated first-person perspective of my view portal onto a three-dimensional landscape. The beauty of the moment had something to do with what the art-historian Hubert Damisch calls the théorie du /nuage/ or theory of /cloud/. The term /cloud/ is written with two slashes in order to reconstruct in text the odd, receding nature of clouds from realism and perspective and their re-apparition within the tableau in the form of a semiotic signifier, almost like a placeholder or an asterisk. Clouds in classical painting are the limit of perspectival representation, the resistance of aesthetics to the mere logics of mimesis and perhaps even of representation. Whatever the case, it is the limit of the realism model of aesthetic forms (cf. Cory Archangel's Super Mario Clouds). This limit of perspective within a three-dimensional simulator takes us back to Battlezone and its visual, artifactual, limits. And this limit speaks to one of the fundamental problems confronting video games today, beyond the problem of figuration and by extension the problem of figuring the human face. This representational limit of the /cloud/ in Proteus is what we could call the limit of realism as a model for what simulations, and therein gaming, seek to achieve. Taken to its limit, these clouds of Proteus have their cousin in a wonderful little game built by two lifetime members of the glory days of the Atelier Hypermedia: Pascal Chirol and Grégoire Lauvin. In their collaborative piece NEVERNEVERLAND Color Suite, a 3D simulator and a joystick open up a landscape of nothing but infinite gradients of color: Consider it a 3D simulator of navigation within the color selector of your favorite painting software. And it is also probably one of the outer limits only an artist can propose to the world of gaming in its relationship to the aesthetic realm: a landscape of color, a perspective of visual artifacts, as itself the "goal" of the game. Via play, via simulation, we are now beyond play, beyond simulation, and even figuration; the play has moved into the aesthetic realm, the domain of sensation, opening up an entirely different sphere of experience than that of the reconstruction of a physical world. This is a playable aesthetic world, not beyond ours, but instead immanent to a new field of perception within our world: the realm of artifactual play. This post first appeared on Douglas Edric Stanley’s blog. For more interesting observations, […]
- Mediated Cityscapes 01: Four Statements About Urban Computing [screen captures from IBM's "Planning for Smarter Cities" commercial / 2010] Conversations about ubiquitous computing and the city often get anchored to specific paradigms: urban informatics, discussions of 'smartphone urbanism', open data drumbeating and any number of other stock frames of reference are usually engaged before more primary topics like civic engagement, class and our moment-to-moment experience of the city are broached. This is not entirely surprising as cities are monstrously complex assemblages – it is difficult to wrap our heads around the scale of the infrastructures, ideological forces and the flows of capital that shape the urban realm. If one were to unflinchingly subscribe to the claims made by advertisements like that pictured above, they'd be inclined to believe that we are on the threshold of a fundamental shift in the way we represent and 'operate' our cities. However, on closer consideration it is clear that we are merely at the end of a very long arc of developments that has seen the increasing deployment of scientific management principles and information technology directly into the urban fabric. What we're really experiencing right now is an exponentially greater data yield from and increasing interoperability between systems that we previously considered insular. While the density of sensors and access to civic data may be increasing, this rationalization of the landscape has been underway since at least the early nineteenth century – Molly Wright Steenson has astutely identified the origin of these phenomena as the intercity railroad and electrical telegraph, technologies that "annihilated both space and time" and "transmitted intelligence".(1) This text is the first of a series entitled Mediated Cityscapes, which will provide a cursory introduction to how emerging technologies interface with the city. The goal of this endeavour is to deliver an overview of current thought in this field, a selection of related case studies and to identify and consider several key historical precedents. There is a breadth of opinion and a lot of moving parts within this discourse, so rather than produce catch-all manifestos this series will be delivered as speculative, topical vignettes. This first post provides four general statements regarding urban computing and information culture more broadly. [Berlin Wall 3D screencaptures / images: layar] Statement one: smartphones are only a means to an end Some of the wildest writing on 'smartphone urbanism' can be found in Benjamin Bratton's 2008 essay "iPhone City", which offers a thorough and mildly psychotropic consideration of the seemingly boundless domain of iDevices. Bratton reads the sensor-laden handsets as altering the use of space so that it is "less about geography and more about opportunity" and acknowledges the pervasive 'appification' of various urban functions: "Phone+city is a composite read-write medium, allowing for realtime communication through multiple modes, now and in situ, and represents, in combination, an important infrastructure of any emergent global democratic society. It can do this not only because it enables physical, communicative and thereby social mobility, but because it dramatically reinserts specific location into digital space and does so by making location gestural." This dramatic reading of the remediated city speaks to a seamless intersection of representational and lived space. While alluring at a conceptual level, we've seen little evidence of a mobile interface or platform that perfectly dovetails with 'waking life' – a case study in the limitations of an existing augmented reality (AR) application will drive this point home. The above screen captures are of Berlin Wall 3D, an AR application released last year by two German developers (Hoppala and Superimpose) for the AR platform Layar. Like other AR apps, the software capitalizes on the built-in camera, accelerometer, magnetometer and gyroscope sensors of contemporary smartphones to provide real time information overlays to allow a mobile handset to function as an 'urban viewfinder'. Launching Berlin Wall 3D in Berlin allows users access to an overlay of a 3D model of the infamous concrete barrier that divided the city—and two worldviews—between 1961 and 1989. Users of the app become atemporal tourists that are granted an inkling of the scale and quality of this massive social partition and can freely move about the present-day city with their attention firmly anchored in the past. The project reveals both the possibilities and limitations of AR served through handheld devices. While these representations are convincing, they are also experienced alone and a user must at least partially withdraw into a state of 'eyes glued to the screen' introversion to access the digital shadow of this ominous historical landmark. Current high-end handsets are clumsy, over-branded accessories that we bring into our lives long enough to tether ourselves to exorbitant data plans before shipping them to the landfill. Perhaps the necessary counterpoint to Bratton's enthusiasm regarding "gestural location" can be found in the hunched, incandescent figures that populate Chris Ware's 2009 New Yorker cover – a scene where fantasy is inseparable from isolation. To look wildly towards the future: note the protagonists in Keiichi Matsuda's Augmented City 3D, where 'interfacing' is unconstrained by gadgetry. When can we skip forward to this kind of effortless computing? My advice is to keep tabs on DIY gestural interfaces, smart surfaces and the wearable technology scene for cues as to how we'll unlock ourselves from our present reliance on mass-market vanity electronics. Be sceptical of anyone who tells you the smartphone is an 'elegant urban interface'—they have either never read Calvino or are in location-based marketing—the devices are merely placeholders for cheaper technologies that will more gracefully engage the body. [Data Collection - ID #01 / 2009] Statement two: the quantified self demands due diligence In her brilliant 2004 essay "Intimations of Everyday Life: Ubiquitous Computing and the city" [PDF], Anne Galloway considers a nuanced vocabulary for thinking about emerging technology and the modern metropolis. Galloway argues that 'the everyday' revolves around "spatialization, temporalization, embodiment and performativity" and that these are the frames of reference through which we should scrutinize ubiquitous computing. Identifying and tracking events across time and space, the graceful execution of computation 'in the world' and engendering action – if our tools can facilitate these goals we'll surely be better off for it, right? Well, while essentially correct this thesis wavers somewhat when you start to consider some of the implications of our increasingly networked identities. The above image is from the Canadian artist Dave Kemp's Data Collection, a photography project that created identification card 'portraits' of approximately one hundred subjects. Each participant had the final say as to which cards were included in their photograph so—as evidenced by the example above—perhaps student and membership cards were suitable exhibit fodder while bank, credit cards and social insurance information remained concealed. The point of this endeavour was to force individuals to make conscious decisions about what information they share and what remains concealed – generally speaking, this mindfulness associated with this exercise is largely lacking on the social web. If Facebook is representative of the digital commons that the masses want (and perhaps deserve) it is very likely that we will see the same kind of market-driven dataveillance associated with this 600 million-strong social network play out in the networked city. In order to meaningfully translate the minutiae of city life—let alone civic engagement—into machine-readable data, we have to be able to critically engage the significance of sharing personal information. As below, so above: if we cannot develop agency in defining our personal transparency, can we meaningfully develop open governance and institutions? Although describing the fragile post-Wikileaks state of global superpowers, comments made by activist Rop Gonggrijp in his keynote speech at the 27th Chaos Computer Club Congress this past December are quite relevant a the municipal level: "As we enter uncharted terrain, we are the first generation in a long time to see our leaders in a state of more or less complete helplessness. Most of today’s politicians realize that nobody in their ministry or any of their expensive consultants can tell them what is going on anymore. They have a steering wheel in their hands without a clue what—if anything—it is connected to." This is the ground-zero moment for DIY citizenship and there is definitely a wealth of opportunity available for individuals that are able to capitalize on this leadership vacuum. The tech-savvy and fiercely imaginative are charged with making sense of big (civic) data, assessing and reimagining crumbling infrastructure, building prototypes, finding business models and inviting themselves into the free-for-all of policy-making. To quote Adam Greenfield's "Elements of Networked Urbanism", we need to shift from being "consumers to constituents" – who would have thought an ethics of interaction design could be the rallying cry for a generation? [Trash Track electronics and diagram / photo: SENSEable City Laboratory] Statement three: there is no truth but in things The SENSEable City Lab's Trash Track is an inspiring example of how 'everyday' computing can cultivate our understanding of fundamental urban processes. Prompted by the simple question "why do we know so much about the supply chain and so little about the removal chain?", the project employs electronics-laden refuse to gain insight into waste management. Produced for the 2009 exhibit Toward the Sentient City(2), the project visualizes the path of waste by by attaching custom designed radio transmitting tags to discarded objects and then tracks their journey to various storage and processing facilities while en route to the landfill or recycling depot. Decrepit consumer electronics, bagged garbage and disposable coffee cups are transformed into geolocated nodes that generate analytics to assess these previously opaque 'migrations' within the life cycle of waste. This data provides a 'bottom up' reading of these processes and could reveal inefficiencies and redundancies in the management of garbage. As a proof of concept prototype, the undertaking is also extremely valuable in generating a more nuanced awareness of our waste footprint and allowing us to trace the journey of an object that we handled or had a personal connection with. Aside from prompting dialogue regarding the implications of consumer culture, Trash Track is important because it clearly illustrates how embedding sensors on everyday objects can generate data that can used to fine-tune municipal services and protocols. We take it for granted that citizens can act as sensors by alerting municipal authorities about deficiencies (potholes, broken streetlights, etc.) so—as it becomes more feasible—it only follows that we should equip the fixtures that populate the city with the means to 'report' as well. Tremendous effort has been expended to transform the smartphone into a robust mobile sensor platform – we will benefit greatly once we start directing some of this energy into outfitting public space with similar capabilities.(3) [Concept diagrams for Chromaroma visualizations / photo: Chromaroma blog] Statement four: territories > maps The above images are a series of concept diagrams for the visualizations at the heart of Chromaroma, a social game that allows users of the London Underground and the Cycle Hire bike sharing service to track their movements through the city while engaging in friendly competition. Players of the public beta of the service register their underground Oyster Card (RFID ID) and bike sharing accounts and log trips and achievements through a related social network. Every subway trip a user makes scores them points, teams compete to capture stations and the network randomly assigns 'missions' that reward bonuses and multipliers for travelling to various destinations throughout the city. The incentives and achievements offered to players borrow heavily from the location-based game Foursquare, but while the latter service (essentially) reduces the city to a banal matrix of commercial establishments, Chromaroma piggybacks on the user experience of a public asset. Harry Beck's 1931 Tube Map is one of the most iconic images in 20th century graphic design and the clarity of that schematic essentialized the representation of a key piece of urban infrastructure and promoted the diagrammatic style of thinking that underpins contemporary information visualization. While the so-called gamification phenomenon(4) is generally quite suspect, Chromaroma achieves a remarkable feat in seamlessly superimposing game mechanics on everyday civic actions of hopping on the subway or utilizing the bike share program – players are quite literally invited into the representational space of the ubiquitous 'subway diagram' and are able to replace a top-down system map with an interactive visualization that charts their engagement with public transit. In a recent interview with The Guardian, Chromaroma creator Toby Barnes described the goal for working with game systems as promoting exploration, facilitating collaborative construction and instilling participants with a "sense of achievement". The social game is slated to expand to incorporate bus, tram and boat transit and when the service moves out of beta it will be very interesting to see if they can build a (presumably advertising-based) business model around promoting the use of public transit. Two decades ago, in an article entitled "The Computer for the 21st Century", ubiquitous computing founder Mark Weiser argued against virtual reality (VR) by highlighting the fact that VR environments "were only maps, not territories" – given that were in the midst of an era that celebrates a constant stream of nonsensical information graphics, we should heap praise on any visualization project that simultaneously promotes exploration of the world and positive civic action. The next post in this series will deal with Memory and the City. Notes: (1) See Steenson's essay “Urban Software: The Long View” published in the catalogue [PDF] essay for last year's HABITAR exhibition at LABoral. (2) Toward the Sentient City is essential – I highly recommend spending a few hours on the exhibit site perusing the work that was produced and the "responses" that were commissioned. (3) Beyond Trash Track, Combing through the SENSEable City Lab's archives reveals a body of work rife with provocative experiments that call into question how we represent and experience the city – the work done by this group will prove foundational for the myriad of applications that will be produced with open data. (4) There is no shortage of perspectives being offered regarding gamification at the moment, if you are unfamiliar with the term Jesse Schell's DICE 2010 presentation "Design Outside the Box" is as good a place to start as any. – About the Author: Greg J. Smith a Toronto-based designer and researcher with interests in media theory and digital culture. Extending from a background in architecture, his research considers how contemporary information paradigms affect representational and spatial systems. Greg is a designer at Mission Specialist, blogs at Serial Consign, writes a column on emerging technology for Current Intelligence and is a managing editor of the digital arts publication Vague Terrain. He currently teaches in the CCIT program (University of Toronto/Sheridan College) and at OCAD […]
- Daito Manabe: being real about being material [Theory] An artist with a background in DJ-ing, electronic music, and visuals for clubs in Japan and now all over the world, Daito Manabe tends to make virtual things real, rather than vice versa. One of the things that I find most compelling in the work of Daito is the materiality, the viscerality of the means of transmitting his bits. As Mitchell Whitelaw points out we can aestheticize transmateriality, that is, the coolness of the not-there, of the purely momentary electric buzz, but there’s no getting around the fundamental essence of a physical entry point. For me to be aware, I must engage my body and the material of the world that lies right beyond it. What’s so refreshing about Daitos work, particularly in contrast to many North American computational artists, is his willingness to engage the body as a canvas, as a site of action, rather than the engine of action. He might be one of the most interesting post-screen artists working today quite simple because of the canvas on which he’s chosen to work and the playfulness with which he approaches the body as a canvas. You can’t help but notice how for a time computational art seemed to focus on the digitalization and augmentation of space and physical properties. With a rash of compelling works that focus on the rematerialization of digital space, the reverse seems to be gaining momentum. Just as we were all once surprised by the digital mimicking the analog, familiar digital tropes now surprise when revealed in analogue form: real, actuated, physical stuff, moving around. A projection is not just a projection onto a blank screen, instead, he paints the wall with phosphorescent paint and illuminates it with a laser to paint a transient image. To write a sentence or draw a hand, he arms a robotic arm with an automatic BB gun and shoots it through a sheet of paper. It's an interesting thing about the aesthetics of these surfaces: they're really about not using the traditional screen even though I'm only aware of them because they are on a traditional screen. Without Youtube, I wouldn't even know the name Daito Manabe, and in all likelihood neither would you. But his videos that he puts up are more than just documentation, more than a simple process video to demonstrate technical considerations, background, the specifics of a set-up, nor are they the slick depth-of-field heavy mini-advertisements of so many design agencies. There's very little hidden magic or un-necessary polish in both his performances and his videos. An electrode to the face is rather difficult to miss, its effects are unmistakable, and that's part of the point: if you're at all empathetic, you can feel it too, even from the audience or from half a globe away in front of your computer. There’s another un-mistakable reference floating around in Daitos work: the club. One could imagine a dance floor as a fluid particle simulation: forces inspire bodies, particles influence one another, density and energy increase and decrease in patterns. It’s the perfect territory for a computational artist. A crowd of women with miniaturized versions of club light displays walking around Tokyo looks like a club. The synchronized bodies of Particles At YCAM resemble nothing so much as either choreographed dancers or generated flocking fireflies. The music is absent but its pattern can almost be heard, synaesthetically. What's interesting is that these objects could very well be virtual, but they simply aren't. Their movement could be played out in a purely virtual plane, but it isn't, and that's what makes them so more interesting. What's a wave without a medium? What's a beat in a club without people to listen? His history of working as a DJ and VJ makes perfect sense: in few places is computational technology as visceral, as embodied, as in making music. "from hardware hacking to modular LEDs and custom software, they participate in what might be called "expanded computing", using the malleability of digital media to reactivate its presence - and thus our presence, too - in the world of things." - Mitchell Whitelaw In his excellent essay "After the Screen" Mitchell Whitelaw talks to as post-screen imaging: making images on things that remind us that they're in a single place and that, despite our illusions to the contrary, in a material sense, we are too. In Daitos bodily works and his physical works the pixels are not simply pixels, they are automata that speak to one another. His physical pixels swarm and synchronize, his human computer interfaces play their people; as in Stelarc, though perhaps in a less frightening way, the machine drives the human. Much of the early cybernetic performance art emphasized the strangeness of a human and machine co-creating a being. It was scary and weird. The lightness of Daitos work comes from his playfulness. I always get the sense that it isn’t so much that cyborgs are weird, just that his cyborgs are a little weird. When he uses the face as a solenoid of sorts, an actuated surface, it’s very similar to things that architects and material engineers design for and dream of. Just made ever-so-slightly weird. Another word might be "quirky". But that minimizes what I think is the importance and the curiosity of the things that Daito makes. More than anything else, his work reminds us that the promise of hyper-surfaces and hyper-actuated technological skins, is at the moment only delivered by us and our bodies with all the strange, funny, and sometimes uncomfortable corollaries that […]
- Drone as Metaphor – Interview with artist/director Alex Rivera There has been no shortage of interest in unmanned aerial vehicles over the past few years. While the legalities of unchecked extrajudicial assassinations is probably not the primary concern of CAN readers, the creative (and critical) potential of semi-autonomous robots is. The New Inquiry Senior Editor Malcolm Harris recently conducted a fascinating interview with artist/director Alex Rivera (of Sleep Dealer fame) in which Rivera discussed his ongoing research into drones and disembodied labour. The exchange between the two is light years ahead of most of the discourse on drones that is occurring within creative technologist circles at the moment and really worth checking out.According to Rivera, the allure of drones is that they perfectly symbolize "the transnational/telepresent world we inhabit." The artist makes some really astute comments on labour, international relations and he just nails the uncanny implications of drone warfare: Returning to the theme of the military drone, a lot of the first round of critique was that they make killing antiseptic or like a video game, or that it’s hyper-alienating for the pilots. But what I tried to depict in my film and what I believe is happening is something not that simple. The drone has produced a third type of military sight. Drone vision is not like the infantry’s vision that sees the opposing forces with their eyes, and it’s not the sight system of the airforce pilots that never really saw what was below while dropping bombs from thousands of feet up, often at night. The drone pilot has a type of vision that no military actor has had before, that of lingering, of observing over extended periods of time, and doing so with absolutely no threat to oneself. I'd go as far as saying that this interview is essential reading for anybody working with (and thinking about) semi-autonomous robotic systems. It should also be noted that The New Inquiry's most recent magazine issue is a full-on drone extravaganza and presumably worth digging into for additional incisive commentary. Interview Alex Rivera | Related projects: Sleep Dealer & Low […]
- Archigram Archival Project [Reference] It's a pleasure to announce to CAN readers that this amazing project I have been working on for the past year is now live and ready for your perusal. The Archigram Archival Project [AAP] makes the work of the seminal architectural group Archigram is now available free online for public viewing. The project was run by EXP, an architectural research group at the University of Westminster, where I teach, and was funded by the Arts and Humanities Research Council. This was also all made possible by the sheer generosity of members of Archigram and their heirs who allowed us to browse through the immense collection of work stored in the attics and basements and collected a total of around 10,000 images. On Monday night was the official site launch and if you were following us on Twitter you would have seen a number of updates regarding the project. I am happy to say even with some major hick-ups just before the announcement (server power failure at the university), with about 150 attendees including collaborators, historians, journalists and fans, the launch was an absolute success. Mike and Dennis were not able to join us but they were there thanks to Skype with Dennis taking us through the different parts of the site. This was followed by Peter's talk on Archigram proteges with David also at the event, always in the mood to kick off an inspiring conversation. We are incredibly pleased that now, finally after all these years, we can all enjoy the work of one of the 'most seminal, iconoclastic and influential architectural groups of the modern age'. The extraordinary influence of the mainly unbuilt 1961-1974 architectural group Archigram is internationally acknowledged. Exhibitions of their work have been touring major institutions worldwide since 1992, they were awarded the RIBA Gold Medal in 2002, and they are recognised influences on many of the world's greatest contemporary architects and buildings. Yet the bulk of their visionary work has to date remained difficult to access, largely stored in domestic conditions or temporary storage. In collaboration with the remaining members of Archigram or their heirs, and funded by a £304,000 grant from the Arts and Humanities Research Council, a team from the University of Westminster has formed an online, searchable database of all the available works of Archigram for study by architectural specialists and the general public. I have collected few projects below, just to highlight how forward thinking Archigram were, foreseeing many things we enjoy and desire today. For full list of projects, make sure you visit archigram.westminster.ac.uk. Geoff Manaugh of BLDGBLOG has also written a wonderful post on his blog about the project which is a must read. For now, I leave you with 7 fantastic but much less known projects somewhat related to CAN. Holographic Scene Setter Speculative proposal for holographic projection of environments, or virtual reality environment. Part of the Instant City project. I had a holographic scene setter - a light space - switch on/walk around/3D/walk thro'/Hollywood Boulevard in my TV room/Death Valley on my patio/Tahiti in my pad/Laurel and Hardy in the morning/The 'Who' at night ... change film - new environment/switch on/off/there - not there ... what's real/it's observable/it's real when it's there/is it a dream? - a ghost? - a turn-on? ... Holographic ceiling - cloud - rainbow - cloud - people - John (pee on your shoes) - scenery - event - television ... great ... switch on the people/turn on the crowd/bring in the whole scene ... turn off the ceiling. more Media Experiments 1-2, 1968 Light/Sound Workshops: television display system set up as an experiment in multi-channel and multi-media display with streams of images flickering across grouped screens. A far more flexible medium is T.V. which, at the moment, is still normally thought of the single channel box, but which whilst utiliising other media such as film as content allows us far more opportunity for selection. If we then consider T.V. used in display systems monitoring a number of channels concurrently from a variety of sources, both from national and international news and entertainment networks and also from personal close-circuit and video-tape and even generated by computer, we can see what colossal potential there is in the medium. So in the not so distant future we can expect to have to deal with the multi-channel multi-media situation both professionally and as an involved audience in our own homes, and one suspects at times the distinction between producer, and audience may become blurred. more Soft Scene Monitor: MK1, 1968 Exhibit designed for Aftenpostle newspaper and Oslo Architectfornung and exhibited at Kunstneres Hus, Oslo for a prototype home access unit to communications, audio-visual entertainments and information technology. As the Instant City study developed, certain items emerged in particular. First, the idea of a 'soft-scene monitor' - a combination of teaching-machine, audio-visual juke box, environmental simulator, and from a theoretical point of view, a realization of the 'Hardware/Software' debate. more Info Gonks, 1968 Speculative design maquette for educational television glasses and headgear. Use of the 1½-inch television as a built-up pair of spectacles with stereo glasses all wired to headgear receiver: everyman his own on-the-eye and in-the-ear environment. more Cushicle & Suitaloon, 1966 Speculative design for a personal, individual and portable dwelling unit which may be ‘worn’ for transport and unpacked for occupation. The illustrations show the two main parts of the Cushicle unit as they expand out from their unpacked state to the domestic condition. One constituent part is the “armature“ or “spinal“ system. This forms the chassis and support for the appliances and other apparatus. The other major element is the enclosure part which is basically an inflated envelope with extra skins as viewing screens. Both systems open out consecutively or can be used independently. The Cushicle carries food, water supply, radio, miniature projection television and heating apparatus. The radio, TV, etc., are contained in the helmet and the food and water supply are carried in pod attachments. With the establishment of service nodes and additional optional apparatus, the autonomous Cushicle unit could develop to become part of a more widespread urban system of personalized enclosures. more Enviro-Pill, 1969 Speculative proposal for a pill for inducing architecture or virtual and imaginary environments in the mind. more. Electronic Tomato, 1969 Speculative proposal for mobile sensory stimulation device. MANZAK is our latest proposal for a radio-controlled, battery-powered electric automaton. It has on-board logic, optical range-finder, TV camera, and magic eye bump detectors. All the sensory equipment you need for environmental information retrieval, and for performing tasks. Optional extras include response equipment for specific applications and subtasks to your own specification. Direct your business operations, do the shopping, hunt or fish, or just enjoy electronic instamatic voyeurism, from the comfort of your own home. For the great outdoors, get instant vegetable therapy from the new ELECTRONIC TOMATO – a groove gizmo that connects to every nerve end to give you the wildest […]
- Screensaver, Revisited [Reference] While the euphoric, dynamic forms of computational artist Marius Watz are probably quite familiar to CAN readers, the artist's curatorial and educational undertakings should definitely not to be overlooked. As an extension of his practice, Marius consistently organizes prescient and formative exhibitions and workshops (see the upcoming Generator.x 3.0: From Code to Atoms) and often teaches within various design and architecture schools across Europe and North America. Marius just posted the above slideshow of an upcoming teaching exercise that he'll be overseeing at the The Oslo School of Architecture and Design (AHO) and it is worth clicking through. Entitled "Screensaver Culture" the assignment is a critical reflection on the current potential of the screensaver given that the days of CRT monitor burn-in have long passed. In setting up the exercise, Marius provides a chronological lists of essential precedents including scrnsave (1983), After Dark (1989), SETI@home Classic (1999-2005) and reconsiders that lineage in light of contemporary computing (and data practice) trends to extrapolate a range of approaches that his students might employ. Check out Marius' slide deck and assignment brief – it offers a surprisingly compelling retelling of an easily overlooked topic within the broader history of computer […]
- Miebach and Posavec – Data, Visualization, Poetry and Sculpture Visualizations are created to make data more legible. They are intended to give us a neutral portrait, so to speak, of how collections of data relate to each other. In so doing, they make information accessible to us that would otherwise be obscured by its scale in a manner that is easily comprehended. Data is presented to us exactly for what it is so that it may foster the communication of information through the recognition of connections or relationships. This method of attempting to show data in an unadulterated, albeit creative way sees data more as its subject matter than its raw material. As laudable as this effort is, data as representation does not have to be the only way visualization is approached. Nor should a traditional visualization ever necessarily be perceived as the full picture. It should always be understood that there is an elusive, human, element, whether knowledge or otherwise, embedded in what is being communicated. For artists such as Nathalie Miebach and Stefanie Posavec, the notion of visualization becomes more broadly defined and expressive. Both women spoke at this year's Eyeo Festival and during their talks expounded upon and expanded the definition of data visualization proper. Firstly, for each the ability to work with their hands or to be able to tactilely interact with something visualized was very important. For Miebach the inability to touch a visualization or to fully explore it with the contours of her hand made it difficult to truly comprehend. Posavec acknowledged there were different emotions associated with hand-making something and making something with a machine. While it might seem odd to focus on the hands when talking about visualizations it is important to understand the approach to it as modeling an object or design out of a raw material rather than to merely attempt to show it. The hand or body as a human experience that is something that can be lost in the flatness of a digital image, interactive or otherwise. Take Miebach's sculptures for example. She uses as her subject matter weather data from various environments and histories. She then either translates it into music or into colorfully elaborate weaved sculptures. For the sculptor cum visualist there is a subjective appeal to how she generates her creations. Whereas typical visualizations are “'didactic” in how they present data she calls her sculptures ”poetic”. She takes joy in the ability to walk around and explore the sculptures rather than sacrifice that dimensionality to the computer screen. For her it's as important to foster an experience with the data as it is to discover new connections. Instead, in the same way folk stories preserve history, she creates narratives that contain traces of information. During her talk at Eyeo she asked whether fact and fiction could coexist and whether information becomes fictional by blending them together. The expression data in service of telling story becomes tantamount to their presentation. Like an abstract painting that does not come right out and say what its about but instead provides parameters for interpretation, her sculptures turn information into a panoply of meaning. By that same token, Stefanie Posavec takes a similar, yet opposite approach. She uses fiction to generate data instead of the other way around. Using novels such as Kerouac's On the Road she employs what she calls “data illustration” to trace patterns in the writing. By personally exploring the texts, she 'visualizes' styles and themes that reveal themselves to her within the immanent space of the book. The content of the book intermingles with her own personal traversal of the text to generate a new way of generating meaning from the 'data' that is already there. A new way of reading then begets a series of colorful illustrations that document her experience. At Eyeo she characterized data as a lens for which to see a subject from an entirely new angle. The angle becomes primary over the data as a tool to see or as she calls it, a "souvenir of human engagement." Posavec is then able to navigate a text, such as The Origin of Species, using data to discover design solutions, as she says, where informational insights aren't the main purpose of the visualization. Taking the edits and updates between different editions of Darwin's famous text she generates imagery that is aesthetically related to the subject matter in the form of botanically and organically inspired abstract images. In both cases, data is not the primary focus of what is being visualized but springboard into something not as scientifically well-wrought but on the contrary is much more human and intangible. They are about not just seeing in a new way, but also creating new objects out of what already exists perhaps in contrast to the character of the so-called New Aesthetic. They are not satisfied with simply foisting a singular means of seeing the world upon us but offering something more shifting and elusive. They are bodily insofar that Miebach's sculptures can be touched and walked around and Posavec's designs are generated out of the physical effort of drawing them out over time. We don't just look and see an image but something that we cannot immediately appertain and qualify. For both artists there is a kind of meaning the data can generate but that isn't necessarily in the data itself. Data visualization is already in some cases an abstract enterprise in how the data is presented. However, in the same way that representational art sought to imitate the appearance of something that exists in real life, so too do representational visualizations. A standard visualization practice typically involves taking a large amount of data that is incomprehensible to an individual on a macro level and presenting it in such away that it is both visually appealing and legible. Often the former effort is an extension of the latter wherein an appeal to aesthetic sensibilities generates an interest in the data that is being showcased. In other cases it is a matter of finding the visual design that most clearly presents the data. In contrast, the data expressionism of Miebach and Posavec doesn't attempt to neutrally visualize the data they are using. And whereas data representations refer to themselves insofar that they are visualizing their own raw material as subject matter, data expressionism uses data more as a starting point to suggest something that is indefinable and ambiguous, yet still truthful. Representationalist visualization is all about pattern recognition and stopping at those patterns as enough to generate understanding. However, there will always be a danger that those patterns subsume what they are intended to represent on a superficial or limited level. Miebach and Posavec remind us that as important as data is for certain ends we cannot forget what could potentially exist beyond the mere image in the form of human experience. Stefanie Posavec: itsbeenreal.co.uk | Nathalie Miebach: nathaliemiebach.com -- Author: Dylan Schenker is a writer based in New York City interested in new media art, culture and theory. You can find him on fragmince.tumblr.com and […]
- “Exhausting Gameplay” by Douglas Edric Stanley / Theory, Games A significant percentage of video games employ in one way or another the figure of death. The thanatological sub-species of video game representations are practically endless: dismemberment, infection, untreatable wounds, explosion, etc. Players can be eaten, crushed, sliced, diced, quartered, electrocuted, impaled, and so on. Many of these representations are more or less approximate: in Doom, for example, a player’s state of “health” is represented by an abstract percentage value where players do not die of any specific organ failure, but instead from some sort of provoked exhaustion. In role playing games, players kill their opponents in a similar manner, i.e. by reducing this all-encompassing numerical value of their enemies to zero. In other games, players simply keel over, or disappear in a puff of smoke when touched, as in Pacman. In Super Mario Bros. players can just run out of time. Death in gaming is more a question of symbol than of substance. While we are still in the realm of simulation, the simulation is so figurative as pull us into an wholly other realm of representation. In his 1972 article on transcendence, gaming and “computer bums”, Stewart Brand used the term “symbolic” to describe the flickering figurations of death slowly taking over university computer science research consoles: “Fanatic Life and Symbolic Death Among the Computer Bums“. //// The need for death in gaming is multifaceted. The cynical argument, largely based on the arcade experience, might describe death in gaming as an economical equation: in order to make more money, games needed to provoke death as quickly as possible in order to get to the next “insert coin”. But death in gaming has been with us practically since the beginning, or at least since 1962′s Space War, long before the video game arcade phenomenon became an economic reality. A less cynical, more narratological reading of eschatology and gaming might look at death as a question of motivation: by introducing death into the game, a certain internal dynamic is created, which in turn heightens the gameplay and structures the temporal form of the game itself. Given that the player will eventually die (from inaction, from inattention, from error, etc), the goal of the game becomes that of survival. The “game over” screen provides some sort of closure to the game and proposes — albeit post facto — a redefinition of the initiating act of the game: “I want to play” has now been translated within the gameworld into “I want to live”. //// A third approach would be to look at the material substrate of gaming itself: video games are played on machines and machines, eventually, break down. To quote Felix Guattari: “Machines are instilled with a desire for abolition. The emergence of the machine is accompanied by failures, catastrophes, and death which haunts it. La machine est travaillé par un désir d’abolition. Son émergence est doublée par la panne, la catastrophe, la mort qui la menace.” Chaosmose, Éditions Galilée, 1992, p.58. From this perspective, we could see the figure of death in gaming as an extension of this fatal impulse of the machine. The figure of death would be an attempt at sublimating the machine’s death drive into a poetic form — a form upon which a game world might be built. //// *Spoiler alert: you probably shouldn’t read any further if you haven’t played the games “Passage” (Free/Cheap, Mac/PC/Linux/iOS) or “Journey” (~15€, PS3). Especially Journey. Read at your own risk. While many video games represent death within their gameworld, or use death as a mechanism for the gameplay, there are three games in particular that employ death as the central raison d’être of the game: Jason Rohrer’s 2007 conceptual/indie gem “Passage“ Tale of Tales 2008 art-game meditation “The Graveyard“ That Game Company‘s latest (and greatest) creation “Journey” (2012). Long story short, all three of these games represent a character advancing towards his or her death. In Passage, a young man in a highly pixellated two-dimensional gameworld begins at the left side of an open maze and advances towards old age and death on the right side of the maze. During his passage from left to right the spritely blond-haired man evolves into a balding gray-haired old man who limps his way to his final steps. Early in the game he can choose love (or not) with a young woman who will age with him throughout the rest of the game. Their coupling makes the game more poignant, especially the ending, but renders certain movements more difficult within the various passages of the maze. In The Graveyard, we embody an old woman visiting a graveyard. The game is played in three-dimensions, using the standard aesthetic of real-time engine based rendering, albeit with the nice touch of a black and white palette. Like Passage, the game is short and spatially limited: the old woman can move forward or backward on a short path leading up to a church and a bench where she can sit down. As she rests, a song about nostalgia takes over the game, and (in the paid version) ends with the death of the woman, slumped over on the bench. Within the world of “indie gaming”, Journey is a Sony-funded super-production/mega-blockbuster behemoth, especially when compared to the one/two-person auteurs of the two previous games. Even if That Game Company, the creators of Journey, remain a relatively small studio, the production values here are on an entirely different scale. As can be expected, there is more or less a traditional game here, full of beautifully rendered levels to explore and tokens to collect, and even an extremely subtle use of networked multiplayer gaming. But the title is a thinly veiled manifesto — precisely in the vein of The Graveyard and Passage –, on the possibilities of gaming as a medium for sensitive experience (aisthetikos) beyond the goal-oriented mechanics of traditional gameplay. Here too, the player advances on a path leading unambiguously to their death: inevitably, inexorably, and joyously. And once this goal of the game has been more or less removed as a form of strategy, or at least relegated to a mere point of reference, the game switches into a more symbolic realm. //// There this great moment at the beginning of the filmed dialogue on Arte TV between game designers Chris Crawford and Jason Rohrer, filmed at the Independent Games Festival in 2009. While discussing Passage, Crawford begins by suggesting that the significant bit of the game comes from the introduction of a relationship between spatial navigation and metaphor: “What is, I think, most important about your approach is that you’re taking the idea of spatial navigation — which has always been done too damn literally —, and suddenly turning it into metaphor. And then exploring, well, what kind of metaphors can be explored with spatial systems?” — Chris Crawford in “Au coeur de la nuit : Jason Rohrer et Chris Crawford“; Arte TV; “Durch die Nacht mit…“; episode 61; July 2, 2009; 08:40. Like the figure of death, the history of spatial design in video games is as long as the history of video games itself. In many ways, the form or shape of a video game world, and the way in which that world is mapped onto its display, is so determinate to the game in all its aspects that it becomes more or less conflated with the game itself. When Super Mario Bros. was released for the NES in 1985, it used a form of spatial representation — sideways scrolling –, which had already been explored extensively by a multitude of games before it (cf. Defender, 1980; Scramble, 1981; Pitfall!, 1982; Moon Patrol, 1982; etc.). So while Super Mario Bros. did not invent sideways scrolling, it nevertheless added a significant novelty by equating this left-to-right movement with not only the advancement of the game strategy (survival), but additionally with the advancement of the game narrative. In Super Mario Bros., this left-to-right movement is not only about advancing the player, it is about advancing the story itself. Taking much of its inspiration from Lewis Carroll’s Alice in Wonderland, Super Mario Bros. allows players to flip through various episodes of a story about a character (Mario) tumbling down the rabbit’s hole (here, a pipe) simply by moving from the beginning of the chapter (left) to the end of the chapter (right). What used to be a mechanism for survival (ex: Moon Patrol) or exploration (ex: Pitfall!) is now a journey of discovery. Many games, especially the “on-rails shooter” sub-genre such as Resident Evil (1996-), are in many ways a continuation of this tradition: as the player advances throughout the (highly linear) game-space, episode after episode of the narrative unfolds. Newer games, such as Red Dead Redemption, attempt to break the narrative into pieces and scatter them throughout the game, allowing players to explore other aspects of the gameworld before irreversibly advancing the more linear form of the narrative. In some senses, this form of exploration is nothing more than a superposition of parallel linear threads on top of the main linear thread of narrative. One can obviously imagine that eventually game designers will be able to write these threads in such a way as to interrelate with one another concurrently. Generative storylines have also yet to be fully exploited in game design. But currently it appears that we are circling round-and-round the end of cul-de-sac of contradiction: classical narrativity wishes to be linear, or at least to be explored linearly, while algorithmic machines desire structures that are more emergent, with bifurcating forks of expansive parallelism. The machine loves multiplicity, whereas narrative experience desires linearity. It is almost as if we’ve reconstructed in video games the figure of the brain itself, especially its’ serial vs. parallel contradictions. //// A few weeks ago, at the Atelier Hypermédia, I was exploring the use of boolean values as “flags” with some students: we were looking at how to detect certain types of activity by setting up an interrelated series of boolean true/false variables that could flip from false to true and vice-versa depending both on changes in the environment and the states of internal variables. Our example was a simple object on the screen: a student wanted to know how to program a single-fired action when the object entered into collision with either another object or the player (via mouse or touch, whatever). As we explored various situations, we eventually were confronted with an fairly straightforward behavior that left most of us stumped and took about a half hour of collective experimentation and debate to code. The behavior itself is of little importance here; it had something to do with an object splitting into two when touching another object. What is important is the fact it took us about a half an hour to describe how one simple “state” would affect a subsequent “state”, and that we were debating it with the code sitting in front of us all as a group, as if it were some sort of enigma that required solving collectively. Anyone who has sat bewildered in front of their television for a half an hour of Lara Croft trying to figure out how to advance the game, should just about now be recognizing the scene. When you pull off the various layers of representation of bodacious ponytailed scientists in dark caves with molten lava, waterfalls, rock formations and dynamite, the player is essentially looking at the same complex interrelation of true/false boolean variables that we were looking at when structuring our code. In other words, a Lara Croft game is just a series of interrelated true/false switches that the player has to enact in the right combination in order to unlock a new series of true/false switches. ########################################################################## # RSG-SMB-TAB-1.1 # ########################################################################## How to Win "Super Mario Bros" Nintendo Entertainment System WORLD 1 - LEVEL 1 +----------------------------------+ Key: < = Left | | > = Right | ^ | ^ = Up | < > O O | v = Down | v select start B A | B = B button | | A = A button +----------------------------------+ < ------------------------------------------------------------------------ > OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO ^ ------------------------------------------------------------------------ v ------------------------------------------------------------------------ B --------------------------OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO A ------------------------------------------------------------------------ In Alex Galloway’s “How to Win Super Mario“, a listing of left/right/up/down/a/b button combinations are printed out in text files organized by chapter (“World 1-1″, “World 1-2″, “World 1-3″, …). If these “how to play Super Mario” instructions look just as obscure as artist Ben Fry’s “Dismap” visualization of the Super Mario Bros. code itself, it is precisely because in many ways the two forms (gameplay vs. game code) are simply different forms of representation of the same algorithmic substrate. In order to play a game, we need to understand something about the algorithmic, and even machinic, structure of how the game was constructed. Playing a game requires a certain process of exploring the game code in reverse, reading it on the player’s end of the equation via the render engine, even if the player knows little to nothing about how pointers, variables, and if/then structures work. But from a purely experiential/intuitive perspective from within the game, the successful player of Super Mario knows exactly how these very-same structures work in order to actually play the game. //// The principle criticism of back-in-the-day text adventures (circa 1977-) was the tedium of this logic of interrelated boolean switches the player had to unlock in order to advance in the game: open mailbox, get letter, open letter, read letter, drop letter, close mailbox, go north, look, pick up shovel, dig hole, get gun, shoot self in head. Here is a map from the first commercial textual adventure game hit, “Dungeon” (later renamed to “Zork”), where we can see the complex interrelations of one room to the next and the objects they contained: — Click for hi-res: Dungeon Map, by Stephen Rost, taken from “You May Be Eaten By a Grue” In order to play the game, players had to map out all these interrelated rooms along with the objects they contained and discover how they were all interconnected. While the unwrapping of this maze/puzzle could indeed lead to storytelling (“It is pitch black. You are likely to be eaten by a grue.” – cf. Grue), for the most part the player was simply trying to poke their way around a hopelessly complex maze and find the right combination of non-sequitur commands. From the documentary, “Get Lamp: The Text Adventure Documentary“, interactive fiction writer Dan Shiovitz: “It’s like, you ring a bell and kick a dog a bunch of times, and it starts crying after you ring a bell a couple of times.” While Shiovitz was originally criticizing the absurdity of mazes in interactive fiction, his description is equally apt in describing the absurdity of interactive narrative at its most basic level. Most of the time, the author as well as player are simply trying to unlock a series of boolean switches in the right combination, in order to advance to the next chapter of the story, i.e. the next set of boolean switches in the code. While puzzles are interesting in and of themselves, and can indeed contain interesting opportunities for storytelling, in the case of interactive narratives we seem more to be playing with the machinic structure that made the story possible, than the story itself. //// Most interactive storytelling, whether it be in the form of a 3D first-person shooter or a 2D sprite-based platformer, evolves directly out of the data structures originally designed by Will Crowther as “Colossal Cave Adventure“. This original “Adventure” was based on a map of a real cave, the Mammoth Cave system of Kentucky. Crowther’s idea was to equate each item of data to another data item via a spatial relationship: item “a” is connected to item “b” via the command “north”, which in turn is connected to item “c” when item “d” is present at item “b”. By using the real cave as a map for the data, he was able to create potential walkways that players could use to move from one datapoint to the next as if traversing the successive grottos within the cave system. It was essentially a representation for navigating a datafield, much like the Mac Finder or Windows Explorer offers their own representation via folders and subfolders. By situating data items as points on a map, one could move around the data as if moving around in space. It was upon this foundation, using data points as map points and placing narrative excerpts at each data point, that contemporary interactive narrative was built. From this idea we get text adventures, such as “Zork“, point-and-click adventures such as “Day of the Tentacle“, maze-monster first-person-shooters of the “Doom“/”Quake” variety, and more poetic propositions such as the “Zelda” series of adventures, or “Ico“. While many of these games contain story, character, landscape, dialogue and all sorts of choices and actions that must be enacted to evolve the narrative, they are still fundamentally structured around finding the location of the next switch that will lead to a new series of switches. At the beginning of the game “Ico”, the player must first climb up a series of ladders, find the right windows to climb out of the castle and then back in again, enabling safe passage to a platform close enough to jump onto a cage that will then lower a trapped girl to the first floor where she can safely exit. Once all of these tedious tasks have unravelled, some dialogue ensues, and the story moves on to the next task at hand which is also the next piece of narrative cue. Unfortunately, from a purely literary perspective, such pulling of levers and pushing of switches embodies all the poetic charm and substance of searching for a missing hardware driver buried deep within an external hard disk. //// *Spoiler reminder: stop reading this if you haven’t played any of these games. The beauty of Passage, The Graveyard and now Journey, is that none of this peeking and poking matters any more. Passage truly began the trend with a powerful opening volley: here is a game in which you will lead a character to his death. And while this goal is not explicit at the beginning of the game, it is part of the beauty of realization that takes you over when playing the game. When demonstrating the game last week to the Media Design students, one of them — upon realizing the fatality (note that I did not use the term “futility”) of the game as the player starts balding and slowing down his gait –, exclaimed “c’est horrible!” His reaction was unambiguously emotional. Teeny pixelated graphics with a dorky 8-bit retro soundtrack, and yet a game can still evoke a sentiment of inevitability. It’s just a funky little pixelated representation on a screen, nevertheless “c’est horrible!” Journey follows this formula fairly closely, to such a degree that I wonder in what way the former influenced the later. If so, it certainly would be a nice touch. We know through interviews with Journey’s central designer, Jenova Chen, that he definitely is looking at fellow games and gaming history with an informed eye, but so far I have yet to see a direct causal relation. Whatever the case, Journey is very much a similar affair. At the beginning of the game we are simply a voyager who picks up a scarf in the desert. In the distance, framed in a sweep of the camera straight out of that pivotal desert scene in Indiana Jones: a mountain beckons us, clearly inspired by aforementioned Hollywood classics, but almost certainly as well the Mount Fuji woodblock prints by Hokusai. The landscape is gorgeous, the simulation of the sand beneath your feet subtle and totally pleasing; we are experiencing an aesthetic audiovisual convergence reminiscent of grandiose cinematography on the scale of Nestor Almendros in Days of Heaven or Caleb Deschanel in Black Stallion (we’ll still have to wait for rendering shaders on the level of Sven Nykvist on The Sacrifice, but I am now hopeful). Unsurprisingly, all of this subtle and-yet spectacular beauty takes place within a highly stylized rendering queue. I say unsurprisingly because it is only in embracing the artificial nature of the image construction that 3D simulation will find its way. We are clearly in the realm of animation, illustration even, and far from the realistic renderings that occupy more and more of the gazillion-dollar 3D shooter blockbusters currently on sale for $75 at your local supermarket. Perhaps, and this might be due to the algorithmic nature of the image, my cinematographic references should instead be harking back to the history of animation, and not live action. For it looks as if we are inching ever closer to the visual plasticity of a Brothers Quay production, à la The Comb or The Piano Tuner of Earthquakes. Again, not quite, not even close, but one can always hope. From this luscious tableau, we begin our journey from desert to mountain. The sophistication-in-simplicity argument comes from the translation of Passage’s 2D left-to-right narrative mechanics into Journey’s 3D near-to-far construction: the next “goal”, which is more the next “step” on your path, is often a barely perceptible building or outlying natural structure that acts as a beacon leading you forward. Amazingly, the mountain itself acts like a character in the story, through mere shifts in lighting and humidity levels which interact thematically with the evolving storyline and landscape to evoke different “moods” as we progress: at first ominous, then distant, ethereal, massive, brooding, violent, festive, and ultimately, ambiguous. //// I won’t linger any further into the journey itself because it is not really important to what is, I hope, shaping into my central thesis: namely that by removing the tedious goal posts as the motivating factor of a game and replacing it specifically (and quite explicitly given the title itself of the game) into the space in-between — i.e. into that aesthetic field where the gameplay can unfold –, the game designers have further evolved a narrative language that was originally suggested by Colossal Cave Adventure, re-articulated in Super Mario Bros. and then Zelda, and finally brought into the more literary realm of figure and metaphor via Passage. //// One of my favorite cinema sequences comes from the third chapter of Akira Kurosawa’s end-of-career tone poem, “Dreams”. The dream is entitled “The Blizzard” and depicts a group of exhausted explorers climbing a mountain ridge in a snowstorm, haggard, nearly extinguished, and clasping to their last breaths. The scene is interminable, and often shot with a telephoto lens from the side so as to further flatten their gasping faces into the underexposed terrain that visually engulfs them. All we hear are their panting breaths, and eventually their complaints amongst one another. No back-story, no character motivation, just the sound of desperate breathing and a vague image of faces dragging their bodies through the blizzard. There is a moment near the end of Journey (although not quite at the end), when two voyagers, ours and the voyager accompanying us, are similarly pulling our heavy bodies up a snow-covered mountain. Our gait is troubled, weighed down by fatigue. We carry on, ever more laboriously, into the headwind. Eventually our body gives out, the controls disconnect from our persona who keels over, face-first, into the snow. It is a moment of inevitability, of loss of control, and yet everything about the scene feels just right. It is progressively clear to the player what is about to happen and yet we advance into the snow nevertheless, resigned to whatever the narrative is holding out for us. In a medium that tends to prize interactive mastery above all else, this loss of control from within the interactive realm comes as a refreshing relief. This is not some pre-rendered cut-scene superimposed into an interactive fiction; this is the interactivity itself leading to its own extinction as a sort of accomplishment. Exhausting gameplay. //// The second time I played through Journey, I was lucky enough to experience a lovely little poetic moment right at the end the game, as the two travelers approach the abyss. For those that have played the game (*at this point, it’s your problem if you have never played Journey and have herein spoiled any future experience of it), you will know that by pressing one of the buttons you can make a little melodic chirping sound, allowing you to communicate via very rudimentary means with your fellow traveller. So it was precisely at this point, just before the abyss engulfed us, that my fellow traveller and I decided to stop, not at first but eventually, in a back and forth choreography of following each other’s lead. At some point, we both simply decided to stop, just standing there before the abyss, the obvious ending point of the game. We had already travelled to this point and given that the game is cumulative, much like Passage, there is no real going back, even if one would want to. So in order to prolong the experience and simply take it in, we both at some point, through subtle character body-language, decided to just stop. At that point ensued several minutes of dialogue: “Piou piou?” “Piou piou.” “Piou piou piou?” “Piou.” While I generally try to avoid cinematographic analogies when speaking of gaming (I in fact loathe such uniformed discourse), at this point I’m so guilty of hyperbole that I might as well just suck it up and give in to the impulse: the moment was damn cinematographic, despite the obvious cheesy cliche-ness of it all. The experience just worked, and on an emotional level rare for me when playing a video game. It was an experience which I have only previously known through now canonic, well-worn aesthetic forms such as music, literature, painting, illustration, photography or cinema. //// Popular rhetoric, even within the video game community, gives great weight to the notion of “choice” in interactive narrative, as if choose-your-own adventure texts had somehow given us a heretofore unexploited key to some future form, despite the fact that no one seems to be interested any longer in the format. Maybe it is just because we haven’t tried enough angles. Perhaps. Count me in on trying all those other angles, and this is in fact precisely what we are trying to do at Media Design and at the Atelier Hypermédia. Some of the experiments students are trying out in this direction are profoundly exciting, in spite of all my doubts. And I do not doubt that there is indeed something yet-to-be-discovered in branching narratives, but in my opinion we are “digging in the wrong place” if we think that choice (more of it, better choices, etc) is going to enrich our experience of interactive narrative. I do not doubt the intent of games like Mass Effect, which prolong players choices all the way through three super-productions, taking the risk of alienating their players at the end of the game as they try to resolve an infinite series of narrative threads. I wouldn’t know anyway, I have’t even played the game, my students have. But given that the apparent solution to their dilemma is to pull out a Deus ex machina (cf. On the Media, New Endings) à la “it was all just a dream”, I figure my long polished reticence on this subject (15-years and counting) is still well founded. To further flog the comparison-with-cinema horse, Robert Altman never felt the need to wrap all his intersecting narratives into a tight little package. //// In the novel and subsequent film “Sophie’s Choice”, there is indeed a scene involving a profound choice that a woman has to make in order to save her child. And while this choice is indeed harrowing enough to be recounted and still retain something of its power, it is ultimately in the pregnant pauses between the lines that the true force of this choice is felt in the narrative realm. In the film rendition, it is on the infinite white screen of Meryl Streep’s face that we project all the horrors of her unbearable choice. The context of the choice itself is merely the frame upon which this face hangs. This we have known ever since the days of Lillian Gish in “The Wind” or Renée Falconetti in “La Passion de Jeanne d’Arc“. More recent film-makers understand this perfectly as well, for example Quentin Tarantino in the “La Louisianne” scene of Inglorious Basterds: we know going into the scene what the stakes are, and most of all we understand the strategic details of the narrative plot (underground bar, offset alcove with an eavesdropping nazi, etc); it is ultimately the dialogue as they intersect the faces, the how and not the why, the adverb and not the verb, that unpacks layer by layer the tension of the 25-minute scene. The whole thing is harrowing and entirely bad-ass, two descriptors that gamers would love to use to describe their games, and yet the entire weight of the scene hangs in the balance of the length of an vocal inflection. Lillian Gish, The Wind, 1928; via Bouchards Unfortunately for gaming — at least for the time being –, there is no equivalent to the face of Lillian Gish, especially when it comes to 3D attempts at realism such as Call Of Duty. Everything must be constructed by hand or through code. Motion-capture isn’t there yet, just watch any recent hollywood film. As a result, games are currently relegated to wide or medium shots of battle scenes, or over-the-shoulder renderings such as in Gears of War. While photography, too, was a originally an affair of landscape before moving in to portraiture, photography nevertheless has a natural relationship with the real, a relationship that algorithmic machines have not yet developed with the physical world and the bodies that inhabit them. //// Given these limitations, I find it encouraging that a new generation of game designers are beginning to experiment with the form of gaming on a level not only of narrative complexity, but also of narrative subtlety, perhaps even maturity. We obviously still find ourselves in a significant dichotomy between the games sitting on big-name physical/digital store shelves, and the more independent/auteur fare that often has to play the distribution game at the margins, via Flash-based websites, etc. But this has always been the case with previous media forms. Solutions are out there and the landscape is currently shifting anyway. Most importantly, it would seem the new generation wants to attack the issue of a more diverse narrative language in gaming. And while death seems to be one of the easier targets, it has been a central trope of the medium since nearly its origins, so why not. In any case, it unambiguously shifts the focus of the game back into the aesthetic and emotional experience of play. By removing all the goals, and keys, and puzzles to unlock, we settle in to a type of narrative where wider themes can be explored. Case in point, a recent exchange between Brooke Gladstone and Sebastian Janisz on the excellent On The Media: — BG: Your game was about depression, and this was the game that for me most fulfilled the goal, if that was your goal, to summon-up what it feels like to be depressed. There is the metaphor of beating your head against the wall. The second you get through to a new place, the same arduous process of beating your head against the wall begins again, or beating your head against various objects with little, very minor, spatters of blood, lest you forget that this is painful. And the whole thing feels very lonely. — SJ: Thank you. That really sounds a lot like what I would have hoped someone might get out of the game. — BG: You picture your own death. Or did I just… Spoiler alert! “Personal Video Games“, On The Media, March 30, 2012 //// For another reading of Journey, try Ian Bogost’s “Portrait of the Artist as a Game Studio“ ------------------------------------------------------------------------------------------------------------------------------------------------------------- This post first appeared on Douglas Edric Stanley's blog. For more interesting observations, […]
Posted on: 16/05/2011
Posted in: Theory
- Freelance Interactive Producers at Psyop
- Senior Digital Designer at CLEVER°FRANKE
- Interaction Designer at Carlo Ratti Associati
- Creative Technologist at Deeplocal
- HTML / CSS Developer at Resn
- 3D Technologist at INDG
- Creative Director at INDG
- Lead Developer at INDG
- Web Developer at &Associates
- Creative Technologist at Rewind FX
- Coder to collaborate with Agnes Chavez
- Data Scientist at Seed Scientific