While the euphoric, dynamic forms of computational artist Marius Watz are probably quite familiar to CAN readers, the artist’s curatorial and educational undertakings should definitely not to be overlooked. As an extension of his practice, Marius consistently organizes prescient and formative exhibitions and workshops (see the upcoming Generator.x 3.0: From Code to Atoms) and often teaches within various design and architecture schools across Europe and North America. Marius just posted the above slideshow of an upcoming teaching exercise that he’ll be overseeing at the The Oslo School of Architecture and Design (AHO) and it is worth clicking through. Entitled “Screensaver Culture” the assignment is a critical reflection on the current potential of the screensaver given that the days of CRT monitor burn-in have long passed. In setting up the exercise, Marius provides a chronological lists of essential precedents including scrnsave (1983), After Dark (1989), SETI@home Classic (1999-2005) and reconsiders that lineage in light of contemporary computing (and data practice) trends to extrapolate a range of approaches that his students might employ. Check out Marius’ slide deck and assignment brief – it offers a surprisingly compelling retelling of an easily overlooked topic within the broader history of computer graphics.

How did we get from command line to computer interfaces we know today? PlosBlogs’s NeuroTribes offers an insight into the sketchbook of Susan Kare, the Artist who’s high-school friend Andy Hertzfeld, the lead software architect for the Macintosh operating system, offered a job to design fonts for the Mac.

Inspired by the collaborative intelligence of her fellow software designers, Kare stayed on at Apple to craft the navigational elements for Mac’s GUI. Because an application for designing icons on screen hadn’t been coded yet, she went to the University Art supply store in Palo Alto and picked up a $2.50 sketchbook so she could begin playing around with forms and ideas. In the pages of this sketchbook, which hardly anyone but Kare has seen before now*, she created the casual prototypes of a new, radically user-friendly face of computing — each square of graph paper representing a pixel on the screen.

Continue reading.... The Sketchbook of Susan Kare [News]

Kent Sheely recreates famous photos from World War II using a video game engine (2009).

Continue reading.... World War II Redux [Games]

Below is a very early digital 3D rendered film (possibly the first one, ever). It was created in 1972 by Ed Catmull (the founder of Pixar) and Fred Parke with a little help from Robert B. Ingebretsen.

The best part of this film is not even the 3D rendering itself, but the outtakes and “making of” footage that has been interwoven throughout, including footage of a plaster replica of Ed’s hand onto which he is meticulously mapping the polygon vertices that make up the three dimensional model (around 1:30). That’s really remarkable. The math that we take for granted for rendering 3D was being invented, real time, to create this video. (Ed’s credited for having working out that math to handle things like texture mapping, 3D anti-aliasing and z-buffering.)

Read more about this in detail on nerdplusart.com

(via kottke)

One of the interesting cultural by-products to have emerged from the American assassination of Osama bin Laden is the public’s sudden fascination with situation rooms. As noted by Alexia Tsotsis on TechCrunch last week, the photograph of Obama and his national security team taken during the raid has received millions of page views and inspired both serious commentary and dumb meme photoshop tomfoolery. Additionally, given the gravity of the action and the secrecy that allowed it to be executed so seamlessly, the media has revelled in celebrating every minute detail of the planning and management of the operation.

I’d been meaning to plug a project proposal by the Argentine artist Gustavo Romano that will be developed at the upcoming Visualizar’11 workshop in Madrid. Considering its mandate, it is undoubtedly the perfect moment to discuss this venture and I think it will serve as a useful point of entry into a related discussion on the visual representation of conflict and power relations.

Given the theme for this year’s edition of Visualizar is “Understanding Infrastructures”, the projects that have been selected for developmentexamine a range of supply chain, consumption and finance-related topics. The above image represents Gustavo Roman’s proposed The Psychoeconomy War Room Table (PWRT), a tangible interface for exploring global economic data. PWRT will utilize the open source computer-vision framework driving theReactable Live! musical instrument to create a collaborative workspace for exploring (quantified) international relations. Roman outlines the goals for his project as

… try[ing] to display the relationship between two or more countries in the world in terms of some specific social and economic variables. The proposal builds on the metaphor of the table of the War Room, the room where are discussed possible tactical moves in a military confrontation. Using a multitouch surface,reacTIVision and fiducials, persons using the table may choose to place different flags on stage of the global economy and they can visualize the relationships between these countries. We will use data related to flows of assets (goods and financial capital), human flows (migration and tourism), energy flows (fuel and food), information flows (corporate media and alternative media). We will collect the data from public websites like the UN, the IMF, the World Bank, the CIA, etc.

I think this is an utterly brilliant idea and look forward to seeing the proof-of-concept interface that Roman and his team prototype next month. Imagine being able to manipulate the contents of the World Bank data catalogue and browse the interdependencies between nations – it would be both engaging and illuminating. Given the project is currently only an elevator pitch, I’ll make a point of mentioning the work that emerges when the results of the Visualizar workshop are posted online later this summer. It is worth noting that PWRT is part of Roman’s larger Psychoeconomy project, an artistic platform for exploring global issues.

For those within striking distance of Madrid, you might consider applying to work on PWRT or any of the other selected projects. Medialab-Prado is accepting applications through June 12, the workshop runs over the second half of June.

The still of Dr. Strangelove in the lower right corner of Roman’s PWRT collage gets me thinking about how depictions of these strategic ‘command’ spaces have evolved over the decades. The above image is a still from Joseph L. Mankiewicz’s opulent 1963 film Cleopatra that hypothesizes what ‘real time naval imaging’ might have looked like during the Battle of Actium (31 BCE). For those unfamiliar with the downfall of Mark Antony, the military leader sailed his flotilla into a trap set by his nemesis Octavian (who had obtained vital intelligence from a defecting general). As Antony’s naval forces were decimated the strategists remotely monitoring the battle at their ‘war room table’ set the appropriate ship models aflame. The interesting thing about this fictional case study is—as it is pre-screen—it functions as a tangible interface…

…and the scale models of Cleopatra bring us back to the inescapable reference of wargaming. The cleverest of the anonymous internet situation room photo edits was a tight crop of the intensely-focused Obama wielding a Playstation controller alongside a Brigadier General hunched over a laptop; drone mishaps notwithstanding, perhaps this is our caricature of warfare for 2011? The absurd addition of a gaming controller brings to mind a 2006 sound bite by Henry Kissinger where he described the (pre-makeover) White House situation room as “uncomfortable, unaesthetic and essentially oppressive” – in this image, wargaming is pure playbour.

I still contend that the most engaging game mechanics that I’ve encountered is the visualization of empire in Sid Meier’s Civilization series. The above screen capture is the city management interface from Civilization V, where you see the players’ borders butting up and flowing around those of their neighbours as well as detailed informatics that reveal the yield of the landscape and chart out where expansion will occur. There is sufficient depth in Civ that competing nations are required to develop extremely nuanced trade and diplomatic relations to acquire needed luxury items and natural resources while forging strategic alliances. However, the game kind of falters in failing to represent these complex flows of goods, materials and capital visually – at times it can be quite difficult to determine exactly what is going on. This is why I’m so fascinated by the PWRT as it aspires to provides a handy interface for exploring the global economy as a field of vectors rather than relying on stale geographic representations of borders and trade routes. In Newsgames, Ian Bogost, Simon Ferrari and Bobby Schweizer describe this kind of open engagement as an exploratory graphic that

…shows data that is meant to be synthesized by the user independently of the creator’s expectations. Both [Edward] Tufte and Benjamin Shneiderman encourage the use of information graphics to offer multiple levels of granularity for maximum flexibility. Tools or controls allow the reader to arrange, filter, or zoom data.

Can the complexity of international relations be distilled down to a work surface or game environment? Presumably, but exploring this kind of data can only be as revelatory as the interface it is delivered in.

–

About this article: The Psychoeconomy War Room Table (And Other Situational Awareness Vignettes) first appeared on serialconsign.com on 2011-05-11.

About the Author: Greg J. Smith a Toronto-based designer and researcher with interests in media theory and digital culture. Extending from a background in architecture, his research considers how contemporary information paradigms affect representational and spatial systems. Greg is a designer at Mission Specialist, blogs at Serial Consign, writes a column on emerging technology for Current Intelligence and is a managing editor of the digital arts publication Vague Terrain. He currently teaches in the CCIT program (University of Toronto/Sheridan College) and at OCAD University.

Cascade is the latest project by NYTimes R&D department that allows precise analysis of the structures that underly sharing activity on the web. Initiated by Mark Hansen and working with Jer Thorp and Jake Porway (Data Scientist at the Times) the team spent the last 6 months building the tool to understand how information propagates through the social media space. While initially applied to New York Times stories and information, the tool and its underlying logic may be applied to any publisher or brand interested in understanding how its messages are shared.

The app is primarily an exploratory tool, Jer explains. NYTimes publishes more than 6,000 pieces of content every month, and the team can now analyse every sharing event involving this content using Cascade. Jer describes the basic app workflow:

- A ‘Story Mode’ which shows a set of stories, and their associated event cascades. These stories can be requested via keyword search, section search, or a variety of ‘interestingness’ metrics. This view has some low-level visualizations of activity over time which allow us to focus in on event cascades which might be particularly interesting.

- A ‘Cascade Mode’ which allows us to view the event cascades. The cascades build over time – one of the things we’ve been most interested in with this tool has the time-based analysis. Rather than seeing static views of the social graph, we can actually see the sharing networks unfold over time. This mode has three distinct views in which each cascade can be examined:
1) A ‘side view’ which shows all of the events over time, and uses the Y axis to indicate degrees of separation from the originating event
2) A ‘radar view’ which views the system from overhead and lets users identify ‘threads’ of conversation
3) A 3D ‘tree view’ which combines views 1 and 2

The tool is built in Processing, with a lot of help from Andres Colubri’s GLGraphics library and toxiclibs. It runs on any machine, but is staged on a 5-screen video wall. This ‘exhibition’ app runs in an automatic mode, in which it explores the terrain of available data and wanders through the various presentation modes. The wall can also be controlled by a custom iPhone app which is a fairly simple and sends OSC commands to the display system. The team considered using touch or gestural input to control the display but in the end this gave them the control they wanted while being able to use the interface at some distance from the screens.

All of the data is stored in a Mongo database, which they access through a Python API. They also used R quite a lot during the exploratory phases. The largest cascades they are currently loading have about 25,000 events. These are all rendered in 3D at full framerate (60fps) across 5 screens (6400×720) by a single machine. Jer suspects the system could handle trees of up to 50,000 events (all thanks to Andres & GLGraphics). The data that the team are currently using is a 2-week sample from July/August, but Jer says they will be moving to a near real-time data feed very soon.

The implementation used right now looks at the sharing of NYTimes content over Twitter but Jer explains that in fact Cascade is a system that could be used to model any kind of sharing activity. They’re already looking at implementing it for other Times properties (boston.com, etc. ) and will be testing it out on other sharing systems over the coming months.

If you would like to know more about the project, make sure you also check out Coverage on Project Cascade from Nieman Journalism Lab. Of course, there is also the Project Page at NYTLabs.

–

Jer will be presenting latest work including Cascades at Resonate, new digital arts festival taking place later this year in Belgrade. Other confirmed speakers are available here + sign up to the newsletter for more info available soon. You can also follow on Twitter or join the group on Facebook.

In 1978 Rem Koolhaas (wiki) published Delirious New York, a “retroactive manifesto” that wildly reframed Manhattan through a rigorous analysis of the street grid, the skyscraper and congestion while excavating the history of the “mythical island”. A few years later Ridley Scott’s film adaptation of Blade Runner (wiki) explored the limits of the human condition against a backdrop of decaying art deco, flickering neon and unchecked corporatism. Syd Mead’s legendary production design for this film induced a sense of speculative nostalgia that simultaneously demonstrates bleak skepticism towards the promise of the future while pining for a romanticized vision of the Los Angeles of yesteryear.(1) Delirious New York and Blade Runner clearly illustrate how scholarly research and cinema can selectively engage broad historical trajectories and recompile new narratives from fragments and ephemera to fundamentally alter the mythos surrounding particular urban environments – it is rare that we get to enjoy meditations on ‘the city’ that are so capably crafted.

In thinking about creative projects that explore memory and the city, one would be hard-pressed to find a more influential (or ambitious) work than Walter Benjamin’s Arcades Project. This masterwork was a meticulous examination of 19th century shopping arcades and Parisian city life largely driven by urban exploration, a reverence for the aesthetics of the poet Charles Baudelaire and countless hours spent at the Bibliothèque Nationale. Benjamin obsessed over the project from 1927 through his untimely death in 1940 and when his unfinished manuscript was published in the 1980s, it was widely heralded as both a comprehensive documentation of the cultural and economic changes caused by the industrial revolution and a prototype for free form historical research.

Built during the 1820s and 30s, the Parisian shopping arcades were exciting and progressive spaces. Utilizing state of the art glass and iron construction technology, gas lighting and heating, these ‘interior avenues’ reconstituted the complexity of street life as an architectural project. Never before had such a variety goods and services been under one roof, and this density of stimulus must have been utterly intoxicating. Anne Friedberg has described the significance of this visual overload as follows:

“Hats, umbrellas, gloves, and cloth materials were displayed in shop windows and vitrines as if they were antiquated objects in a natural history museum. The passage was not a museum or a warehouse, but a sales space where the purchase was a transaction endowed with near-philosophic significance. Commodities were transformed into souvenirs, memory-residue of the already passé.”(2)

The Arcades Project was a sustained investigation of this ‘theatre of purchase’ that uses these retail districts as a lens through which to consider the history of aesthetics, economic and social relations, technology and urban design of Paris. Given this contextualization, an inevitable question arises: in an era of increasingly mediated urban experience, what strategies and tactics can we glean from Benjamin’s preoccupation with the arcades? This query is best answered by turning our attention to the structure and organization of his manuscript.

[The Arcades Project, overview]

It only takes a few moments of leafing through the Arcades Projects to realize the text is far from a standard historical treatise. Acting as an archivist rather than an essayist, Benjamin examined Paris through collecting short fragmentary thoughts and ‘filing’ them according to a broad thematic taxonomy. Iron construction, flâneurism, photography and fashion were all used as ciphers for understanding the day to day street life and broader architectural and economic transformations that occurred during the 19th century. Many of these snippets of text were romantic, enigmatic observations penned by the author, but the vast majority of this content were excerpts culled from literature and poetry, journals, newspapers, social theory and historical documents. Benjamin’s experimental technique was bold and nonlinear and could be considered as anticipating the rhetoric of hypertext and speculating how a sampling-based approach to historical scholarship might play out. Sifting through the hundreds of entries in the tome is a revelatory experience, in many ways Benjamin created what can only be described as a meta-guidebook.

The Arcades Project is a key precedent for thinking about the passage of time and the city because it so capably leverages bits of granular content to delineate a broad range of interrelated social phenomena. Benjamin created a system that demands a data miner rather than a reader, and while this kind of media artifact was an anomaly at the time, it is now completely commonplace – look at how services like EveryBlock schematize news and events. A contemporary city dweller might use the new foursquare recommendation engine to find a restaurant to meet friends at for dinner, plot directions on a GPS device for the drive across town, use a RFID passcard to access a toll highway, dine under the watchful eye of a CCTV network and then upload geotagged photographs of the proceedings to flickr. In varying degrees, we are all now authoring these inventories of interactions across public and mediated space – this is before we even broach the topic of open municipal data and the transparency and the civic engagement it engenders. The generation and management of metadata and media libraries is now routine and this is the backdrop against which artists, designers and scholars develop tools to represent and call into question the nature of urban experience. Everyday ritual and ephemera, emergent narratives, archive-induced anxiety and the ubiquitous timestamp – the Arcades Project is practically a user manual for codifying personal and shared urban experience and tracking how the city changes over time. The following topical sketches describe three Benjamin-inspired discourses pertaining to memory and the city in the age of big data.

[4Gentleman / Tiananmen SquARed, Tank Man in situ]

History as overlay

Not every urban trauma ends up being ‘permanently acknowledged’ as a brick and mortar memorial, an emotional fixture embedded within the cityscape. For every event that is commemorated through architecture, public space or ritual ceremony, the recognition of many others are ignored or suppressed. One of the more infamous debates about the role and how memorials should perform was the controversy that erupted over Maya Lin’s proposal for the Vietnam Veterans Memorial in Washington, D.C. Although now revered, the manner in which Lin’s angled wall gently cut into the earth was widely panned as being “too abstract” or a “black gash of shame” when her design was selected in 1981. Concessions were eventually made and a more traditional figurative statue, Frederick Hart’s The Three Soldiers was added to the site in 1984. Although most events don’t have the the same emotional weight (or nationalist implications) of the Vietnam War, these kind of contentious dialogues prove how invested the public in how remembrance is expressed in the urban realm.

The above image is an illustration of 4Gentleman’s Tiananmen SquARed overlay for the Layar augmented reality (AR) browser. The application allows users to view 3D models of iconic scenes from the 1989 student uprising through their smartphones when visiting the appropriate Beijing sites. Given that the Chinese government continues to blot this revolt from the public record, this is a subversive albeit subtle intervention that will permanently alter the experience of the site for some visitors. In a supporting blog post, the authors describe their motives:

“Although it has been more than twenty years since [the] Tiananmen Protest took place in 1989, the authority persistently uses all means erasing the facts that Chinese people pursued democracy in this democratic and anti-corruption movement. In China, nowadays, young people are not aware the courageous actions, such as ‘Tank Man’ and erecting ‘Statue of Democracy’ facing Mao’s portrait on Tiananmen Tower, emerged during [the] student movement of 1989. Nonetheless, history should not be forgotten.”

Tiananmen SquARed clearly illustrates how platforms like Layar can be leveraged to archive historical information. In addition to foregrounding suppressed narratives and battle erasure, AR overlays (or map layers) can also be used to browse unrealized futures. The iPhone app Museum of the Phantom City: Other Futures (2009) allows an explorer of New York City handy access to images and information regarding a selection of unbuilt speculative proposals including Frank Lloyd Wright’s St. Mark’s Tower (1931) and Superstudio’s Continuous Monument (1969). In engaging this tool, a user equips themselves with what Geoff Manaugh succinctly described as an “architectural dowsing rod” and is drawn into the tension between the city that is and that which might have been. Furthermore, a user must travel to the sites of the various proposals in order to ‘unlock’ related content thus forcing participants to excavate rather than simply consume.(3) Although relatively constrained in scope (and admittedly smartphone-centric), these examples highlight how various media platforms can be deployed as time capsules to provide ready access to historical information that future urbanites might seek out, sift through or stumble across.

[Cassidy Curtis & Stamen / Graffiti Archeology, Bluxome St. wall in San Francisco - 2005 and 2007]

Ephemera/inscription

By focusing on the flow of commodities and bodies through Parisian arcades, Benjamin was able to carefully parse the ephemeral nature of urban experience. One only need watch the movement of a crowd on a busy street to understand that public spaces have a ‘refresh’ rate and their use and occupation varies tremendously depending on the season, weather or time of day. The city is rife with these kind of fluctuations and they happen so quickly or discretely that we often overlook them. In the same way that media can be used to commemorate historically significant events, it is also equally adept at logging at (comparatively) inconsequential changes. The above image is a composite of two screen captures from Graffiti Archaeology (2003), a project produced by Cassidy Curtis and Stamen that provides users with an interface for tracking graffiti activity on a number of key walls in San Francisco over the last decade. Once a wall is selected, a viewer can ‘scrub’ the timeline of available images to note the incremental addition of tags and partial or complete cover-up of a piece with a fresh mural – it is a fascinating example of vernacular ‘media architecture’ that seems straight out of Stewart Brand’s 1994 text How Buildings Learn. Beyond one-off photo archives, street art owes a tremendous debt to universally accessible web-based photo sharing services and blogs that allow the documentation of relatively short lived murals and stencil art to be archived, distributed and resonate internationally.

There are of course many other examples of media being used to inflect our inscriptions on and utterances across the urban landscape. Christian Marc Schmidt and Liangjie Xia’s Invisible Cities is a recently launched social media browser that geolocates and aggregates twitter and flickr activity. The application provides users with a first-person vantage point for exploring the narrative cartography of Manhattan through the delineation of nodes of content and “topic vectors”(4). If we were to expand the breadth of this project it starts to resemble some of the developments forecasted by Jeremy Hight in “Writing Within the Map”, an essay published on NeMe last year. In this text Hight offers a thorough and imaginative consideration of how publishing is becoming an increasingly spatial project:

“Publishing and distribution will soon also be in maps. Yes. The news stand is to also be within that red dot. You are here. But what is here? How many stories have been set in Chicago? How many essays have been written on the crumbling cores of cities like Detroit? … These places and all other places have many faces, aspects, and these speak to many voices, investigations and (re)iterations. So why not publish in these places? Why not in their maps as well?”

While Hight’s predictions are opaque, his outlook is invaluable when thinking about an endgame for the types of spatial narratives that emerging technologies (AR, increasingly accessible mapping APIs, etc.) might engender and how they may extend and complement more traditional notions of authorship. The city is a space of not only substantial but fleeting discourse: what tools are at our disposal for tapping into and exploring this chatter?

[Left: Police kettle citizens during the G20 Summit in Toronto, photo: Eldar Curovic / Right: Sukey]

Refuge in the crowd

This last discussion is not so much about the passage of time, but the evolution of power relations. The image on the above left documents one of the more widely publicized moments during the security debacle that accompanied the 2010 G20 Summit. In this picture, police have surrounded 200 citizens whose decision to visit a major downtown retail district during a global trade summit resulted in their being kettled for several hours. Kettling is an increasingly common police tactic where lines of officers surround and intimidate crowds into submission by containing them for extended periods of time. While the detainment minimizes bodily harm, it is a flagrant violation of civil liberties and effectively transforms tracts of the city into temporary open air prisons. In response to several instances of kettling conducted by the UK police during the student protests last fall, Sam Gaus and Sam Carlisle used Google’s My Maps functionality to provide real-time updates about police activity in London so that protesters could remain “safe mobile and informed”. A related suite of tools named Sukey was released two months ago to extend this functionality by providing demonstrators with the ability to interact with the service through various smartphone and SMS protocols. Users now have range of options for reporting and receiving information about which nearby road junctions are clear and obstructed and where police actions are occurring. Given that policing strategies for managing organized demonstrations have become increasingly draconian over the last decade, it only follows that we’d see a new breed of tools emerge that harness locative media and citizen sensors as a form of non-violent resistance.

Tim Maly wrote a brilliant summary of mediated resistance to G20 police violence that focused on the ubiquity of recording devices and sensors. The following particularly glib excerpt highlights the ‘disposability’ of individuals within a crowd:

“At highly documented events, the rate at which recordings are made far outstrips the rate at which we can view them. Any given photo or video can be lost but the loss is not that great. Any given observer can be beaten, arrested, even killed, and the loss is not that great. At least not that much greater than if it was any other participant.”

Although depressing to dwell on, this is the logic of Sukey – through presence and feedback, individuals work to increase the safety and decisionmaking capabilities of the collective. Conversations regarding citizen action against militarized urban space are hardly new, Benjamin dedicated an entire section of The Arcades Project to Baron Haussmann’s 19th century urban renewal program, which cut wide swaths through Paris in the hopes of sculpting an urban fabric that was more retail friendly and revolution-resistant. To paraphrase and expand on Benjamin: The mighty seek to secure their stature with cunning (fashion) and blood (police), the crowd responds with a many-eyed gaze (surveillance).(5)

Epilogue

While the streets of Paris don’t figure that prominently into the layered narrative of Christopher Nolan’s Inception (2010), they do function as kind of a staging ground where the base rules of the film’s dream logic are established. Conversations between protagonists Cobb and Ariadne lead into several bombastic CGI-driven scenes which culminate in the Parisian street grid folding in on itself, the city reimagined as giant setpiece. Right before this sequence the young architect Ariadne wonders out loud about the implications of her total control: “My question is, what happens when you start messing with the physics of it all?” Benjamin often described technology and progress as creating a universe (and city) of ‘phantasmagoria’ – an endless montage of illusion and desire. That definition still stands, so those of us thinking about how the presence of history might figure into new forms of representing urban experience had best heed her question.

The next post in this series will deal with DIY Mapping and Counter Cartography.

Notes:

(1) Mike Davis’ perfectly describes Rick Deckard as a “postapocalypse Philip Marlowe” in the concluding chapter of Ecology of Fear.
(2) Friedberg, Anne. “The Passage from Arcade to Cinema” in Window Shopping: Cinema and the Postmodern. Berkeley: University of California Press, 1994. Pg. 50.
(3) A detailed overview of Museum of the Phantom City: Other Futures is available here.
(4) See the related “Invisible Cities: Representing Social Networks in an Urban Context” in the Parsons Journal for Information Mapping (Volume 3, Issue 1) for thorough documentation of this project.
(5) A rejigged [E5a,8] from the Arcades Project

–

About the Author: Greg J. Smith a Toronto-based designer and researcher with interests in media theory and digital culture. Extending from a background in architecture, his research considers how contemporary information paradigms affect representational and spatial systems. Greg is a designer at Mission Specialist, blogs at Serial Consign, writes a column on emerging technology for Current Intelligence and is a managing editor of the digital arts publication Vague Terrain. He currently teaches in the CCIT program (University of Toronto/Sheridan College) and at OCAD University.

He writes in his essay On The Frustrations Of Collaborating With Artists:
..I was developing experimental programming languages and methods — terms loosely defined, in those golden days of Bell Labs, and thus “artistic applications” was a plausible use of part of my time. I did not have to defend that interpretation in detail to my superiors, which was fortunate because I’m not sure that any of us really knew what we were doing, or why. As Yogi Berra (is reported to have) said: If you don’t know where you’re going, you’ll surely end up somewhere else. Yes, indeed, for better or worse. Anyway, from these experiences, I think the issues and caveats cited in my intro above have turned out to be more or less valid, for reasons that I would now say come from the nature of art and the nature of people.

Few years ago, Zach Lieberman met Ken Knowlton through a magician, Marc Settaducati, who is also a toy designer. The three of them, Marc, Ken and Zach decided to collaborate on bringing the idea of something magical to the masses inspired by Ken’s previous work.

This project was really about Ken, and Ken’s artwork. He’s been doing mosaic work since the 60s, when he came to Bell Labs and saw different ways of making images computationally Zach writes. Marc’s role was to turn those creative ideas into a toy.  Zach’s job has been to push Ken’s algorithms, modernize his code, and to try to get the software to give the best possible results. The system was developed in openFrameworks to adjust and tweak the software. Team developed three different toys so far — a monochrome version of the Jigazo (which is out in Japan and now US/Canada), a color version which takes a images such as a painting and rearranges.

What it is really significant about JiGaZo puzzle (meaning ”self portrait” in Japanese) is that you can realize almost any image, with the enclosed 300 pieces of puzzles. The puzzle uses rotating method to generate 4 different pixels out of each puzzle piece. So, for the first 2 pieces, the total number of pictures is given by 4 x 4 = 16. Now, if you do the same thing with the third piece, you would have made 4 x 4 x 4 = 64 different pictures. So if you keep multiplying that how big a number is 4^300? It is about equal to this number: 10^180 – or the number 1 followed by 180 zeros, larger than the number of protons in the entire known universe (source).

The projects began with code that Ken has written in C / Dos (16 bit variables, text mode, etc). Zach ported this code to OF, and expand on the algorithms with alternative approaches. The team use the ofxControlPanel gui, and did a lot of adjustment and tweaking.  They also use a fair amount of opencv to process incoming images to make them work better in the system.

In Japan, that code is then made to work on a server, and all the interaction is through cell phones / email.  You take a photo of your face – or whatever you want to make a puzzle of and send a blank email to iphone@jigazo.com with the photo as an attachment. An email will arrive from info@jigazo.com after about ten seconds. Open it, click the hyperlink in the email, and wait for your browser to navigate to the page where you can progress to the image with a 15-by-20 grid of symbols that correspond to the symbols printed on the pieces of the puzzle. Line up the pieces according to the grid on the page to recreate your photo.

In the hasbro version of the software, there’s a cd-rom with a flash port of the code, done using alchemy, which allows chunks of the algorithm to be written in native code.

The puzzle is currently available in Japan and finding itself on the european and north american shelves thanks to Hasbro. Japan import is available on Amazon and Hasbro version via their site.

To learn more about Ken’s work see: knowltonmosaics.com. Made with openFrameworks

JiGaZo Puzzle videos on YouTube and here is a video of Ken in his studio shot by Zach.

Created by Roger Ibars (2002–2011), Hard-wired devices combines game controllers and alarm clocks. Daniel puts it beautifully “frees the controller from it’s console and lets the beauty of it’s interface to sing”.

Each item includes manufacturer, designer, serial number, newly mapped control details and much more..

rogeribars.com

In July 2003 Roger Ibars graduated with a Master in Interaction Design from the Royal College of Art in London. Here he initiates a research interest in how objects would be loosing interest in interacting with the users. He illustrated this research in the book “Self-made objects”, a collection of daily objects that experience their functions. These objects investigate a new area for interaction design where things take control of their functions and therefore use themselves.more..

/via danielrehn, retrothing.com

Created by Steve Varga and what originally began as his thesis in the MFA DT program at Parsons the New School for Design, Pennant is an interactive history of baseball now available for the iPad. Pennant’s rich interface allows fans to browse and view data from over 115,000 games that have taken place from 1950 to 2010. Seasons, games and events are graphically represented and visualised in a manner that takes them beyond the numbers.

We asked Steve about the app, how it came about, how it was made, design decisions and considering openFrameworks was used, we wanted to know the ins and outs…

How does it work Steve?
The app consists of two main parts, the application itself (written in OF) that lives on the ipad, and the data, which exists on external servers. The application it written almost entirely in C++ besides the basic GLES wrapper that OF provides. I wanted the experience to feel fluid and different, so really what I’ve made is a video game main menu, which was the intention. The application basically loads in a level of data (teams, seasons, games, single games) and lets you move horizontally within that level without having to reload anything. Once you choose to dig deeper into the data the program reaches out to the server and grabs the requested data, and then you move to the next level. This only happens moving forward, because while you have a number of choices moving forward you only have one choice moving backwards, which means that the app caches xml from previous views so they show up instantly. In summary the app is just a lot of GL programming with OF and a lot of data parsing into usuable objects. It isn’t extremely complicated per say but it was extremely time consuming.

What about the addons?
Other people’s addons:
ofxThread – loading data and displaying animation at the same time
ofxTweenzor – by Nick Hardeman for all animation
ofxBox2d – (Originally by Todd Vanderlin but I might have modified it a bit)
ofxVectorMath (included in oF)
ofxXmlSettings for reading files (included in oF)

My own addons:
ofxFreeType2 is a rewrite of the font engine for OF that makes things display crisper and true to photoshop @72dpi
ofxVtronImageTexture is basically an ofImage that clears its pixels after loading so the footprint is much smaller
ofxFileLoader is a class for loading files from a URL/network address using Poco and returning them as a string or saving them to disk

The data is a whole other story.

Obtaining the data?
The data that I am dealing with is mainly available in its rawest form as bare text files on Retrosheet. This data is pretty much unusable, but there are command line unix tools called Chadwick (named after the first person to invent the baseball box score) that will parse these files and return them in a usable .csv format. From there I am moving them into a structured MySQL database. Luckily there was a website and a really great guy (Wells Oliver) who are into this kind of thing and have written some Python scripts that integrate with Chadwick and fetch the Retrosheet files. I had to make some modifications (mainly add in playoff data since it wasn’t there) and getting MySQLdb up and running with MAMP was a bit of a pain, but otherwise it went smoothly. This process took ~2 hours to run or so, and I was left with a basic database. I then downloaded the Baseball Databank (baseball-databank.org) which was thankfully in MySQL format. This database has more basic information about teams and seasons vs individual plays.

Parsing the Data?
Once I actually had the data I figured out what I didn’t have and wrote PHP scripts to create new tables based on the existing data. Things like the standings for every team every day for 60 seasons. These scripts took FOREVER to run so a lot of times I would have to carry my laptop with me everywhere and open it up and resume the terminal process (using PHP CLI). Once I had absolutely everything I needed (or at least thought I did) I created an API that let me query some PHP files and return either JSON or XML. I’m using XML for the project mainly because it is built into OF.

This is when I ended up at version 1, which was presented last May for the Parsons MFA DT thesis show.

Version 2
Immediately after finishing school I was admittedly burnt out on the project and took a month or two to get my business started (Vargatron) and get client work. I was also in negotiations with a company to possibly license Pennant but thing ended up not working out and I think we were both happy to walk away from this in the end. This reinvigorated me to get the project out there and going, but I had some big problems: I had furiously coded it to get it done for a deadline and it was in many parts held together with duct tape. It would crash frequently and the data loaded slowly. So I spent December and January rewriting the program and rethinking my data strategy. The epiphany I had with the data was that the API requests were the biggest bottleneck, but the data never really changes. So I rewrote some of the API, added in a few extra things, and then let a script run for multiple days that saves out the entire database to flat XML files. I then hooked this up with Amazon AWS services (A3 and CloudFront) and the data flies now for super cheap. This took the stress off my servers and made the app a lot better.

What influenced the design of the app?
The idea behind the design of the app is that we have been viewing the same box score for baseball games for almost 100 years now. There has been an enormous push for live content to be at the forefront of modern technology, but after the game is over stats are tossed into a morgue, neatly tagged and put away for few to see. Meanwhile a few dedicated people have been collecting these stats and making them publicly available, but in a very dry, numerical format. I wanted to create something that took these detailed accounts and brought them back to life, made them feel like baseball.

The app itself is about, overall, memories. It isn’t about statistics per-say, but more about what memories are hidden in the statistics. The most rewarding part of the app is watching someone pick it up and find a game they went to 20 years ago and instantly start to remember it. The smells, the sounds, the mood, who they were with etc.

This also drove the look and feel. I am a huge fan of nostalgic baseball memorabilia, and there has been a large retro push in sports over the last 10 years, so this made the most sense to me. The project also originally started as a way to create new baseball cards using a touch interface, so this paradigm followed through to the menus.

Final words?
I think one thing that I’d like people in the data visualization world to get out of this app is that they are missing out on sports. I have been to a number of talks by people in the data visualization industry where they poke fun at sports or make comments like “Who cares about sports”. It is almost like a reverse of Revenge of the Nerds, where the nerds (I completely consider myself a nerd!) hate the jocks and don’t want to be seen as associated with them in any way. I don’t think that you have to like sports (or anything that you don’t want to), but you can’t deny that the amount of data that is available and untapped in the sports world is immense. There are an enormous amount of stories to be told if you give it a chance.

Many thanks Steve!

You can download/read Steve’s thesis document here (15mb PDF) and read more about the app here.

Platform: iPad
Version: 1.0
Cost: $4.99
Developer: Vargatron