Leave a comment

CLOUDS – Code, Creativity, and Immersion


Few interactive documentaries have the pedigree of CLOUDS. After a hackathon experiment into harnessing the Kinect for computational photography begat a novel framework for RBG+D filmmaking (the ‘D’ is for depth, if you were wondering), artists James George and Jonathan Minard dreamed up an ambitious application of their new form of wireframe videography, and set out to make a documentary delving into the nature of code and creativity. To gain insight they interviewed several dozen leading artists, curators, and instigators, and engaged in profound and personal conversations about software, technology, and culture. A successful Kickstarter, interview marathons at festivals like Resonate and Eyeo, institutional support from the Studio for Creative Inquiry and Eyebeam, and then—after considerable development—interactive installation and VR builds of the project were featured at film festivals like Sundance and Tribeca last year. Now, after four years of development CLOUDS has been released and is available for purchase. CAN has been exploring CLOUDS for the last six weeks and we feel obliged to go on-record and share our thoughts about this project.

During one of the documentary’s several hundred interview clips, Casey Reas comments on virtual reality and says “it’s not cinema, it’s not a game, it’s not abstract visuals—but it somehow puts them together,” in an attempt to identify that medium’s inherent weirdness. Ironically, this sound bite on VR perfectly frames the tropes and tensions running through CLOUDS, which is fundamentally fragmented in both its aesthetic and narrative. As recorded and reconstituted by DepthKit, each of its interview subjects is a chatty rendering (complete with ‘frayed’ corporal falloff), and each conversation has been chopped up into dozens of quotes and statements. While taking liberties with source material has always been the purview of editing, in this documentary it is taken to another level altogether and the ‘film’ makes use of a custom story engine to organize and sequence its content based on metadata. Yes indeed, CLOUDS is a generative documentary.

↑ Many voices, many visuals: CLOUDS is not lacking in perspectives or scenography

CLOUDS revels in abundance. Paola Antonelli, Régine Debatty, Zach Lieberman, John Maeda, Bruce Sterling, Chris Sugrue, Marius Watz—its interview roster is expansive and the knowledge it taps runs deep. This wealth of source material has been broken down into quotes, comments, and speculations, all expressed in response to high level questions. Does the network promote creativity? When is an artwork done? How does complexity emerge from simple rules? On launching CLOUDS in it’s default ‘play’ mode, a viewer navigates down a 3D tunnel and selects one of several floating questions (represented by a blue and white circular prompt) to begin their session. A selection of commentary related to the query is cued up and begins playing; sometimes the sequence of clips is serendipitous, ingenious, and neatly complimentary, other juxtapositions are repetitive and don’t really gel. it’s about the thrill of the hunt though, right? When a viewer has got their fill of a selected topic they can exit the scene by selecting new questions at the side of the current talking head (and torso) and deep dive into the next provocation of their choosing.

While DeptKit’s portraiture is compelling, Minard and George have wisely opted to change things up sporadically and provide some visual variety. An inventory of 2D and 3D interstitial animations sometimes cycle-in in lieu of a interview subject and these range from Minecraft-y landscapes, to unruly tangled ribbons, to mazes and cellular automata—many coded by the interview participants themselves. It seems as if the abstract animations are tagged with the same taxonomy as the interview snippets—so often you’ll get a voiceover narration that is a thematic match for a visual sequence, yielding some nice contemplative moments. Additionally, when ‘jumping’ between questions the interstitial load screen conveys a network diagram of the entire corpus, and you get a sense of the cartography of the sprawling networked conversation you are exploring.


CLOUDS inventory of conversations requires its own cartography and a distinct user interface

If CLOUDS default mode is ‘play’ it’s default user experience is on a standard display and while this is perfectly engaging it’s a pale flicker of what the documentary is like in VR. When viewed through the Oculus Rift DK2 the interview subjects have a much more tangible presence and an almost visceral texture, you feel a much more embodied relationship to the ebb and flow of the interstitial animations, and the interface is nicely tuned to your gaze (want to go somewhere? look there). This is the first VR project I’ve spent more than five hours in and—coming from this community—I find it oddly comforting to have the assembled ‘cast’ of interview subjects as tour guides for venturing through this proposition of what immersive documentary could be.

While the project shines in VR mode it has its quirks and idiosyncrasies that permeate all its iterations, many of them tied to interface and general navigation. While I’m all for new forms, the generative clip sequencing doesn’t always yield engaging results. Sometimes you can get a ‘bad run’ of marginally related clips. In one of my sessions Kyle McDonald flashed by and told me he could “imagine eye tracking embedded in all cameras” soon and then Julia Kaginsky appeared and wondered aloud about “who owns the internet?” Recombinatory narratives can be jarring, and navigating CLOUDS is kind of like gambling. Minard and George have provided an alternate ‘research’ mode to alleviate this by allowing viewers to scroll through an index of topics, people, and visuals and methodically drill down into whatever it is they are interested in—and this is a great workaround. That said, it feels weird choosing between efficient content navigation and immersion (research mode is not available in VR) but rather than describe this either/or situation as a fault, I think it actually underscores why CLOUDS is interesting and vital at this particular moment.

DepthKit setup
↑ DepthKit Kinect/HD DSLR rig and it’s OpenFrameworks-powered editing/post-processing software

CLOUDS is a generative documentary filmed with emerging computational photographic techniques that features multiple modes of navigation and VR support that has been released via a post-Netflix video distribution service—it’s 2015 and all bets are off about what media is or should be. Yes, the film industry has been beating the transmedia drum for years and latched on (much like a barnacle) to VR as ‘the next frontier’ in cinema but all that really means is they are worried about their relevance in light of emerging technology. CLOUDS is absolutely worth exploring because it is a weird hybrid that draws from several traditions and forges a contemplative place for reflection in the middle of all this uncertainty. It’s a software art fever dream art directed by GMUNK; it’s a provocation about immersive media before James Cameron crashes the party; it’s the faces and discourse of the 2012-13 creative technologist festival circuit frozen in amber (and that discourse is still about five years ahead of the curve); and it’s proof that the most inventive media making comes from independent voices who aren’t saddled with the baggage of market-driven (and medium-driven) modes of working.

It’s not often you find a documentary that is big enough to get lost in and alien enough in its aesthetics that the primary point of reference is videogames. While its chorus of voices have considerable insight about computation, creativity, and this cultural moment, CLOUDS itself does not offer any concrete answers about ‘what’s next?’ in media making—but it certainly poses some timely questions.

CLOUDS | Jonathan Minard | James George
Executive Producer: Golan Levin
Producer: Winslow Porter
Design Director: Bradley G. Munkowitz
Music: R. Luke DuBois
Lead Interaction Developer: Elie Zananiri
Lead Visual Systems Developer: Lars Berg
Story Engine Developer: Surya Mattu

CLOUDS, Casey Reas