Designing Programs [Theory]

This essay was commissioned by Centre national des arts plastiques for Graphisme en France 2012

Edited by Casey Reas and Chandler McWilliams

Technical mastery and innovation are part of the rich history of visual design. The printing press is the quintessential example of how a shift in design technology can ripple through society. In the Twenty-First Century, innovation in design often means pushing the role of computers within the visual arts in new directions. Writing software is something that’s not typically associated with the work of a visual designer, but there’s a growing number of designers who write custom software as a component of their work. Over the last decade, through personal experience, We’ve learned many of the benefits and pitfalls of writing code as a component of a visual arts practice, but our experience doesn’t cover the full spectrum. Custom software is changing typography, photography, and composition and is the foundation for new categories of design practice that includes design for networked media (web browsers, mobile phones, tablets) and interactive installations. Most importantly, designers writing software are pushing design thinking into new areas. To cut to the core of the matter, we asked a group of exceptional designers two deceptively simple questions:

1. Why do you write your own software rather than only use existing software tools?

2. How does writing your own software affect your design process and also the visual qualities of the final work?

The answers reflect the individuality of the designers and their process, but some ideas are persistent. The most consistent answer is that custom software is written because it gives more control. This control is often expressed as individual freedom. Another thread is writing custom software to create a precise realization for a precise idea. To put it another way, writing custom code is one way to move away from generic solutions; new tools can create new opportunities. Experienced designers know that off-the-shelf, general software tools obscure the potential of software as a medium for expression and communication. Writing custom, unique tools with software opens new potentials for creative authorship.

LUST / lust.nl

In our studio, form is a result of an idea, which is the result of a process. When approaching a project we try to be as open as possible. Through research and analysis we let the idea emerge from something already embedded in the project itself. Something that was perhaps already present, but that needed to be highlighted. From there we look for the best way to execute that idea, and in doing so develop the form and concept further.

Because we approach projects in this way, existing software/tools are often insufficient to properly execute an idea. We also tend to arrive at ideas that require new ways of thinking about how to deal with everything from typography to data to interactivity. In these cases the development of custom software and tools is a natural extension of the process, and can be instrumental in the development of the idea.

While designing in code is quite different from ‘traditional’ design methods, these kinds of processes have always been present in our work. Since we started our studio 15 years ago, we have adhered to a process-based methodology in which an analytical process leads eventually to an end-product that designs itself. This coincides very well with the idea of writing your own code and building your own tools. The transition to these kinds of working methods from more traditional approaches was a very natural one.

At a certain moment you realize that there is no other way to execute an idea than to build it yourself from the ground up. This frees you from the constraints of pre-packed software and allows you to maintain a closeness to your ideas that wouldn’t be otherwise possible. While any medium will have an impact on the visual outcome of a project, we feel that building your own project-specific tools gives you back the opportunity to control and manipulate the inherent visual qualities of the tools your using. In the end, the visual quality of a work should be relevant to the project itself, rather than rooted in a particular approach or technique – the outcome should speak for itself.

LUST’s cover for the book Form+Code in Design, Art, and Architecture is generated from frames taken from the movie 2001 A Space Odyssey. Together with the back cover and front matter of the book, the circle is revealed as an O, spelling FORM+CODE.

Nicholas Felton / feltron.com

A few years ago my work became almost exclusively data-driven and my design process became increasingly centered on a rules-based approach. I developed a set of processes for creating maps and charts that were effective, yet laborious and time consuming. It soon became apparent that in order to produce more and to tackle larger data sets, I would need to find a way to automate the routines I relied on. With Processing, I have been able to design applications that channel my methods, instead of bending my approach to work with existing software. These applications are accountable, so that if the output doesn’t match my expectations I am able to audit the code and find the issue. They are also inherently malleable, allowing me to mold the code to fit each project.

When I first began writing software, the programs I designed simply allowed me to do more of the same work in a shorter period of time and in a more flexible manner. As a result, the final product was not impacted by the use of software. With more practice and familiarity with the tools, I have started to produce work that would have been unfeasible or impractical using manual methods. I have experimented with maps that rely on difficult algorithms and developed tools that allow me to test a range of variables before outputting a final visualization.

Spread from the 2010 Feltron Annual Report, designed by Nicholas Felton. The 2010 Annual Report catalogs the life of Felton’s father, who passed away earlier in the year,

Amanda Cox (The New York Times) / nytimes.com

Mad Libs is a game where key words in a short story have been replaced with blanks. Players fill in the blanks with designated parts of speech (“noun”, “adverb”) or types of words (“body part”, “type of liquid”), without seeing the rest of the story. Occasionally, hilarity ensues, but no one really believes that this is an effective method for generating great literature.

In the same way, fill-in-the-blank templates rarely generate great news graphics. Admittedly, generic solutions work perfectly well in some cases. For example: “The _______ (stock index) has fallen _______ (adverb) since _______ (year). [Line chart].” But these are rarely the sorts of graphics that reveal anything unexpected or inspire new ways of looking at the world.

Instead, some of the most compelling news graphics exploit structure that is unique to a particular data set. This may require more control than a prepackaged solution, as the same form applied to a different topic would reveal nothing of interest.

But if you are nimble enough with points and lines and text, you can do pretty much whatever you want. And this means you get to spend your time exploring and dreaming and wondering “what if” and instead of trying to override the default choices in a software program. If you get lucky, it might even infuse the work with a sense of wonder.

This news graphic by The New York Times shows discontent with the political party in power. Nearly all districts voted more Republican (red arrows) in 2010 than they did in 2008.

Erik van Blokland (LettError) / letterror.com

When I started developing my own tools, I became more critical of existing tools. I had less patience for limitations imposed by others. Building tools offers a powerful perspective on design: the code is there to serve the idea, not the other way round. It means fewer compromises, and when there are compromises to make, at least they are mine and easier to live with. I don’t think I am a purist, I will happily use existing tools if they are fit to the task (with wildly varying criteria). But the idea, the direction of a design, should lead the process, not the arbitrary limitations imposed by existing tools. When something can’t be done with a specific tool, one should try to improve it or build a better one, but not necessarily compromise the idea. Good ideas are rare, we need to patiently farm millions of them to find one. Killing them before they grow is wasteful.

In some of my projects the code gets to touch the shapes. Like filters that synthesize detail or generate patterns or texture. Complex things can be generated (relatively) quickly and evaluated. Doing such things “by hand”, even while using a computer could take much longer, forcing me to commit to a set of parameters without realizing the full implications. These kinds of projects are very specific, personal and close to the design. Write, generate. Evaluate. Tweak (code, parameters or both), generate again, repeat. Design iterations are code iterations, the code is as open ended as the design.

But often though my code seems to be behind the screen, not touching the shapes directly, enabling directions, whole trees of trees of designs. For instance libraries that standardize objects representing specific data, or an open, documented file format to replace a proprietary and undocumented “industry standard.” These abstractions makes it a bit harder (sometimes) to find the energy to build, but the tools that grow on them are very powerful. Here structure and collaboration are important. At which level are the things we agree on and can share, and where are we going to go our own way? These are very interesting questions. Difficult to answer, but part of a very interesting discourse about creative work (design, art, more code) about the methods we have (and the ones we’d like).

The top letters are from typefaces that have been designed and generated with Erik van Blokland’s tools: Trixie HD, Eames, Federal. The images below are Robot fonts created with Just van Rossum, the Superpolator color field and the roboFab object model.

onformative / onformative.com

Existing software often restricts implementation possibilities and can even predetermine solutions by dictating what can be done with these possibilities. By writing our own software we break through such barriers and simultaneously create new ways of working with the design process. This process, in which the tool grows and develops with the design, is what excites us.

Of course, we also use existing software when it makes sense to do so, because we believe the skillful combination of existing software and our own software is the most effective way to reach the best results. Ultimately, one rarely writes everything anew but rather builds on existing components, or takes elements from the libraries or code snippets of others. One combines the existing with the new, and, through this combination, creates new results based on one’s own ideas. This is possible thanks to the active exchange in communities like the processing.org forum.

The design process is no longer divided into concept, design, and production, but rather the design process blends with the production process and the product is created in many small iteration steps in which idea, design, and programming are always closely entwined. When writing one’s own software, the creative work and the implementation of these are mutually dependent, and the separation of design and production is abolished. Because one has very different insight into the working methods and detailed processes of one’s own software, there is far more room for experimentation at one’s disposal, which has a direct effect on the quality of the work.

“Fragments of RGB” by onformative is a disintegration of an LED screen as an interactive installation and a series of photographs of the transformation.

Catalogtree / catalogtree.net

You are what you eat. We write our own software mainly to automate repetitive tasks but it is also important to us as a part of our ongoing attempts at experimental tool-making. Choosing a production technique is an important design decision to us and building tools, hardware or software, is a way of avoiding obvious choices.

But this process is uncertain, tinkering really. We are amateurs at it by choice. So not all tools find their way into commissioned work: we still have no direct use for our iPod controlled carrousel slide-projector, a fairly accurate push pin firing device for moving targets, a 3D scanner, water rockets or a swarm of vibrobots to draw on a printing screen. But what is important is the belief that the most beautiful sites are just outside the reservation.

Programming is the process. Some time ago, we spend a year working on separate locations. Daniel worked in an office building in Rotterdam (great view), and Joris was working from a studio in his backyard (whistling birds). We designed by talking on the phone. Though it was not ideal at all, it wasn’t completely unnatural to us either. We have our own vocabulary when we discuss projects and sketch by describing designs to each other. This means that most of our designs are language-based and finished before becoming visual. Over the phone, it is of no use to describe every single page of a book, every margin, every adjustment to the kerning of a headline. But to describe a system that generates from the smallest information unit in the content a possible flock of birds, can take five seconds.

In swarm systems, the behavior of one unit does not predict the behavior of the swarm as a whole. We aim at designs that have some swarming capacity, where we know what the smallest information unit should do but not what the final design might look like. To us, a good design is more that the sum of its parts.

However, good design also means picking the right pink and the right typeface. We will not follow an algorithm in a dogmatic way if it generates the wrong results. It is not in the first place about what the rules are, it is about what you do when you end up in a place not covered by those rules. There is room for a cherry on top.

Catalogtree building a crystal radio beneath the gaze of their Thomas Castro woodcut portrait.

Boris Müller / esono.com

It is quite intriguing that most of the software tools we use in our everyday life resemble a specific activity from the analog world – or even the analog past. Even creative, visual work on the computer is still based on manual input. The designer uses software tools to manually produce a formal output.

But creativity is not only about manual work – it is also about ideas. And in terms of ideas, software is a vast space. Like any other language, programming languages are about expressing ideas. They allow one to create enormous complexities that remain consistent and stable.

So instead of manually crafting an image, I generate an idea in a formal language and turn this idea into any number of images.

Beautiful ideas do not necessarily generate beautiful images. In the design process, I have to work on two different levels. The first one is about turning the idea into an abstract system. The second one is for the translation of the system into a visual form. The translation process is not deterministic. Sometimes it is obvious and strongly related to the abstract system – but very often I have to make a lot of design decisions that are purely based on the quality of the visual outcome. Turning an abstract idea into a meaningful image still needs the mind of a designer.

Generative visualization of the poem “Nr. 12” by Eugene Ostashevsky

Jonathan Puckey / jonathanpuckey.com

I love the feeling of being immersed in the functioning of a visual language of my own making. I put on my developer hat and think about the features and limitations I need as a user. Balancing the authorship I embed through the conceptualization and engineering of the software and the authorship I or others can create by using the software in different ways. When I use my tools, I want to forget about the underlying complexity of their functioning and focus purely on mastering they way my input sparks an output.

It splits up the process of design into two. I design the tool and then use it to design with. Often I am able to catch the concept of the project I am working on in the functioning of the tool, making it malleable and explorable while designing with it. I consider my tool based works successful when the viewer is able to visually recognize the collaboration (or even struggle) that is happening between the simplicity of the tool and the complexity of its input.

There is No Thirteenth Step designed using Lettering Tool, a Scriptographer typography tool by Jonathan Puckey in 2005.

Marcus Wendt (FIELD) / field.io

As a student I loved the new aesthetics of modern painting and architecture. I wanted to combine elegant and minimalist structures with complexity and emotional richness – as if bringing together Zaha Hadid with Gerhard Richter.

It took many attempts to realize that writing code could play a significant role in getting onto this route. Traditional design tools are following the aging metaphor of a single artist working tediously at his desk creating static images with high manual labour. The more I learned to code I realized how immensely dynamic working with these new tools can be — you can create living digital creatures; films that look different every time you watch, and design tools making 10,000 digital paintings in a day.

Writing code follows a bottom-up architectural approach and therefor emphasizes the process over the final result. Instead of working towards a single image, you start to think in the possibilities of a system. Designing a process rather than the end result forces you to be open for and work with unexpected results, and sometimes surprisedly embrace the outcome.

It’s hard to cheat when you’re working with code – before you can write something down, you have to clarify your ideas. It’s a bit like planning and building a house – with the major difference that once you’re done you can go back and change your foundation to get a dramatically different result.

Communion is a site-specific generative installation designed and coded by FIELD, with creative direction by Universal Everything
and sound by Freefarm. Image courtesy James Medcraft.

Sosolimited / sosolimited.com

Programming is a lot like cooking. When you learn how to do it yourself, you derive great pleasure from combining ingredients of your choosing and tasting the resulting dish. After a while, it becomes second nature and you no longer have to rely on processed foods for nutrition. Another similarity between cooking and programming is that they are both powerful instruments of seduction.

MIT was instrumental in teaching us this way of thinking. If your tire gets a flat, why buy a new one when you can re-invent the wheel? We were not taught explicitly how to use off the shelf programs. Come to think of it, they didn’t even teach us how to use computers. The attitude was: build whatever it takes to do what you want to do, but use technology to do it. So naturally, when it comes time to design, we write software.

When you’re writing your own software, the design is never set in stone. It is a constant improvisation. We don’t always know how our changes will propagate, but a deep trust in the process and a willingness to play often lead to wildly unexpected and pleasing results. It’s like handing your child a marker, writing down a list of things he can and can’t do, and letting him loose in your living room.

The visual qualities of our work reflect the structures, iterations, recursions, and limitations of code running on a computer. If we are looking to create an organic visual, we might actively work to hide the digital origin. If we are trying to reveal structures in a stream of information, we might embrace and amplify these same coded qualities. No matter what though, the final work looks the way it does because it is a continuous extension of the thinking machines that made it.

Prime Numerics by Sosolimited was a live remix of the final UK Prime Ministerial Debate on 29 April 2010.

Trafik / lavitrinedetrafik.fr

Graphics and programming are at the very heart of so many of our projects and this association has been the founding basis of Trafik (since 1997). Right from the beginning, we have believed that programming could be used for creative purposes, even if programming languages have essentially been devised to make tools. When we write a program, we are faced with technical issues which we have to address: to resolve the code in order to guarantee the running of the program. However, by studying the generated esthetic results and by making our own visual choices, we have thus adopted an artistic approach.

To apply programming to graphic design is an unusual approach by its very nature. In reality, the code, used as the base material, is abstract and disconnected from the generated forms. To write a code, to compile it, and to see it generate itself into tangible shapes, creates a sensitive rapport with programming. Used in such a way in graphic design, it enables us to develop artistic objects which outmatch existing tools. However to constrain ourselves to produce our own instruments is an empirical method which gives correct, precise and adapted results but which also sometimes provokes unexpected results. Thus, the code, by the complexity and diversity of what it can produce, manages to surprise us and to go beyond what we imagined at the outset. For example, the code often generates a unique esthetic which produces a sort of visual radicalism devoid of any sophistication.

By using programming, the creative process seems to us to be more complete: when producing visuals, installations or animations, we work first on the functioning of the program, on its “life”. The project builds itself throughout the development, through a permanent exchange between suggestions from the graphic designer and those of the programmer. These two professions and their interactions inspire all of our creations and produce specific and precise objects of art.

Pierre Rodière, Graphic Designer / Joël Rodière, Programmer


Casey Reas is a professor in the Department of Design Media Arts at UCLA and a graduate of the MIT Media Laboratory. Reas’ software has been featured in numerous solo and group exhibitions at museums and galleries in the United States, Europe, and Asia. With Ben Fry, he co-founded Processing in 2001. He is the author of Process Compendium 2004-2010 and co-author of Processing: A Programming Handbook for Visual Designers and Artists (MIT Press) and Getting Started with Processing (O’Reilly).

reas.com | users.dma.ucla.edu/~reas/

Chandler McWilliams is a writer, artist, and programmer. He has studied film, photography, and political science; and completed graduate work in philosophy at The New School For Social Research in New York City. He lives in Los Angeles where he teaches in the department of Design Media Arts at the UCLA School of the Arts. His current work focuses on themes of affect, repetition, computation, and epistemology.

brysonian.com

/++

/+

/++

4 comments on “Designing Programs [Theory]

Click here to cancel reply.