iOS, openFrameworks
comments 4

Second Surface – Multi-user spatial collaboration system

multiSofa (1) copy

multiSofa (1)

Collaborative drawing in 3D space has a long history at MIT and on CAN. Starting in the early 2000’s where translucent screen was used to observe three-dimensional objects drawn in space (sorry no link) to our own GD3D app published on the AppStore in september 2010 (free). There is general interest in populating virtual environments and we have seen a large number of projects in the past that do just this. Unfortunately this virtual space is very fragmented, each project relying on its own interface and technologies not to mention different devices used to navigate it.

Tangible Media Group at the MIT have developed T(ether) – a Spatially- and Body-Aware Window and the latest iteration of the research comes in the form of  Second Surface created by the new team including Shunichi Kasahara (Sony Corporation, MIT Media Lab), Valentin Heun, Austin S. Lee and Hiroshi Ishii (MIT Media Lab).

The project aims to create an environment for creative collaboration which can adapt to everyday environment. Second Surface, a novel multi-user augmented reality system fosters a real- time interaction for user-generated contents on top of the everyday environment. This interaction takes place in the physical surroundings of everyday objects such as trees or houses. Second Surface uses image based AR recognition technology that recognizes natural image as the target. Based on the AR recognition, they estimate the pose of user’s device from the target object. Then their system allows users to place three dimensional drawings, texts, and photos relative to such objects and share this expression with any other co-located users who use the same software at the same spot.

Our system can provide an alternate reality space that generates playful and natural interaction in an everyday setup for multiple users. Our goal is to create a second surface on top of the reality, invisible to the naked eyes could generate a real-time spatial canvas on which everyone could express themselves. We believe that our system can create an interesting and new collaborative user experience and encourages playful content generation. We also believe that our system can provide new ways of communicating within everyday environment such as cities, schools and households.

The Second Surface, is collaborative work between MIT Media Lab and Sony Corporation and was introduced at the Emerging Technologies, SIGGRAPH ASIA 2012 and was awarded the Emerging Technologies Prize.

Client mobile application and server application were built using OpenFrameworks.

More about the project here.


  • Roland Legrand

    so, is this available as an app?

  • Roland Legrand

    so, is this available as an app?

  • Roland Legrand

    so, is this available as an app?

  • harrisonweber

    So much potential here. Imagine this + Google glasses. A framework for leaving digital notes, grafitti, etc.

    Any idea when/if an app will be released?