Created by Cambridge (US) based creative studio Design I/O (Theo Watson, Emily Gobeille and Nick Hardeman), Connected Worlds is a large scale immersive, interactive ecosystem developed for the New York Hall of Science.
The installation is comprised of six interactive ecosystems spread out across the walls of the Great Hall and connected together by a 3000 sqft interactive floor and a 45ft high waterfall. Visitors can use physical logs to divert water flowing across the floor from the waterfall into the different environments, where children can then use their hands to plant seeds. As the different environments bloom creatures appear based on the health of the environment and the type of plants growing in it. If multiple environments are healthy creatures will migrate between them causing interesting chain reactions of behaviours.
The installation is designed to encourage a systems thinking approach to sustainability where local actions in one environment may have global consequences. Children work with a fixed amount of water in the system and have to work together to manage and distribute the water across the different environments. Clouds return water from the environments to the waterfall which releases water to the floor when it rains.
Each environment in Connected Worlds runs on a separate machine with software developed with openFrameworks. There are a total of 8 Mac Pros running the whole experience. There is a total of 15 projectors used: 7 for the floor, 6 for the wall environments and 2 for the Waterfall. For the floor tracking the team are using fabric logs made out of a retro reflective fabric being tracked by three IR cameras. For the wall interactions they are doing custom hand tracking and gesture detection from above using 12 Kinects (two Kinects per environment). To eliminate perspective issues from using multiple Kinects they combined their point clouds into a single space and then use an orthographic ofCamera to look at them from a single viewpoint.
The eight machines driving Connected Worlds all communicate with each other using OSC and the broadcast port of the router. This allows all the machines to exchange information without needing a server / client relationship. When creatures migrate between machines they are turned into a series of ofParameters, added to an OSC blob as a zipped binary buffer, the environment they are going to migrate to then unzips the blob and creates a duplicate creature from those ofParameters.
Connected Worlds is made possible with generous support from The JPB Foundation, the National Science Foundation, Google Inc., and The Nasdaq Educational Foundation, Inc. Created with additional design and animation by Josh Goodrich and game design by Zach Gage. Sound design by M-OST