Weather Worlds is a new interactive installation by design I/O that grants children weather controlling superpowers. Utilising cameras and real-time greenscreening via openFrameworks, the installation allows children to see themselves immersed in an interactive and dynamic environment. The custom computer vision system tracks the heads, hands, feet and movement of children on the platform and also recognizes gestures. Using their bodies children can conjure a storm, release a twisting tornado or rain down bolts of lightning from their fingertips. There are mighty wind fields to move through, stomping earthquakes, light bending sunshine and blizzards that will make you shiver!
Weather Worlds uses an HD CCTV camera, combined with a Blackmagic UltraStudio Mini to capture 1080p video of the audience standing in front of a greenscreen. The audience is then chromakeyed in realtime by the openFrameworks based software, which extracts the contours of each person and builds a skeleton of them, allowing specific gestures to be detected. The chromakey happens in two passes once on the CPU at low res for the tracking and analysis and then separately on the GPU as a shader to allow fast compositing at 1080p.
The skeleton tracking code will be released as an openFrameworks addon and a standalone app which others will be able to use in their projects.