Google’s got a new consumer hardware initiative. It’s a mobile phone with machine vision eyes, ultra-fast inner-ears and spatially aware brains. And around that 5″ Android reference hardware, they’re trying to coordinate the biggest party in Spatial Computing this side of the Wiimote. Could this be all your AR Kickstarters come true?
Project Tango is the first public move by the “Advanced Technology and Projects Group” (ATAP), a lab Google pick-pocketed from Motorola in their recent revolving door dance with the company (Google bought Motorola Mobility for $12.5bn back in 2012, and then sold it on to Lenovo less than 2 years later for $3bn, but not before pulling out some the most promising patents from the company). ATAP look like a group of computer vision and robotics experts, who are likely sharing coffee right now with the Boston Dynamics guys (another Google spin-in, famous for their robotic beasts which can walk on ice and run like a cheetah).
Heading up this promising parade is none other than DIY computing punk rockstar Johnny Chung Lee. He hit notoriety in 2007 with a string of fun and inspired Wii remote hacks, combining Instructables charm, computer vision clout and supermarket cheap. Next stop Microsoft, where he coddled the Kinect from birth to blast off, whilst slipping a $3k bounty to the open source community via Adafruit. His bounty called for the first open source hack to bring Kinect data to artists and engineers. Whilst many at Microsoft were focusing the Kinect’s eyes onto gaming, Johnny banked on the broader transformative effect of that high street gem, opening the border crossing for a generation of projects using 3D depth data.
Johnny’s marriage to big corporations and ongoing affair with the open source community might be what makes Project Tango a blast for artists and designers. Google sells Tango to itself and Android developers as the next generation phone tech for apps and games, whilst hackers will be using it as a new one-stop shop for measurement and tracking for interactive projects. And genre defying Android apps could offer an ideal cross-breeding ground for artists and designers to put out their wares, much better than say Xbox games for Kinect hacks.
The little touch-slab comes with the usual smartphone bits and pieces, and then brings to the senses a depth camera (think low resolution Kinect), a motion tracking camera (machine vision camera with wide angle lens, likely factory calibrated) and a couple of ‘Computer Vision’ processors (at least one of these seems to be Movidus’ Myriad 1 processor, for doing CV routines on low power platforms).
The final organ is a relatively new development, an Inertial Measurement Unit that works so fast (more than a quarter of a million measurements per second), that the device is able to figure out where it is even with its eyes closed. Project Tango teams up data from this souped up accelerometer with corrections from SLAM (a technique which calculates the phone’s position from camera image). Combined, this system serves up a robust stream of millimetre accurate location data for developers to work with.
The phone will know exactly where it is, what direction its looking in, and in all likelihood, what it’s looking at. Compare this to the iPhone 5G’s M7 motion processor which can just about figure out whether you’re running, walking or in a car.
If this phone was stolen, it could tell you when and how it left your pocket, how the thief made their getaway, and in which seat of which cafe of which mall they’re sitting in right now, probably trying to figure out what all those goggley eyes on the back might be.
One common challenge that Tango conquers is ‘pose recovery’, calculating where something is in a space and what angle it is at (technical details of Tango’s solution). This challenge has been notoriously expensive and time consuming in the past (think $20k Vicon/OptiTrack multi-camera rig). Whilst Tango’s one stop shop makes it an ideal ‘strap on sensor’ for many spatial challenges.
– Want to fly your quadrotor drone through a tightly controlled path? Strap a Tango on it.
– Want to know where somebody’s head is in an interactive installation? Strap a Tango on them.
– Want to measure where your camera is in the room as you take photos? Guess what? Strap a Tango on it.
– Want to navigate your intercontinental ballistic missile without borrowing GPS signals from the United States? Just strap a Tango on it (your mileage may vary).
– Want to track your skateboard runs so that you can make neat visualisations of how your board did fancy things? You get the point…
That last example references Design I/O’s Skataviz, which tidily demonstrates the ‘strap on sensor’ concept. An iPod Touch taped onto a skateboard tracks the motion of the board during the run. Custom software logs the accelerometer and gyroscope data, then a visualiser app interprets this to offer a new perspective on the form of motion. Tango takes this this type of dataset to the next level.
Another arts application might be 3D light painting, using the screen of the phone to paint onto a long exposure photograph whilst all the time knowing where the phone is in space.
The second set of Tango capabilities are its Kinect-like faculties centred around the depth camera. From what we can see, the depth camera in itself seems relatively low resolution, but this is likely a trade off with power consumption, portability and suitability to range of environments. This however gets much more interesting with the ‘fusing’ capabilities of the software, which can combine data from many angles to build up large detailed models of scenes. We’ve seen many iterations of this concept, but Google’s size and Tango’s portability could take this in many new directions. Tango partner company Paracosm say they want to “3Dify the world”, and this device might just be the ticket. No doubt there’ll be privacy issues since there’s new types of data here being gathered in new ways, and ultimately on a platform which is in the house of Google.
Whilst cutesy app Seene demonstrates an instagram-like future for 3D scanning using current gen phone technology, it also shows that 3D on the web is no longer unwieldy or out of place. No doubt Google are imagining some of the ways Tango feed into new rich shareable datasets of 3D content on the web. This device could scan the interiors of buildings if you took it out and waved it around, but also it could map out internal walking paths whilst still in your pocket. It can make lightweight photographic 3D assets out of things you hold in front of it, or positioned in the right place might identify back problems throughout the day.
There’s also a strong hint that more robots will be coming to the Tango party very soon, especially with Google’s recent purchase of Boston Dynamics, and quadrotor UAV company ‘3D Robotics’ getting excited in the Tango video.
Now that the the big tech companies are in the Spatial Computing space (Google’s Tango, Intel’s Perceptual Computing, Microsoft’s Kinect and Apple’s recent purchase of PrimeSense depth camera company), we’ll be seeing a lot of competition for capabilities and usefulness in this field. Each company has a mix of institutions on board, which may cause schisms in a discipline which up until now has been searching for relevance.
It would be handy to have one of these to take simple 3D measurements when lining up installations, to sketch out details on 3D scans and then share them with collaborators. We’re sure to see a raft of detailed AR game experience with this device, some handy measurements tools and 3D scanning bits, but as is the nature of these things, we’re about to see some not so obvious uses too..
Project Tango: http://www.google.com/atap/projecttango/
(Thanks to Hsing Wei)
