We all carry decades of history in our posture. Layers of inherited, performed, and forgotten cultural memories are etched into how we stand, point, or fold our arms: these motions are not arbitrary; they are scripts passed down through generations like heirlooms. Gesture analysis, once the realm of anthropologists and artists, has become embedded in daily life through pervasive technologies, transforming our everyday physical interactions into texts to be continuously read, interpreted, and police – but gestures also have an extraordinary ability to survive history’s disruptions, carrying meaning forward through time in ways that other cultural expressions might not.
These conceptual threads materialised in Uncoded Gestures, our exhibition at Palo Alto’s Escoleta building in Poblenou, Barcelona, where we reimagined participants’ bodies as both archaeological tools and sites of discovery. We experimented with AI to match gestures and body movements to archival images – a way to trace how we might be echoing the past without realising it. We began building a prototype for pattern matching using historical photographs from the city archives of Hospitalet and later Poblenou, both rich photographic archives from our hometown.
For the main installation, a camera mapped the silhouettes of every visitor, translating joints and angles into a constellation of data points. This digital skeleton is then compared to thousands of gestures extracted from historical photographs of the Poblenou archive – images of dockworkers, schoolchildren, lovers, and protesters, each frozen in a moment of unguarded, genuine self-expression. The algorithm performs a kind of séance: it identifies the participant’s closest kin across time, not by name or face, but by the curve of a spine or the tilt of a head. When a match is found, the historical image stands side by side with the live participant’s gesture — each showing such a striking resemblance that they seem to share a silent dialogue across time. Part archive, part software, part flesh, the image asks: who is haunting whom? Are we summoning the past, or is the past summoning us?
The installation was built around a real-time pose estimation pipeline. A camera captures the visitor’s full body, detecting and mapping skeletal keypoints into a structured vector representation of their posture. This digital skeleton is then compared against a custom dataset of gestures extracted from the Poblenou photographic archive, a collection of images spanning the early 20th century through the 1990s. The matching system uses a pattern recognition model trained on this archive, computing similarity between the visitor’s pose vector and those extracted from historical photographs. Rather than matching by face or identity, the algorithm operates purely on the geometry of movement, looking at the relative positions of limbs, the tilt of a head, the angle of a shoulder. The closest historical match is retrieved and displayed alongside the live image, creating the side-by-side projection at the core of the piece. The system runs locally on-site, with no external data transmission. The archive dataset was curated and annotated by our team in collaboration with the Poblenou photographic archive, ensuring the historical material was handled with care and contextual integrity.
Project Page | Domestic Data Streamers | Instagram
