Created by Valère Zen Ruffinen at ECAL, Memoria is an experimental photo album application that visualizes the fading of human connections: as the frequency of shared moments decreases, the people in the photo album gradually disappear.
Memoria explores the fragility of memory, and how we maintain — or allow to fade — our connections through it. Through a process of gradual disappearance, the people in the photos slowly fade if no new memories shared with them are added. To keep their faces visible, users need to regularly enrich their album with new shared moments.
All those moments will be lost in time, like tears in rain.
Roy Batty, Blade Runner (1982)
When the photo is taken, Memoria analyzes it to identify the people in it using a face detection algorithm. Each person is then cropped out and grouped together in a shared mask. This mask is sent, along with the original image, to an external Magic Studio API, which processes it and returns a cleaned-up image with the individuals removed. Over time, Memoria gradually alters each photo based on the frequency of shared memories with the people appear in it. Each individual detected in the image is assigned a time-based value that decreases as weeks pass without new shared moments. When this value reaches certain thresholds, a visual noise effect is applied to the area corresponding to that person — parts of their face or silhouette begin to disappear in an irregular way. As time goes on, the noise intensifies until the person completely vanishes — unless new shared memories are added, which restore their image.
Memoria is a web application, built with HTML, CSS, Javascript and Python for face recognition and person segmentation. Valère also used Three.js for the gallery and to generate the noise effect that creates the fading of people in the photos. For the image clean up process, ClipDrop API was first used during the development, but later switched to Magic Studio API since it produced better results.
Valère Zen Ruffinen | ECAL Media & Interaction Design
