In Japanese folklore, tsukumogami are everyday objects that acquire spirits after long periods of existence, a belief that offers a striking counterpoint to today’s throwaway culture. Synthetic Spirits is a diptych of kinetic installations that transforms screens into digital lenses, revealing this hidden spirit world.
One sculpture features a sake bottle, the other a straw sandal. Both are inhabited by their respective tsukumogami spirits, Kameosa and Bakezōri, famously depicted in medieval illustrations. The physical objects glide past the displays in a mechanical choreography, while generative AI conjures the yokai inhabiting them. Each pass unveils a unique iteration of the spirit, ensuring a constantly evolving presence.
The work was originally displayed in a historic sake store at the Setouchi Triennale in Japan. Fabrication was supported by Autodesk (Fusion for CAD) and Qidi (Plus 4 for 3D printing).
The installation merges mechanical design, custom electronics, and a hybrid generative AI workflow.
Mechanical Design: Design began by defining the motion using openFrameworks as a “mechanical sketchbook.” These code-driven sketches utilized levers and geared sprockets to simulate the choreography. Once refined, these sketches were translated into Autodesk Fusion to model the physical assemblies for CNC milling (wood), water-jet cutting (aluminum), and 3D printing (using a Qidi Plus 4). Prototypes were then used to refine the mechanisms further.
Generative Workflow: To create the spirits, antique sake jars and straw sandals were 3D scanned. The antique ceramics were unsuitable for the drive system due to their weight, so lightweight replicas were modeled in Fusion and 3D printed. The digital forms were then converted into depth maps and processed through a custom ComfyUI workflow. This allowed for aesthetic control while using ControlNets (depth maps) to constrain the generated spirits to the physical object’s shape. These 2D generations were then processed by Hunyuan3D 2.0 to create meshes. Hundreds of unique spirits were generated, creating a large library for the display software to draw from.
Playback: The display software, built in Unity, handles the rendering. The custom electronics are controlled by a Teensy 4 microcontroller, which drives the machines with a stepper motor. A rotary encoder in each piece keeps the digital lenses synchronized with the physical objects, triggering a new, unique spirit mesh to load each time the object passes behind the screen.
Project Page | Neil Mendoza | Instagram
