Reversediffusion.xyz visualizes how AI-generated images emerge from a complex mathematical space built on millions of human-created training images. By zooming in on this space as a context for everything the model produces, it provides a reversed perspective on generative AI.
Where do AI generated images come from?
Reversediffusion.xyz is a project by artist Richard Vijgen that shifts the focus away from the output of generative AI to explores its origins. It’s widely known that generative AI relies on training data scraped from the public web, but the connection between this input and the final output often remains elusive. When you ask the model to create a picture of a landscape, the image you get isn’t copied from any single training image. Instead, it emerges from the complex mathematical space also known as latent space that lies between all the landscape images and their descriptions the model has learned from. Although the output can’t be traced back to one exact source, it can be positioned somewhere within this space of training data to understand the relations between the model’s input and output.
Reversediffusion.xyz visualizes the latent space of the open-source Stable Diffusion model by probing it with the very training data used to build it. Like many other models, Stable Diffusion was trained on the LAION dataset—a massive collection of billions of images scraped from the web.1 While the full dataset is enormous, many models focus on a curated subset called LAION-5B Aesthetic. This subset includes only images that another human-trained AI model has rated as having high aesthetic quality2. Though aesthetics are subjective, this filtering explains why generative AI often produces hyperrealistic and oversaturated images.
By encoding the 625,000 images from the LAION-5B Aesthetic >6.5 dataset into Stable Diffusion’s latent space, we can see where each training image fits within the model. Using the UMAP algorithm to reduce this complex, high-dimensional space to three dimensions creates a spatial map–a visual representation of the model’s structure that helps us contextualize its outputs.
On the page, you can generate new images with Stable Diffusion and see their position on the map. This shows where the synthetic image belongs in latent space and which training images–whose features may have influenced the output– are nearby. A generated image of a landscape might cluster with training images of landscapes, or it might group with images sharing a similar visual style, like oil paintings or black-and-white photos. Because image generation begins with a random seed, each landscape you create will be different and appear in different locations on the map. Once your generated image is placed in latent space, you can explore the surrounding training images. Click “see training images” to browse a selection of nearby training data3. Here you’ll find the original human-created images scraped from the web that informed your AI image. Clicking on any of them takes you to their original source. This way, generating an image exposes the human-made images that it originated from. And if you decide to use them, don’t forget to credit the authors :-).
The visualisation is based on LAION-Aesthetics dataset used for training many image generating AI models including Stable Diffusion. To place the training images in the latent space of Stable Diffusion 1.5, each image was encoded using both the model’s Variational Autoencoder (VAE) and the Contrastive Language-Image Pre-training (CLIP) to obtain it’s position in latent space. These high dimensional embeddings were then reduced to a threedimensional point in space using a UMAP algorithm. The resulting cloud is rendered using Three.js and visualizes the context for newly synthesized images. When a uses enters a prompt, it is sent via socket.io to a GPU backend that generates an image with Stable Diffusion 1.5 and calculates it’s position in the cloud. It is then sent back to the user to be visualised in the context of the training data that is close to it in latent space and might have informed the structure of the generated image.
The backend is running a python script for generating and placing images in the models latent space using a RTX 3080 GPU. The front-end is running a node.js server and uses three.js for visualisation.
- Most images in the LAION dataset were scraped from the web without the authors’ permission. ↩︎
- This project uses LAION-Aesthetics 6.5+, which filters images rated with an aesthetic quality of 6.5 or higher on a scale from 1 to 10. No images are hosted on this website. ↩︎
- The nearby image selection combines proximity in both UMAP space and the model’s latent space. ↩︎
