Latent Space as a Material

A 512 by 512 pixel colour image is 786,432 numbers. When Stable Diffusion works on that image, it does not work on those numbers. It works on 16,384 of them. The model card is precise about it: the autoencoder "uses a relative downsampling factor of 8 and maps images of shape H x W x 3 to latents of shape H/f x W/f x 4". Sixty-four by sixty-four by four. Forty-eight times smaller than the picture.

Everything a generative image model does happens in that smaller room. The diffusion process, the conditioning on your text, the denoising steps you pay for in seconds: all of it is arithmetic on a 16,384-number array that is then handed to a decoder which inflates it back into pixels. If you want to understand what kind of material you are working with, start there, because the shape of that room is the shape of the grain.

What a latent space actually is

The idea is older than the current models and is not mystical. Kingma and Welling's Auto-Encoding Variational Bayes, posted in December 2013, framed the problem as "efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets". A latent variable is simply a hidden quantity you never observe but which you assume generated what you did observe. The trick of the variational autoencoder is to train two networks at once: an encoder that maps data to a distribution over those hidden quantities, and a decoder that maps hidden quantities back to data.

So a latent space is a coordinate system that a network invented for itself during training, in which every point decodes to something that looks like the kind of thing it was trained on. Nobody designed the axes. Nobody named them. The axes are whatever the optimisation found convenient for reconstructing millions of images cheaply.

This is the first thing that makes it behave like a material rather than a tool. Wood has a grain because of how the tree grew, not because a carpenter chose it. The latent space has a structure because of how the training ran, not because anyone specified it. You can work with the grain, across it, or against it, and the results differ, but you do not get to relocate it.

In the latent diffusion paper that Stable Diffusion is built on, Rombach, Blattmann, Lorenz, Esser and Ommer described the point of moving the work into that compressed space as reaching "a near-optimal point between complexity reduction and detail preservation". That phrase is worth holding on to. The space is a compression. Compression is lossy. The losses are not random, and they are not neutral.

Interpolation, and the thing almost everyone gets wrong

The most quoted property of these spaces is that you can move smoothly through them. Radford, Metz and Chintala's DCGAN paper from November 2015 is where a lot of designers first met the idea. Their figure caption reads: "Interpolation between a series of 9 random points in Z show that the space learned has smooth transitions, with every image in the space plausibly looking like a bedroom."

Their reasoning about why that matters is worth quoting in full, because it is a diagnostic claim, not an aesthetic one:

"Walking on the manifold that is learnt can usually tell us about signs of memorization (if there are sharp transitions) and about the way in which the space is hierarchically collapsed."

Smooth interpolation is evidence that the model built a representation rather than a lookup table. It is a test, and it was presented as a test.

The same paper is the origin of the latent arithmetic demonstrations that get reproduced endlessly: subtract the vector for a neutral woman from the vector for a smiling woman, add the vector for a neutral man, decode, get a smiling man. What almost never gets reproduced is the caveat the authors put immediately next to it: "Experiments working on only single samples per concept were unstable, but averaging the Z vector for three examplars showed consistent and stable generations that semantically obeyed the arithmetic." The arithmetic works on averages of examples. On single points it is unreliable. That is a fact about the material, and it is the sort of fact that gets sanded off in retelling.

Now the correction. If you have ever written a script to blend between two latents, you probably wrote a linear interpolation, because that is what interpolation means everywhere else. In a high dimensional latent space it is the wrong operation, and Tom White's Sampling Generative Networks (arXiv 1609.04468, 2016) explains why in a way that is easy to check. Latents are typically drawn from a Gaussian prior. In high dimensions, the probability mass of a Gaussian is not near the origin: it sits in a thin shell at a characteristic radius. White gives the concrete case of a 100 dimensional space, where random vectors have a length near 10. Linearly interpolate between two such vectors and the midpoint's magnitude drops to roughly 7, which he notes is more than four standard deviations away from the expected norm. His phrasing is exact: "linear interpolation traverses locations that are extremely unlikely given the prior."

The fix is spherical linear interpolation, which White describes as treating "the interpolation as a great circle path on an n-dimensional hypersphere (with elevation changes)". The practical consequence is that the middle of a linear blend tends to be mushy and grey, and people read that mush as "the model is bad at blending". The model is fine. The path was wrong.

Why the midpoint goes soft (White, 2016)

Prior:
    z ~ N(0, I) in d dimensions

Typical norm:
    |z| is concentrated near sqrt(d)
    d = 100 -> |z| is about 10

Linear blend:
    lerp(a, b, 0.5) = (a + b) / 2
    if a and b are roughly orthogonal (usual in high d),
    |(a + b) / 2| is about sqrt(d)/sqrt(2) = about 7
    7 is not "a bit low". It is off the shell the model was
    trained to decode from.

Spherical blend:
    slerp keeps |z| on the shell for every value of t, so
    every frame of the blend is a sample the model considers
    ordinary

Why the midpoint goes soft (White, 2016)

Prior:
    z ~ N(0, I) in d dimensions

Typical norm:
    |z| is concentrated near sqrt(d)
    d = 100 -> |z| is about 10

Linear blend:
    lerp(a, b, 0.5) = (a + b) / 2
    if a and b are roughly orthogonal (usual in high d),
    |(a + b) / 2| is about sqrt(d)/sqrt(2) = about 7
    7 is not "a bit low". It is off the shell the model was
    trained to decode from.

Spherical blend:
    slerp keeps |z| on the shell for every value of t, so
    every frame of the blend is a sample the model considers
    ordinary

Why the midpoint goes soft (White, 2016)

Prior:
    z ~ N(0, I) in d dimensions

Typical norm:
    |z| is concentrated near sqrt(d)
    d = 100 -> |z| is about 10

Linear blend:
    lerp(a, b, 0.5) = (a + b) / 2
    if a and b are roughly orthogonal (usual in high d),
    |(a + b) / 2| is about sqrt(d)/sqrt(2) = about 7
    7 is not "a bit low". It is off the shell the model was
    trained to decode from.

Spherical blend:
    slerp keeps |z| on the shell for every value of t, so
    every frame of the blend is a sample the model considers
    ordinary

I got this wrong for months. My blends looked soft in the middle and I blamed the model's "weak mid-range", which is the kind of sentence that sounds like craft knowledge and is in fact a bug report about my own code.

Why outputs cluster

Three separate mechanisms push a generative model's outputs together, and they are worth keeping separate because only one of them is a defect.

1. Mode collapse, which is a training failure

Ian Goodfellow's NIPS 2016 Tutorial: Generative Adversarial Networks gives the definition and, pleasingly for a design journal, the nickname: "Mode collapse, also known as the Helvetica scenario, is a problem that occurs when the generator learns to map several different input z values to the same output point." The tutorial traces it to an ordering problem in the adversarial game. If the minimisation over the generator ends up in the inner loop rather than the outer, then "the generator is thus asked to map every z value to the single x coordinate that the discriminator believes is most likely to be real rather than fake", and simultaneous gradient descent does not reliably pick the ordering you wanted.

This is a genuine pathology, specific to adversarial training, and it is not what is happening when a diffusion model keeps giving you the same lighting.

2. Guidance, which is a dial you are holding

Ho and Salimans's Classifier-Free Diffusion Guidance (July 2022) is the paper behind the guidance scale slider in nearly every interface. Its own abstract describes the result as "a trade-off between sample quality and diversity similar to that obtained using classifier guidance". That is the honest framing: turning guidance up does not make the model better, it makes it more obedient and less varied. If your outputs are converging, check the slider before you theorise about the model's soul.

3. The training distribution, which is the actual house style

This is the mechanism people mean when they say a model "has a look", and it is fully traceable in public documentation for Stable Diffusion 1.x. The chain runs like this.

LAION published a set of subsets called LAION-Aesthetics, described on their own site as "several collections of subsets from LAION 5B with high visual quality". The filter was an aesthetic predictor: a linear model on CLIP ViT-L/14 image embeddings, trained to predict answers to the question "How much do you like this image on a scale from 1 to 10?" The version 2 predictor was trained on roughly 176,000 image-rating pairs from the Simulacra Aesthetic Captions dataset, 15,000 LAION-Logos ratings, and about 250,000 photographs from the Aesthetic Visual Analysis dataset. From 2.37 billion English-captioned images, roughly 600 million scored 5 or above, 12 million scored 6 or above, and 625,000 scored 6.5 or above.

Then the model card for Stable Diffusion v1-5 records what was done with that: the checkpoint was "Resumed from stable-diffusion-v1-2 - 595,000 steps at resolution 512x512 on 'laion-aesthetics v2 5+' and 10 % dropping of the text-conditioning".

So: 595,000 training steps on the half of the web that a small linear probe, calibrated on a few hundred thousand human ratings, predicted people would like. That is the house style. It is not a mood. It is a filter with a threshold, applied to a scrape, for a documented number of steps.

The same model card is unusually blunt about the second-order effects: "white and western cultures are often set as the default", and "Texts and images from communities and cultures that use other languages are likely to be insufficiently accounted for." Bianchi, Kalluri and colleagues measured this in Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale, finding that "a broad range of ordinary prompts produce stereotypes, including prompts simply mentioning traits, descriptors, occupations, or objects", and, crucially for anyone hoping to prompt their way out of it, that "stereotypes are present regardless of whether prompts explicitly mention identity and demographic language or avoid such language".

A material with a grain will resist you in one direction. This material also has a default subject, a default skin, a default continent. That is not a metaphor about wood any more. It is a reason to check outputs against your own criteria rather than against how good they look.

The claim I repeated that turned out to be wrong

For a while I told clients, confidently, that these models do not store or reproduce their training images, because they are far too small to hold them. I had the DCGAN paper's line in my head: "Theoretically, the model could learn to memorize training examples, but this is experimentally unlikely as we train with a small learning rate and minibatch SGD."

That was a claim about a 2015 GAN, and it does not transfer. Carlini and co-authors, in Extracting Training Data from Diffusion Models (January 2023), ran a targeted attack on Stable Diffusion: they took "the 350,000 most-duplicated examples from the training dataset and generate 500 candidate images for each of these prompts (totaling 175 million generated images)". They recovered 94 images meeting their formal extraction criterion, plus 13 further near-copies found by inspection. Their comparative finding is the one to remember: "diffusion models memorize 2x more than comparable GANs", and "diffusion models are much less private than prior generative models such as GANs".

Read the numbers carefully in both directions. 107 or so extractions out of 175 million generations is a rate that no ordinary studio workflow will stumble into. But "essentially never" is a different claim from "never", and the difference matters most exactly where duplication in the training set is highest, which is to say for famous images, stock photography and heavily reposted artwork. Those are the images a client is most likely to recognise.

What a material practice looks like when the material is statistical

A painter who works in oil owns their palette, their grounds, their varnish and their judgement about when a thing is finished. The paint manufacturer owns the paint. That division is obvious in a physical studio and keeps getting lost in a digital one, where it is tempting to think the interesting part is the generator.

It is not. The generator is the pigment supplier. Here is the division I now work to.

Three of those deserve a sentence more.

References before prompts. A reference set is a claim about what you want that does not depend on any particular model's vocabulary. Models change; your references do not. If you cannot assemble twenty images that show what you are after, the prompt will not rescue you.

Prompts as specifications, not spells. The useful prompt is the one another person in the studio could execute by hand, without the model. If a line in your prompt would not help a photographer or an illustrator, it is probably doing nothing except moving you a little along some axis nobody named.

Selection criteria written down in advance. This is the one that actually improves work, because generative tools make the bottleneck selection rather than production. Guidance settings trade diversity for fidelity, so a high-guidance batch of forty will look impressively consistent and tell you almost nothing. Write the criteria first, then generate, then apply them.

The grain metaphor holds up better than I expected, with one amendment. Wood's grain is visible on the surface of the plank. This grain is not. You infer it from behaviour: from what the model gives you when you ask for nothing in particular, from which of your requests cost you many attempts, from the direction in which the defaults drift. That inference is the craft. It is unglamorous, it is mostly record-keeping, and it is the difference between using a material and being used by one.

Sources

• Diederik P. Kingma and Max Welling, "Auto-Encoding Variational Bayes", arXiv:1312.6114, submitted 20 December 2013. arxiv.org/abs/1312.6114

• Alec Radford, Luke Metz and Soumith Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", arXiv:1511.06434, submitted 19 November 2015, ICLR 2016. arxiv.org/abs/1511.06434

• Tom White, "Sampling Generative Networks", arXiv:1609.04468, 2016 (v3, December 2016). arxiv.org/abs/1609.04468

• Ian Goodfellow, "NIPS 2016 Tutorial: Generative Adversarial Networks", arXiv:1701.00160, submitted 31 December 2016. arxiv.org/abs/1701.00160

• Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer, "High-Resolution Image Synthesis with Latent Diffusion Models", arXiv:2112.10752, submitted 20 December 2021, CVPR 2022. arxiv.org/abs/2112.10752

• Jonathan Ho and Tim Salimans, "Classifier-Free Diffusion Guidance", arXiv:2207.12598, 26 July 2022. arxiv.org/abs/2207.12598

• Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito and Eric Wallace, "Extracting Training Data from Diffusion Models", arXiv:2301.13188, 30 January 2023. arxiv.org/abs/2301.13188

• Federico Bianchi, Pratyusha Kalluri and co-authors, "Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale", arXiv:2211.03759, 2022, published in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. arxiv.org/abs/2211.03759

• CompVis, "Stable Diffusion v1-4 Model Card", Hugging Face. huggingface.co/CompVis/stable-diffusion-v1-4

• "Stable Diffusion v1-5 Model Card", Hugging Face, training procedure section. huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5

• LAION, "LAION-Aesthetics", laion.ai blog. laion.ai/blog/laion-aesthetics

Related reading

• What AI Taught Me About Describing My Own Work

• Taste Is the Scarce Thing Now

• Albers Ran Experiments, Not Poster Campaigns

• We Killed the Room

More from the Moc Mallen Studio journal