Contents

03 / 153

The Foundations

Orientation: the one-paragraph version, and the one-page version

Chapter 02

3 min read

Reviewed v78 · August 2026

Before we go anywhere, let me give you the entire field in one paragraph and one page, so you have a north star to come back to whenever something later feels overwhelming.

The one-paragraph version

A modern image AI is a giant pattern-matcher that has looked at hundreds of millions of pictures with captions. To make a new image from your prompt, it does not draw, it does not paint, and it does not search. It starts with a screen of pure television-static noise, looks at the noise, looks at your prompt, and asks itself a very specific question: 'if this noisy mess were a slightly-less-noisy version of a real image that matches the prompt, what would the slightly-less-noisy version look like?' It produces a guess, takes a small step toward that guess, and then asks the same question again. After 20 to 50 of those steps, the noise is gone and an image that matches your prompt is sitting in its place. That is the entire trick. Everything else, every model, every refinement, every architectural choice, is a variation on how to ask that question and how to take that step.

The one-page version

The one-paragraph version is true but it leaves a lot of natural questions hanging. Where does the 'pattern-matcher' come from? How does it learn what the prompt means? How does it know what 'slightly less noisy' looks like? Why does this work better than just trying to draw the image in one shot? Here is the slightly longer story, in plain English.

The pattern-matcher is a neural network, which is just a mathematical function with billions of adjustable knobs (called parameters) inside it. To 'train' the network, researchers take a real image, add a tiny bit of random noise to it, and ask the network: 'guess what noise I added.' At first the network guesses badly. Each time it guesses, the training procedure nudges the knobs slightly so that next time it will guess a little better. Repeat this hundreds of millions of times across hundreds of millions of images, with varying amounts of noise added each time, and the network slowly becomes a noise-spotting expert. It can look at any noisy image and accurately predict what noise was added to it, which means it can also subtract that noise back out and recover a cleaner image.

Once the network is trained, you flip the procedure on its head to generate an image. Start with pure noise (no real image underneath at all). Ask the network: 'pretend there was a real image underneath this, what noise would have to be added to produce this exact mess?' The network gives its best guess. Subtract that guess from the noise. You now have a slightly-less-noisy image. Ask again. Subtract again. Repeat. After enough iterations, an image emerges from what started as pure static. This sounds like it shouldn't work, but it does work, beautifully, and the rest of the foundations is about why.

The 'how does it know what the prompt means' part is a separate trick: a different neural network (called a text encoder) reads your prompt and converts it into a numerical fingerprint that captures its meaning. This fingerprint gets fed alongside the noisy image at every denoising step, so the network's guesses are biased toward 'noise consistent with this meaning.' That is how 'a red barn at sunset' produces a red barn at sunset and not a blue car at noon.

That is the entire field in one page. If you understand just the two callouts on this page, you already have the conceptual framework for everything that follows. The rest of the foundations fills in the details: who figured this out, when, and why their specific choices matter. The rest of the document is about which companies built which versions and how to actually use them. But the core idea, the thing that makes all of it possible, is what you just read.

Check your understanding

pass: 5 of 7

Answer at least 5 of 7 correctly to unlock the next chapter.

  1. 1. When a modern image AI begins generating from your prompt, what is actually on the screen at the very start?

  2. 2. Roughly how many denoising steps does it take to turn that starting noise into a finished image?

  3. 3. Why is the denoising approach described as a much easier problem than drawing an image?

  4. 4. How is the noise-predicting network actually trained?

  5. 5. What does the text encoder produce from your prompt?

  6. 6. According to the orientation, how many networks are really doing the work, and what do they do?

  7. 7. How does the prompt actually bias the output toward 'a red barn at sunset' rather than 'a blue car at noon'?

The weekly briefing

Get the week's moves in your inbox.

A short, sourced digest of what actually moved across generative AI, every week. Free.

Free. One email a week, no spam, unsubscribe anytime. Prefer a reader? RSS.