Contents

21 / 153

The Foundations

Pulling it all together: the modern stack

Chapter 20

3 min read

Reviewed v78 · August 2026

Fig.timeline
2015Diffusion, from thermodynamics2020DDPM: it actually works2021Latent diffusion2022SD, Midjourney, DALL-E 22023SDXL, ComfyUI everywhere2024DiT era, Sora demo, FLUX2025The video year2026Open weights close the gap
A decade of generative visual AI on a single line. Color-coded by event type: yellow for watershed moments, red for foundational papers, blue for major model releases, green for tools and infrastructure.

Let us recap what we have built so far, because the rest of the document depends on you having this picture clear in your head.

A modern image or video generation model is a latent diffusion transformer trained on a massive dataset of images or videos paired with text captions. The architecture has three main components. First, a variational autoencoder (VAE) that compresses pixels into a much smaller latent representation, and decompresses latents back into pixels. For video, this VAE is three-dimensional, compressing across both space and time. Second, a text encoder (or several text encoders, usually some combination of CLIP variants and a large language model like T5) that converts the text prompt into an embedding the diffusion model can use. Third, the diffusion transformer itself, which takes a noisy latent and the text embedding and predicts the noise, allowing the noise to be subtracted iteratively until a clean latent emerges, which is then decoded into an image or video.

Fig.diagram
promptENCODERmeaning ceilingDiTthe denoiser(gets the attention)VAEfidelity ceilingpixelsCEILINGCEILINGThe glamorous part is rarely the binding constraint.A bigger denoiser cannot recover meaning the encoder dropped or detail the VAE cannot rebuild.Video adds a third ceiling in the 3D VAE, and audio a fourth in the codec.
The glamorous denoiser sits between two humbler ceilings, the text encoder and the VAE, that cap what it can do.

The training is done with a flow-matched objective, predicting straight-line trajectories from noise to data, which produces better quality and faster sampling than the original DDPM-style stochastic objective. The architecture uses MM-DiT or similar dual-stream designs to handle text and image as equal modalities rather than treating text as auxiliary conditioning. For very large models, mixture-of-experts routing may be used to scale capacity without scaling compute. After the main model is trained, a distilled student model is often produced that runs in fewer sampling steps, trading a little quality for a lot of speed.

The training cost is millions to hundreds of millions of dollars. The inference cost is a few cents per image or a few dollars per video clip. The model is either released as open weights, allowing anyone to download and run it, or kept behind a closed API. Either way, end users typically interact with it through an aggregator platform like Krea or Flora that provides a unified interface across multiple models, or through ComfyUI for local/advanced use.

Every model in the company-by-company section of this document is some variation on this template. The variations are about which specific components they use, how big each piece is, what data they trained on, what novel architectural tricks they introduced, and what trade-offs they made between speed, quality, and cost. Once you have the template internalized, the variations are easy to follow.

If you only remember three things from the foundations, remember these. First: it is all denoising. Every image or video you have ever seen from one of these models was produced by a network whose only job is to predict noise, run iteratively from random static toward a clean image, guided by your prompt. Second: latent diffusion is the trick that makes it economically viable. Without VAE compression, none of this would be feasible at modern resolutions. Third: the field is dominated by a small number of well-funded labs whose access to compute is the binding constraint on what they can build, and whose decisions about open versus closed weights shape what the rest of us can do with their models.

With that in hand, we can turn to the actual models. The Image Models part covers image generation, organized by company. The Video Models part covers video generation, also by company. The specialized-models part covers the tools, editing, lipsync, 3D, and upscaling, that do not fit neatly into either category. The comparative analysis then shows how all of these models actually differ in practice, and how to choose between them. From there we close with a look at where the field is heading, a glossary, and a guide to further reading.

Check your understanding

pass: 5 of 7

Answer at least 5 of 7 correctly to unlock the next chapter.

  1. 1. What is the job of the variational autoencoder (VAE) in the modern stack?

  2. 2. How does the VAE differ for video models compared with image models?

  3. 3. Which combination typically serves as the text encoder in these models?

  4. 4. What does the flow-matched training objective predict?

  5. 5. Why is latent diffusion singled out as the trick that makes the field economically viable?

  6. 6. What is described as the binding constraint on what the dominant labs can build?

  7. 7. What is the purpose of producing a distilled student model after the main model is trained?

The weekly briefing

Get the week's moves in your inbox.

A short, sourced digest of what actually moved across generative AI, every week. Free.

Free. One email a week, no spam, unsubscribe anytime. Prefer a reader? RSS.