Let us recap what we have built so far, because the rest of the document depends on you having this picture clear in your head.
A modern image or video generation model is a latent diffusion transformer trained on a massive dataset of images or videos paired with text captions. The architecture has three main components. First, a variational autoencoder (VAE) that compresses pixels into a much smaller latent representation, and decompresses latents back into pixels. For video, this VAE is three-dimensional, compressing across both space and time. Second, a text encoder (or several text encoders, usually some combination of CLIP variants and a large language model like T5) that converts the text prompt into an embedding the diffusion model can use. Third, the diffusion transformer itself, which takes a noisy latent and the text embedding and predicts the noise, allowing the noise to be subtracted iteratively until a clean latent emerges, which is then decoded into an image or video.
The training is done with a flow-matched objective, predicting straight-line trajectories from noise to data, which produces better quality and faster sampling than the original DDPM-style stochastic objective. The architecture uses MM-DiT or similar dual-stream designs to handle text and image as equal modalities rather than treating text as auxiliary conditioning. For very large models, mixture-of-experts routing may be used to scale capacity without scaling compute. After the main model is trained, a distilled student model is often produced that runs in fewer sampling steps, trading a little quality for a lot of speed.
The training cost is millions to hundreds of millions of dollars. The inference cost is a few cents per image or a few dollars per video clip. The model is either released as open weights, allowing anyone to download and run it, or kept behind a closed API. Either way, end users typically interact with it through an aggregator platform like Krea or Flora that provides a unified interface across multiple models, or through ComfyUI for local/advanced use.
Every model in the company-by-company section of this document is some variation on this template. The variations are about which specific components they use, how big each piece is, what data they trained on, what novel architectural tricks they introduced, and what trade-offs they made between speed, quality, and cost. Once you have the template internalized, the variations are easy to follow.
If you only remember three things from the foundations, remember these. First: it is all denoising. Every image or video you have ever seen from one of these models was produced by a network whose only job is to predict noise, run iteratively from random static toward a clean image, guided by your prompt. Second: latent diffusion is the trick that makes it economically viable. Without VAE compression, none of this would be feasible at modern resolutions. Third: the field is dominated by a small number of well-funded labs whose access to compute is the binding constraint on what they can build, and whose decisions about open versus closed weights shape what the rest of us can do with their models.
With that in hand, we can turn to the actual models. The Image Models part covers image generation, organized by company. The Video Models part covers video generation, also by company. The specialized-models part covers the tools, editing, lipsync, 3D, and upscaling, that do not fit neatly into either category. The comparative analysis then shows how all of these models actually differ in practice, and how to choose between them. From there we close with a look at where the field is heading, a glossary, and a guide to further reading.