Realtime and interactive, a different mode

Almost everything in this book assumes batch generation: you submit a promptPromptThe text description you provide to a model to specify what you want it to generate., you wait, you get a file. A different mode is arriving that breaks that assumption, and it changes what the tools are for.

The first form is realtime generationRealtime generationGeneration fast enough to respond as you type, paint, or move, turning the model from a batch render job into a live instrument.. Products like Krea's realtime canvas and Decart's streaming models generate as fast as you can move, so generation becomes a live instrument rather than a render job. The technique underneath is aggressive distillationDistillationA technique where a smaller, faster 'student' model is trained to mimic the outputs of a larger 'teacher' model in fewer sampling steps. Used to produce fast variants of slow models (FLUX schnell, SDXL Turbo, etc.)., the few-step models covered earlier, pushed until the frame rate crosses into interactivity. The use cases are different in kind: live visuals, VJ-ing, on-the-fly avatars, and design that responds as you sketch.

The second form is world models, covered in the part on where the field is headed. Instead of a fixed clipCLIPA text encoder developed by OpenAI in 2021 that learns to align text and images in a shared embedding space. Foundation of most text-to-image models from 2022 onward. they generate an environment you move through, predicting the next frame from your input. That is not a video you watch, it is a space you are inside.

Both share a property that matters for the stack: they collapse the pipeline. When generation is instant and interactive, the sharp line between production and consumption softens, and some of the post-production and even distribution steps fold into the moment of generation itself. A live generated world is not edited and uploaded, it is experienced as it is made.

The operator takeaway is to watch this mode as a separate track, not a faster version of the old one. Batch tools and realtime tools are built differently, priced differently, and used for different things. The businesses that win in realtime will not be the ones that made the best batch clip a little faster, they will be the ones that understood generation as a live medium.