The workflows opening up right now

The single biggest jump in directorial control this year is first-and-last-frameFirst-and-last-frameA video workflow where you fix the opening and closing frame and the model generates only the motion between them. generation. Most video models now let you fix a starting frame and an ending frame and generate only the motion between them, which turns a slot machine into a tween. You generate a clean still for each end with an image model, where you have full control over composition, and hand the video model the one job it is good at, solving the movement from A to B. Kling, Luma, Runway, and Pika all support it, and it is how most shots that have to hit a specific composition actually get made now, rather than prompting and praying the model lands the frame you needed.

The oldest complaint about generative video is that the character's face drifts between clips. The workflow that answers it is a stack. Build a character sheetCharacter sheetA set of reference images of the same character from several angles and expressions, used to lock identity across generations., the same person from several angles and expressions, then hold that identity steady with a small trained LoRALoRA (Low-Rank Adaptation)A fine-tuning technique that adapts a base model to a specific style or concept using a small additional file. The standard way to customize open-source models. or by feeding a reference through an adapter like IP-AdapterIP-Adapter (Image Prompt Adapter)A method for conditioning generation on a reference image, so a subject or style from a picture carries into new outputs. or InstantIDInstantIDA technique for preserving a specific person's identity across generations from a single reference photo, without training a LoRA., and keep the seed fixed across generations. The same recipe locks a product for commercial work, so the bottle or the shoe looks identical in every shot. It is fiddly and it is the unglamorous heart of professional AI production, the difference between a sequence that reads as one character and a sequence that reads as a glitch.

Mask-based editing is giving way to instruction-based editingInstruction-based editingEditing an image by giving a text instruction, such as change the background to a beach, and letting the model decide what to change and what to leave alone, without a hand-drawn mask., where you hand the model an image and a sentence, remove the sign, make it dusk, add a jacket, and it performs the edit without you painting a region. FLUX Kontext, Nano Banana, Qwen-Image-Edit, and Runway's Aleph for video all work this way. The trending move is to chain them: edit, judge, edit again, each instruction building on the last result rather than starting over, until the frame is right. It turns editing from a technical operation into a conversation, and it is far faster than regenerating from the promptPromptThe text description you provide to a model to specify what you want it to generate. every time you want a small change.

No single model wins every step, so the workflow that has taken over professional pipelines is to route. Generate in the model with the best base aesthetic, edit in the one with the best instruction-following, upscale in a dedicated restorer, add audio in another, and treat each as a specialist rather than a suite. Runway shipped a literal Media Router for exactly this, and aggregators like fal and Krea exist to make the hand-offs cheap and keep your assets in one place. The skill this rewards is no longer mastering a single tool, it is knowing which tool to reach for at each stage, which is the same judgment a director brings to a crew.

The frontier models increasingly accept many conditioningConditioningAny extra input that steers generation beyond the prompt, such as a depth map, pose, edge map, or reference image. inputs at once: a subject image, a style reference, a pose or depth mapDepth mapA grayscale image encoding how far each pixel is from the camera, used as conditioning to control 3D structure., an audio track for timing. Seedance advertises taking dozens of references in a single generation. The workflow that follows is to stop describing what you want in words and start showing it, feeding the model a small dossier of references so the promptPromptThe text description you provide to a model to specify what you want it to generate. only has to carry intent, not a paragraph of description the model will half-ignore anyway. A good reference is worth more than a good sentence, and assembling the reference set is becoming a real part of the craft.

Real-time tools like Krea's canvas and Decart's streaming models generate as fast as you move, which changes the loop from write, wait, judge into steer and watch. Creators use this less for final frames than for exploration, finding a direction by moving through the space of images live, then locking a seed and rendering it properly at full quality. It is the closest the medium has come to sketching, and it is quietly changing how people search for an idea, because when iteration is instant you try things you would never have queued up one slow generation at a time.

The last shift is agentic. A ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. graph or a coded pipeline runs without a person at each step, generating variations overnight, batching an entire product catalog, or being called by other software through an APIAPI (Application Programming Interface)The programmatic endpoint you call to run a hosted model from your own code instead of a web interface.. The power user's graph, as the ComfyUI chapter put it, becomes the agent's API. This is how generation stops being a thing you sit and do and becomes a thing your tools do on your behalf, and it is the bridge between the hands-on craft of the rest of this chapter and the industrial, at-scale production the operator's playbook is about.