The single most important piece of infrastructure: ComfyUI

In January 2023, five months after the public release of Stable Diffusion, an anonymous developer who goes only by the handle 'comfyanonymous' published the first version of ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. on GitHub. The person behind the handle has never publicly identified themselves. What is known is that they were working alone, that they built ComfyUI in part to learn how Stable Diffusion worked under the hood, and that they made an unusual design choice: instead of building a traditional web interface with text fields and buttons, they built a nodeNodeA single block in a visual pipeline like ComfyUI that does one operation and wires to others to form a workflow.-based graph editor.

If you have ever used Blender's compositingCompositingCombining multiple image or video layers into one final frame, a core post-production step. nodes, or Houdini, or Unreal Engine's Blueprints, or Touch Designer, you know what node-based interfaces look like. The screen is a canvas. On the canvas sit boxes (the nodes), each of which represents a single operation. Boxes have inputs on the left and outputs on the right. You connect outputs to inputs by dragging lines between them. The whole graph forms a directed flow of data from the source nodes (loaders) through processing nodes to output nodes (savers and previewers). When you press 'queue promptPromptThe text description you provide to a model to specify what you want it to generate.,' the graph executes from left to right.

This kind of interface is unusual for a deep learning tool. Most other Stable Diffusion interfaces, AUTOMATIC1111AUTOMATIC1111An older web-based User Interface (UI) for Stable Diffusion, originally the dominant interface in 2022-2023, still widely used by casual users.'s web User Interface (UI) was the dominant one in 2023, looked like a normal form: a text box for the prompt, a slider for the steps, a dropdown for the samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs., a button to generate. ComfyUI's node graphNode graphA visual pipeline where each box performs one operation and wires to others, the interface ComfyUI made standard. was harder to learn, more verbose for simple tasks, and visually intimidating. It seemed like a worse interface.

It was not a worse interface. It was a much better one, for reasons that took a year for the community to fully appreciate.

The first reason is transparency. In a form-based UI, the steps of the diffusion process are hidden inside the application code. The user does not see that there is a model loader, a text encoderText encoderA neural network that converts text prompts into numerical embeddings that the diffusion model can use. Modern models use combinations including CLIP and T5., a noise generator, a samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs., a VAEVAE (Variational Autoencoder)A neural network that learns to compress images into a small numerical representation (a latent) and decompress them back. Used in latent diffusion models to make the generation process tractable. decoder, and a save step, they just see a 'generate' button. In ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org., every one of those steps is a visible nodeNodeA single block in a visual pipeline like ComfyUI that does one operation and wires to others to form a workflow. on the canvas. You can see what is happening, in what order, and you can intervene at any point. You learn how the model actually works just by looking at the graph.

The second reason is composability. Once every step is a node, you can rearrange them. You can insert custom processing in the middle. You can run the same promptPromptThe text description you provide to a model to specify what you want it to generate. through multiple samplers and compare. You can chain models together, generate with one model, refine with another, upscale with a third. You can do things that the original interface designers never anticipated, because you are not constrained by the choices they made about which features to expose. ComfyUI is closer to a programming environment than a tool, and that gives it a much higher ceiling.

The third reason is reproducibility. ComfyUI workflows can be saved as JSON files, but more importantly, they can be embedded in the metadata of the PNG files that the workflow produces. If somebody on Twitter posts an image they generated in ComfyUI, you can drag that image into your own ComfyUI canvas, and the entire graph that produced it appears on your screen. You can see exactly which model was used, which prompts, which samplers, which custom nodes, in what order. This reproducibility is unique to ComfyUI in the open-source generative imagery space, and it has had a profound effect on how the community shares techniques. Workflows are the unit of knowledge transfer.

The fourth reason, and this is the one that turned out to matter most for the trajectory of the field, is extensibility. ComfyUI supports custom nodes. Anyone can write a Python script that adds new nodes to the canvas, and other users can install those nodes through a package manager called ComfyUI-Manager. Within a year of launch, there were thousands of community-developed custom nodes for everything from advanced samplers to face restorationFace restorationA finishing pass that rebuilds facial detail softened or damaged earlier in a pipeline, for example after an aggressive upscale. to video frame interpolationFrame interpolationGenerating in-between frames to raise a video's frame rate or smooth its motion. to Low-RankRank (LoRA rank)The inner dimension of a LoRA's two small matrices, which sets how much capacity it has to learn. Higher rank captures more complex behavior but risks overfitting and produces larger files. Adaptation (LoRALoRA (Low-Rank Adaptation)A fine-tuning technique that adapts a base model to a specific style or concept using a small additional file. The standard way to customize open-source models.) training to depth mapDepth mapA grayscale image encoding how far each pixel is from the camera, used as conditioning to control 3D structure. generation. Every time a new technique was published in a research paper, somebody would wrap it in a ComfyUI node within days, often within hours.

If your first look at a ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. graph felt like being dropped into an airplane cockpit, that reaction is correct, and it is worth sitting with, because the complexity you are seeing is not ComfyUI being needlessly technical. It is the actual machinery of image generation, uncovered. Every other tool you have used hides that machinery behind a single button, the way a microwave hides thermodynamics behind the word Start. ComfyUI takes the cover off. The graph looks complicated because the thing it is showing you genuinely is complicated, and for the first time you are looking straight at it instead of at a friendly facade.

The reason there are so many boxes is that generating an image really does take that many distinct steps, and each box is one of them. A model has to be loaded. Your words have to be turned into numbers the model understands. A canvas of pure noise has to be prepared. That noise has to be denoised step by step under the guidance of your promptPromptThe text description you provide to a model to specify what you want it to generate.. And the result has to be decoded from the model's compressed inner space back into pixels you can actually see. That is not an arbitrary pipeline someone chose to make painful. It is close to the shortest honest description of what diffusion does. A one-button app runs the exact same steps, it just performs them offstage where you cannot reach them.

So the real distinction between the two kinds of tools is not simple versus complicated, it is sealed versus open. A one-button app is a microwave meal: fast, foolproof, and impossible to change the moment you want something the menu does not offer. ComfyUI is a professional kitchen: every station is exposed, which is overwhelming on the first day and is precisely why a cook there can make something the microwave never could. The intimidating dials are not obstacles standing between you and the image. They are the controls that were always there, finally within your reach.

And here is the part that makes the whole thing learnable. That wall of nodes is not hundreds of separate things to memorize. Almost every workflow, however elaborate it looks, is built from the same short list: load the model, encode the prompt, start from noise, denoise with the samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs., decode with the VAEVAE (Variational Autoencoder)A neural network that learns to compress images into a small numerical representation (a latent) and decompress them back. Used in latent diffusion models to make the generation process tractable., save the result. A complicated graph is usually just that same skeleton with a few extra nodes spliced into the middle to do one specific job. Once you can recognize the recurring skeleton, the complexity stops being a wall and starts being a language, and every new workflow you open becomes a sentence you can already mostly read.

By 2024, ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. had become the dominant power-user interface for Stable Diffusion. AUTOMATIC1111AUTOMATIC1111An older web-based User Interface (UI) for Stable Diffusion, originally the dominant interface in 2022-2023, still widely used by casual users.'s web UI still had more casual users, but anybody doing serious work, anybody chaining multiple models, doing complex pipelines, using ControlNetControlNetA conditioning technique that lets you provide structural guidance (depth, edges, pose) to a diffusion model to control spatial layout., running custom training, was using ComfyUI. Stability AI itself started using it internally to test new model releases. When Stability AI released Stable Diffusion 3 in 2024, the official launch came with a ComfyUI workflow as the reference implementation. When Black Forest Labs released FLUX in August 2024, the same was true. New models now ship with ComfyUI workflows because that is how the community expects to consume them.

The interesting thing is that this dominance has spread beyond just Stable Diffusion. ComfyUI now supports workflows for image models from many vendors, video models including Wan, Hunyuan, LTX, and others, audio models, 3D generation models, and more. Custom nodes exist for accessing the APIs of closed-source models from Runway, Luma, Kling, and OpenAI, so you can mix open-source local models with closed-source cloud models in the same graph. In effect, ComfyUI has become the universal substrate on top of which the entire field operates, the same way Linux became the universal substrate underneath modern computing.

In June 2024, comfyanonymous left Stability AI (where they had been hired during 2023) and, with several core contributors, formed an organization called Comfy Org to put the project on more sustainable footing. Comfy Org has since received support from companies including NVIDIA, which announced ComfyUI integration into its RTX Remix modding software in July 2024, and joined the Open Model Initiative under the Linux Foundation. As of late 2025, the ComfyUI GitHub repository has more than 89,000 stars, putting it among the most-starred AI projects in history. The pseudonymous solo developer who started the project on a whim two and a half years ago is now the technical lead of an organization that effectively stewards the ecosystem layer beneath every major image and video model.

Why this matters for understanding the field. When you use Krea or Flora, you are using a curated, polished interface that hides the underlying complexity. When you use ComfyUI, you see all of it. The reason these two perspectives matter is that almost every clever technique and creative workflow in the generative imagery world is invented in ComfyUI first and then, sometimes, finds its way into the consumer tools. If you want to know what Krea or Flora will support next year, look at what is happening in the ComfyUI custom nodes scene this month. The lag is real but small.

Just so you have a concrete picture, let us build one specific image and follow it through every nodeNodeA single block in a visual pipeline like ComfyUI that does one operation and wires to others to form a workflow.. Say you want a cinematic portrait of an astronaut standing in a golden wheat field at sunset, and you are going to make it with an SDXL checkpointCheckpointA saved set of model weights. The .safetensors or .ckpt files you download to run a model., one of the popular community ones like Juggernaut XL. Here is what that basic Stable Diffusion text-to-image workflow looks like as a graph. At the far left is a 'Load Checkpoint' node, and this is where you point at the Juggernaut file sitting on your disk. When it loads, it hands out three separate outputs, each wired off to whichever node downstream needs it: the diffusion modelDiffusion modelA generative model that learns to produce images by training a neural network to reverse a noise-adding process. Generates by starting from random noise and iteratively denoising to produce a clean image. itself (called the 'unet' or 'transformerTransformerThe neural network architecture built on attention that underpins modern language and, increasingly, image and video models. It scales well and handles long-range relationships better than convolutional networks.' depending on the architecture), the text encoderText encoderA neural network that converts text prompts into numerical embeddings that the diffusion model can use. Modern models use combinations including CLIP and T5. (the CLIPCLIPA text encoder developed by OpenAI in 2021 that learns to align text and images in a shared embedding space. Foundation of most text-to-image models from 2022 onward. model) that will read your words, and the VAEVAE (Variational Autoencoder)A neural network that learns to compress images into a small numerical representation (a latent) and decompress them back. Used in latent diffusion models to make the generation process tractable. that will turn the final result back into pixels.

Next come two 'CLIP Text Encode' nodes, and this is where your astronaut actually gets described. The positive one holds what you want, so you type something like 'an astronaut standing in a golden wheat field at sunset, cinematic, 85mm lens, shallow depth of field.' The negative one holds what you want kept out, the usual suspects like 'blurry, low resolutionResolutionThe pixel dimensions of an image or video, for example 1024x1024. Higher resolution shows more detail but costs more memory and time., extra fingers, text, watermark.' Both take the CLIP text encoder as input and turn your words into conditioningConditioningAny extra input that steers generation beyond the prompt, such as a depth map, pose, edge map, or reference image. embeddings, the numeric form the model can actually steer toward. Alongside them an 'Empty Latent Image' node creates the blank canvas your astronaut will emerge from, and you set it to 1024 by 1024, SDXL's native resolution.

Then comes the 'KSamplerKSamplerThe ComfyUI node where denoising actually happens, taking the model, prompts, and a latent and running the sampling loop under its seed, steps, CFG, sampler, and scheduler settings.' node, the heart of the workflow and the place your astronaut is actually pulled out of the noise. It takes the model, the positive and negative conditionings, that empty latent, and the handful of parameters that decide how the sampling runs: the number of steps, the guidance scaleGuidance scale (Classifier-Free Guidance (CFG) scale)A parameter that controls how strongly the model adheres to the prompt versus producing diverse outputs. Higher values mean stricter prompt following. Typical range 5-10., the sampling algorithm, and a random seed. For this SDXL portrait you might set 25 steps, a guidance scale of 7, the DPM++ 2M samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs. with the Karras schedulerSchedulerThe algorithm that decides the sequence of denoising steps and how each is applied. Often called the sampler. Different schedulers trade speed against quality., and, importantly, a fixed seed, because that seed is what lets you reproduce this exact astronaut later instead of rolling a different one. It runs the diffusion loop and produces a denoised latent.

Finally a 'VAE Decode' node takes that denoised latent and converts it back into a real image, your actual astronaut in the wheat field, and a 'Save Image' node writes the PNG to disk. Press queue, the graph executes left to right, and the picture appears. That is text-to-image, start to finish, for one concrete image.

Once you have this, every variation is a small modification to the same graph, and it helps to keep the astronaut in mind as you make them. Want image-to-image, so the composition follows a real photo of a field you shot? Replace the 'Empty Latent' with a 'Load Image' node connected through a 'VAE Encode.' Want the astronaut to consistently be a specific character or painted style across many images? Insert a 'Load LoRALoRA (Low-Rank Adaptation)A fine-tuning technique that adapts a base model to a specific style or concept using a small additional file. The standard way to customize open-source models.' node between the model loader and the sampler. Want a larger, sharper version for print? Add an upscalingUpscalingIncreasing an image or video's resolution after generation, often with a model that invents plausible fine detail rather than simply stretching pixels. node after the VAE decode. Want the astronaut in an exact pose you sketched? Add a ControlNetControlNetA conditioning technique that lets you provide structural guidance (depth, edges, pose) to a diffusion model to control spatial layout. loader and route its output into the sampler. Want the astronaut to move instead of stand still? Replace the SDXL loader with a Wan or Hunyuan video loader, swap the KSampler for a video sampler, and replace the save image with a save video. Same skeleton, one piece swapped at a time. The graph composes.

The KSamplerKSamplerThe ComfyUI node where denoising actually happens, taking the model, prompts, and a latent and running the sampling loop under its seed, steps, CFG, sampler, and scheduler settings. nodeNodeA single block in a visual pipeline like ComfyUI that does one operation and wires to others to form a workflow. is where the denoisingDenoisingThe core loop of diffusion. Starting from pure noise, the model repeatedly predicts and subtracts a little noise until a coherent image emerges. actually happens, and it exposes the handful of settings that most determine what your image looks like: the number of steps, the CFGCFG (Classifier-Free Guidance)A technique that pushes a diffusion model to follow the prompt more strongly by contrasting its prediction with and without the prompt. Higher values obey the text more but can look oversaturated or 'fried'. scale, the samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs., and the schedulerSchedulerThe algorithm that decides the sequence of denoising steps and how each is applied. Often called the sampler. Different schedulers trade speed against quality.. Every other node in the graph exists to feed this one. Understanding these four fields is the difference between operating ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. and merely clicking through a workflow someone else built.

Start with steps. A step is one iteration of denoising. Recall from the diffusion chapter that the model does not paint an image in a single stroke. It starts from pure noise and removes a little of it at a time, re-evaluating after each pass. The step count is simply how many of those passes it runs. At step one the latent is almost entirely noise. By the final step it should be a clean image.

The intuition most people start with is 'more steps, more quality,' and for the first stretch that is true. Going from 10 steps to 20 is usually a visible jump: edges firm up, small objects stop being mush, text gets closer to legible. But the returns flatten fast. The difference between 30 and 50 steps is, for most modern models, almost invisible in a side by side, and past a point extra steps can even slightly degrade an image by over-sharpening or introducing artifacts. You pay linearly in time for a benefit that decays.

The reason few-step models exist at all connects back to distillationDistillationA technique where a smaller, faster 'student' model is trained to mimic the outputs of a larger 'teacher' model in fewer sampling steps. Used to produce fast variants of slow models (FLUX schnell, SDXL Turbo, etc.).. A base model needs many steps because each step can only safely remove a small amount of noise before it starts guessing wrong. Distillation trains a student model to take much larger, more confident steps. This is why you cannot simply run a normal model at 4 steps and expect a distilled result. The normal model was never taught to make jumps that big, and it will hand you a blurry, unresolved mess.

CFGCFG (Classifier-Free Guidance)A technique that pushes a diffusion model to follow the prompt more strongly by contrasting its prediction with and without the prompt. Higher values obey the text more but can look oversaturated or 'fried'. stands for classifier-free guidance, and the CFG scale is the single most misunderstood setting in the entire stack. It controls how hard the model is pushed to follow your promptPromptThe text description you provide to a model to specify what you want it to generate. versus how much freedom it has to produce a natural-looking image. It is, in effect, a leash.

The mechanism is worth understanding because it explains the tradeoff. At each denoisingDenoisingThe core loop of diffusion. Starting from pure noise, the model repeatedly predicts and subtracts a little noise until a coherent image emerges. step the model actually makes two predictions: one conditioned on your prompt, and one conditioned on nothing at all, an empty prompt. The difference between those two predictions is the direction that points 'toward your prompt.' The CFG scale is how far the samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs. travels in that direction. A CFG of 1 ignores the prompt-direction entirely. A CFG of 7 pushes several times harder toward the prompt than the raw conditioned prediction alone would.

So why not set it to 20 and get perfect adherence? Because the prompt-direction is only an approximation, and the further you extrapolate along an approximation, the more it breaks. Crank CFG too high and images become over-saturated, contrast blows out, skin turns plasticky, and fine detail fractures into a fried, crunchy texture. Too low and the image drifts off-prompt, ignoring elements you explicitly asked for, looking washed out and directionless. The craft is finding the point where the model is listening to you but not being tortured.

This is the most common mistake when people move a FLUX or Turbo workflow onto SDXL settings, or the reverse. The connection back to distillationDistillationA technique where a smaller, faster 'student' model is trained to mimic the outputs of a larger 'teacher' model in fewer sampling steps. Used to produce fast variants of slow models (FLUX schnell, SDXL Turbo, etc.). is exact. Classifier-free guidance normally requires running the model twice per step, once conditioned and once unconditioned, which doubles the cost. Guidance distillation trains that behavior directly into the weightsWeightsThe learned numbers inside a model that encode everything it knows. 'Open weights' means these are downloadable. so the model only has to run once and still behaves as if guided. That is why the fast models both need fewer steps and want a CFG near 1: the guidance is already inside them.

The samplerSamplerThe algorithm used to perform the denoising steps. Different samplers (Euler, DPM++, etc.) have different speed-quality trade-offs. is the algorithm that decides how to move from one step to the next, and the schedulerSchedulerThe algorithm that decides the sequence of denoising steps and how each is applied. Often called the sampler. Different schedulers trade speed against quality. decides how much noise to remove at each step, the so-called noise scheduleNoise scheduleThe recipe that decides how much noise is added at each step of the forward process and removed on the way back. It strongly affects quality and speed.. ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. exposes dozens of samplers with intimidating names: Euler, Euler a, Heun, DPM++ 2M, DPM++ 2M SDE, DPM++ 3M SDE, UniPC, DDIMDDIM (Denoising Diffusion Implicit Model)A sampling method that generates an image in far fewer steps than DDPM by following a deterministic, non-random path through the denoising process., LMS, and more. The names look like a wall, but they cluster into a few families, and the honest truth is that the sampler matters far less than steps or CFGCFG (Classifier-Free Guidance)A technique that pushes a diffusion model to follow the prompt more strongly by contrasting its prediction with and without the prompt. Higher values obey the text more but can look oversaturated or 'fried'..

The first useful split is deterministic versus ancestral. A plain sampler like Euler or DPM++ 2M is deterministic: same seed, same settings, same image every time, and the image 'converges,' meaning it settles down and stops changing as you add steps. An ancestral samplerAncestral samplerA sampler that injects a little fresh noise at each step, adding variation but making results less reproducible for a given seed., marked with a trailing 'a' as in Euler a, or 'SDE' as in DPM++ 2M SDE, injects a little fresh noise at every step. That makes it non-converging: it keeps changing as steps increase, and it often adds pleasing texture and variation at the cost of reproducibility. For LoRALoRA (Low-Rank Adaptation)A fine-tuning technique that adapts a base model to a specific style or concept using a small additional file. The standard way to customize open-source models. work and anything you need to reproduce exactly, prefer a deterministic sampler.

The second split is order, which is really a speed versus accuracy tradeoff. Higher-order samplers such as Heun and the DPM++ 2M and 3M family do more math per step to follow the denoisingDenoisingThe core loop of diffusion. Starting from pure noise, the model repeatedly predicts and subtracts a little noise until a coherent image emerges. path more accurately, so they reach a clean result in fewer steps but each step is slower. Euler is first-order: cheapest per step, needs a few more of them. In practice this mostly washes out.

The manual node graphNode graphA visual pipeline where each box performs one operation and wires to others, the interface ComfyUI made standard. is not the end state. In 2026 the same workflows started being driven by agents rather than by hand.

Comfy MCP, launched by Comfy Org on June 30, 2026, is the official bridge. It exposes ComfyUIComfyUIThe dominant node-based interface for running diffusion models locally. Started by 'comfyanonymous' in January 2023, now stewarded by Comfy Org. through the Model Context Protocol so an AI agent, Claude among them, can search for models and nodes, assemble and run workflows, and generate image, video, 3D, or audio, executing on Comfy Cloud GPUs. The agent does the wiring you used to do by dragging.

This matters for the same reason nodes mattered in the first place. Once a workflow is a program an agent can read and write, generation becomes something you can orchestrate in language and fold into a larger automated pipeline. Community tools such as ComfyUI-Copilot push the same idea inside the canvas, suggesting and building graphs for you.

The mental model to carry forward: ComfyUI made every step of generation explicit and composable, and agentic ComfyUI makes those explicit steps something a machine can assemble on your behalf. The power user's graph becomes the agent's APIAPI (Application Programming Interface)The programmatic endpoint you call to run a hosted model from your own code instead of a web interface..