Contents

36 / 153

The LoRA Deep Dive

The training dataset is where LoRAs are made or broken

Chapter 35

6 min read

Reviewed v78 · August 2026

Almost every failed LoRA training run can be traced to a problem with the dataset. The hyperparameters matter, but the dataset matters more. The community's hard-won consensus is that twenty-five well-curated images will outperform seventy-five inconsistent ones. The job of the dataset is to teach the model exactly the thing you want it to learn, with enough variation that it generalizes, and exactly nothing else. Curating a good dataset is more like writing a brief than like collecting samples, every image needs to contribute something specific.

01

Image count and variation

For a single subject (a person, a product, a specific character), 15 to 30 images is a sweet spot. Fewer than 15 makes it hard for the model to disentangle the subject from its surroundings, it doesn't have enough examples of the same thing in different contexts to figure out what stays constant. More than 40 starts to introduce more noise than signal, especially if your images are not perfectly consistent. For a style LoRA (an art style, a photographic look, a particular aesthetic) the right count is usually 30 to 60, because styles are higher-dimensional than subjects and need more examples to characterize.

Variation matters more than quantity. For a person LoRA, you want photos from multiple angles (front, three-quarter, profile, even some from behind), multiple lighting conditions (soft daylight, hard sun, indoor, dramatic), multiple backgrounds (so the model learns that the background is incidental), multiple expressions, and a mix of distances (close-up faces, medium shots, full body). For a style LoRA, you want a wide range of subject matter rendered in the same style, so the model learns that the style is what is consistent and the subject is what varies. The single most common mistake in LoRA training is uploading thirty selfies all taken in the same room with the same expression, the model learns to generate that exact pose in that exact room and treats the face as incidental detail.

02

Resolution and quality

Train at 1024x1024 for FLUX and SDXL. Train at 512x512 only for legacy SD 1.5 work. The resolution of your training images sets a hard ceiling on the resolution at which the LoRA will produce good outputs, a LoRA trained on 512px images will look soft and lossy when used at 1024px, even if the underlying base model handles 1024px fine. Use lossless formats (PNG, not JPEG) when possible, especially for fine details like hair, fabric texture, and subtle skin tone variations. JPEG compression artifacts are subtle to the human eye but the model will learn them and start producing them as a feature.

Bad source images cannot be rescued by training. Out-of-focus images teach the model to produce blurry outputs. Low-light images with heavy noise teach the model to produce noisy outputs. Heavily edited or filter-laden images teach the model to produce that specific edit/filter as part of the subject. The rule is: if you would not be happy with an output that looked exactly like your training image, do not include that image in your training set.

03

Captioning: the make-or-break decision most people get wrong

Each training image is accompanied by a text caption that describes what is in it. During training, the model learns to associate the caption with the image, when you later prompt the model with similar text, it produces a similar image. The captions are how you tell the model what to learn from each image. Getting captioning right is harder than it sounds, because the captions interact with the LoRA in a counterintuitive way.

The basic principle is that anything you put in the caption becomes a controllable variable, the model learns that this specific feature is associated with this specific word, and it will let you turn that feature on or off via the prompt. Anything you do not put in the caption becomes part of the implicit identity of the subject, the model learns that this feature is always present and bakes it into the LoRA itself. This is the rule that makes captioning subtle and important: you caption the things that should be variable, and you omit the things that should be constant.

Take a concrete example. You are training a character LoRA for a fictional character named Aria, who has long red hair and always wears a green coat. If you caption every image as 'aria, a woman with long red hair wearing a green coat, [background description],' the model learns that 'long red hair' and 'green coat' are independent attributes of women in general, not features of Aria specifically. When you later prompt 'aria standing in a forest,' you might get a generic woman in the forest, because the red hair and green coat were captured by the captions, not by the LoRA. The fix is to caption every image as just 'aria, [background description].' Now the model learns that aria is the implicit name for the entire visual identity, the hair, the coat, the face, all of it, because all of those things are present in every training image and none of them are mentioned in the captions.

Conversely, if Aria sometimes wears the green coat and sometimes wears a leather jacket, you should caption the coat-wearing images as 'aria wearing a green coat' and the jacket-wearing images as 'aria wearing a leather jacket.' Now the model learns that the coat and the jacket are variable attributes, and you can prompt 'aria wearing a red dress' and get a coherent result with the dress instead of either the coat or the jacket. The captioning structure encodes which features are part of the identity and which are part of the styling.

This is also why the choice of trigger word matters. A trigger word is the unique token you use in captions to refer to the subject, 'aria' in our example, but more often something deliberately weird like 'ohwx' or 'sks' or '0r4n93h31r' to make sure the word does not collide with anything the base model already knows. You want the trigger word to be something the model has not seen during pretraining, so that you are teaching the model a brand new association rather than fighting against an existing one.

Practical captioning workflow

In practice, most people use an automated captioning tool to generate initial captions and then edit them by hand. The two common tools are BLIP (which produces natural-language descriptions) and WD14 tagger (which produces tag-style descriptions like 'long_hair, red_hair, green_coat, outdoor, daylight'). Tag-style captioning works better for FLUX and SDXL, but you have to manually remove the tags that describe features you want to be part of the implicit identity. The auto-captioner does not know which features are supposed to be learned by the LoRA and which are supposed to be variable, that judgment is the artist's job.

Check your understanding

pass: 5 of 7

Answer at least 5 of 7 correctly to unlock the next chapter.

  1. 1. What is the community's consensus about LoRA dataset size versus quality?

  2. 2. Why does using fewer than about 15 images for a subject LoRA cause problems?

  3. 3. Why do style LoRAs typically need more images, around 30 to 60, than subject LoRAs?

  4. 4. What is described as the single most common mistake in LoRA training?

  5. 5. Why does the resolution of your training images matter?

  6. 6. What is the key captioning rule in LoRA training?

  7. 7. For a character named Aria who always wears a green coat, why should you omit the coat from the captions?

The weekly briefing

Get the week's moves in your inbox.

A short, sourced digest of what actually moved across generative AI, every week. Free.

Free. One email a week, no spam, unsubscribe anytime. Prefer a reader? RSS.