Text-to-Image Generation

Previous · Course overview · Next

Your Goal

Generate Egyptian Mau images with a text-to-image diffusion model, add them to the training data, and find out which generation settings produce useful training images. Along the way, learn to see why a plausible-looking cat is not automatically a useful training example.

What the Repository Gives You

synth-generate runs Stable Diffusion 1.5 or SDXL-Turbo and writes a pool: the images, one manifest line per attempt with every setting needed to recreate it, and a contact sheet of the first 24 images. A pool has 110 images, and each training seed adds a different random 55 of them.

uv run synth-generate --name m2-base --prompt "<your prompt>"
uv run synth-train --name m2-base --method synthetic --pool m2-base

The model starts from random noise and removes it step by step, steering each step towards the prompt. The same settings and seed always give the same image.

Do This

1. Write a prompt and generate the base pool. --prompt has no default: describe the images you want yourself. Open the pool’s contact-sheet.jpg and name the failures you see: wrong breed or coat, impossible anatomy, painting style, repeated poses or backgrounds.
2. Train with it and compare the result with your Meeting 1 baselines.
3. Change one setting per pool, keeping all others as in the base pool. Try at least three of: guidance (for example 2, 7.5, 15), steps (5, 10, 30), prompt (one simple prompt or several varied ones), model, or scheduler.
4. Train every pool with five seeds and record the setting you changed with --tag, for example --tag guidance=15. Then plot each comparison from the results.json files.
5. Choose the pool you take forward and justify the choice with the validation results and the contact sheets, not with looks alone.
6. Test its run with synth-test and compare its test values with those of the Meeting 1 baselines. This is the result you carry into Meeting 3.

Keep everything: the manifest records failed and blocked attempts too. Do not delete images or pools you dislike; describe them.

Examples

The contact sheet of a base pool shows typical Stable Diffusion 1.5 failures next to convincing images: leopard and giraffe patterns, stretched bodies, and striped coats that do not belong to an Egyptian Mau.

In the grid below, each column uses the same seed, so every row starts from the same noise, and each row changes one setting of the base pool, which cycled through four varied prompts; the second row uses one simple prompt instead. Fewer steps give softer images, low guidance follows the prompt loosely, high guidance exaggerates colours and spots, and SDXL-Turbo draws polished close-up portraits. Whether such differences matter to the classifier is what your experiment measures. Some attempts come back black: the safety checker blocked them, and the manifest keeps them as failed attempts.

Checkpoint

Before the next meeting, you should be able to show:

• the base pool and at least three pools that each change one setting, with their manifests and contact sheets;
• the failure types you found, with example images;
• a results table and plots for every comparison;
• the pool you take forward, and why; and
• its test result compared with the baselines.

Useful Materials

• Stable Diffusion 1.5 model card
• SDXL-Turbo model card
• Diffusers: text-to-image
• Diffusers: schedulers
• Diffusers: reproducible pipelines