Synthetic Data Generation with Diffusion Models
A project-based laboratory course on generating synthetic training images with diffusion models and measuring whether they help a classifier.
Course Format
This course consists of five three-hour laboratory meetings. Every student runs the same controlled experiment: can images generated with diffusion models improve recognition of an underrepresented class in an image classifier? The task uses eight cat breeds from Oxford-IIIT Pet, with Egyptian Mau as the minority class. All course materials and project work are in English.
Create your project from the starter repository. It downloads the data, fine-tunes the classifier, runs the diffusion models, and saves every result as JSON; the tables and plots are yours to make. In each meeting you add Egyptian Mau training images in a new way, keep everything else fixed, and measure what changes. You choose on the validation photos and end every meeting with the test result of your choice, so the next meeting starts from what you have measured.
| Meeting | New way to add Egyptian Mau images | Question |
|---|---|---|
| 1 | Oversampling and augmentation of real photos | How far do the classical methods get? |
| 2 | Text-to-image generation | Which generation settings give useful training images? |
| 3 | Image-to-image and IP-Adapter from training photos | How close to the real photos should synthetic images be? |
| 4 | Inpainting and outpainting of training photos | Does the classifier need new cats or new backgrounds? |
| 5 | Final comparison of the test results | Which method would you recommend, and at what cost? |
Bring a coding agent to Meeting 1. If you already use Codex or Claude Code, nothing else is required. Otherwise, follow the short Antigravity CLI setup.
Image generation needs a GPU with about 4 GB of memory (8 GB for SDXL-Turbo): your own NVIDIA card, a faculty computer, or a free cloud GPU such as Google Colab or Kaggle. On our laptop GPU, a pool of 110 images took 1–3 minutes and a five-seed training run about a minute. The repository’s README lists the details for every mode.
Start with Real-Data Baselines.
Assignments
- Real-Data Baselines
- Text-to-Image Generation
- Generating from Training Photos
- Editing Training Photos
- Final Comparison and Presentations
Assessment
| Component | Weight |
|---|---|
| In-class project development | 30% |
| Final project and presentation | 50% |
| Individual project total | 80% |
| Final test | 20% |
| Course total | 100% |
| Total score | Grade |
|---|---|
| 92–100% | 5.0 |
| 84–<92% | 4.5 |
| 76–<84% | 4.0 |
| 68–<76% | 3.5 |
| 60–<68% | 3.0 |
| Below 60% | 2.0 |