Generating from Training Photos

Generate new Egyptian Mau images from the 15 training photos with image-to-image and IP-Adapter, and find how close to the real photos they should be.

Previous · Course overview · Next

Your Goal

In Meeting 2 the model only read a description of an Egyptian Mau. This time it also sees the 15 real training photos. Find out how close to these photos the synthetic images should be: near copies add little new information, while images far from the photos may lose the features that make the breed.

What the Repository Gives You

Two new modes of synth-generate. Both use the 15 Egyptian Mau training photos in turn, so each photo is used equally often. Image i of every pool starts from the same photo with the same seed, so pools are directly comparable.

Mode What the photo does Setting Meaning
img2img the starting point: noise is added to it and then removed --strength share of the photo replaced by noise; near 0 a copy, near 1 a new image
ip-adapter a reference: the model draws a new image of a similar cat --ip-adapter-scale how strongly the photo steers the new image 
uv run synth-generate --name m3-strength-0.5 --mode img2img --strength 0.5 \
    --prompt "<your prompt>"
uv run synth-generate --name m3-ip-0.5 --mode ip-adapter --ip-adapter-scale 0.5 \
    --prompt "<your prompt>"

Both modes still need --prompt. Keep it the same within a sweep, for example the prompt of the pool you took forward from Meeting 2, so that only the strength or the scale changes.

You do not pass a photo yourself: the command takes the training photos, prints which ones it uses, and records the source of every image in the manifest. Add --source Egyptian_Mau_117, repeated for more photos, to start only from the training photos you choose.

img2img runs only steps × strength denoising steps: at strength 0.3 the default 30 steps become 9. The manifest records this as effective_steps.

Do This

  1. Generate one pool of each mode at 0.5 and compare every image with its source photo on the contact sheets.
  2. Sweep one setting at a time: strength 0.3, 0.5, 0.7, and 0.9; IP-Adapter scale 0.3, 0.5, 0.7, and 1.0.
  3. Train every pool with five seeds, tagged with its mode and setting, for example --tag mode=img2img --tag strength=0.5. Then plot each sweep with the setting on the x-axis.
  4. Choose the best pool of each mode on the validation results and compare it with your Meeting 2 pool and the Meeting 1 baselines.
  5. Explain with examples from the contact sheets which images help: near copies of the photos or new poses and backgrounds.
  6. Test the two chosen runs with synth-test and compare their test values with those of Meetings 1 and 2.

Training photos only: the tools generate only from the 15 training photos. Using validation or test photos would let the classifier be evaluated on cats it has effectively seen.

Examples

Each pair of rows starts from the same training photo. Image-to-image keeps the pose and background and changes more of the cat as the strength grows; IP-Adapter keeps the coat colour and pattern but draws a new scene. Look also for things that are copied but should not be, such as a photographer’s signature.

Two Egyptian Mau training photos, each with its image-to-image results at four strengths and its IP-Adapter results at four scales

Checkpoint

Before the next meeting, you should be able to show:

  • pools for at least three strength values and three IP-Adapter scales, with their contact sheets;
  • plots of both sweeps compared with the baselines and your Meeting 2 pool;
  • a paragraph, with example images, on how close to the real photos the synthetic images should be; and
  • the test results of the best run of each mode.

Useful Materials