Final Comparison and Presentations

Compare the test results of the runs you chose in every meeting and present a recommendation backed by the evidence.

Previous · Course overview

Your Goal

Answer the course question with the test results you collected at the end of every meeting: which way of adding Egyptian Mau training images would you recommend for this task, and at what cost?

Do This Before the Meeting

  1. Collect the test results of the runs you chose in Meetings 1–4; no new tests are needed.
  2. Make your final table and plot from their test-results.json files: the mean and spread of every method over the five seeds, and its differences from oversampling and augmentation.
  3. Compare the test results with your validation results. Did the ranking hold? Explain any change.
  4. Turn the evidence into a recommendation: use, revise, or reject synthetic data for this task, including generation time and the share of failed images.

How to Read the Final Comparison

Layer Example
Result A synthetic pool adds +0.05 Egyptian Mau recall over oversampling.
Uncertainty Seed by seed, the gain ranges from −0.03 to +0.13.
Trade-off Macro-F1 did not drop; generating the pool took a few minutes.
Decision Promising, but the test photos cannot confirm it: revise.

The numbers in this table are made up for illustration. If some seeds show a gain and others a loss, that does not prove the method useless; it shows that 30 test photos per breed and five seeds cannot tell the two methods apart.

A rigorous null or negative result receives full credit. The project is evaluated on the quality of the experiments and reasoning, not on making synthetic data win.

Five Questions to Answer

  • Does any synthetic method beat oversampling and augmentation on the test photos?
  • Which generation setting mattered most in Meeting 2, and why?
  • How close to the real photos should synthetic images be?
  • Does the classifier need new cats or new backgrounds?
  • What does each method cost in generation time and failed images?

Presentation

Each student has 12 minutes: 8 minutes to present and 4 minutes for questions. A simple structure is:

Time Story
1 min Question and the fixed protocol
2 min Generation methods and the failures you found
3 min Validation and test results, with their uncertainty
2 min Recommendation, cost, and limitations

Final Checkpoint

  • The final repository commit and the commands that reproduce it.
  • EXPERIMENTS.md covering all four meetings and the final comparison.
  • Each pool’s manifest and contact sheet, and all runs with their validation and test results.
  • A technical report of no more than four pages.
  • Individual presentation and defence of the submitted work.

Useful Materials