AI Campus • Practical resources
Data Science & Evaluative Benchmarking
Choose a metric that matches the task and inspect failures behind aggregate scores.
Before you start
- Python and basic fractions; a small labeled dataset and saved outputs.
The reading and planning exercises need no paid course or API. Optional inference, training, IDE features and hosted services may require hardware, accounts or paid access. Use the local or manual alternative before spending money.
Start with the official learning material
Free-to-read references checked 3 November 2026. Reading documentation does not activate a provider or create a subscription.
Practice with a concrete exercise
1. Define the decision and labels
Read the classification-metrics section. Build a 20-item toy table of expected and predicted categories. Calculate true positives, false positives and false negatives for one class. Explain which error is more costly in your task.
2. Compare useful metrics
Calculate precision and recall, guarding against zero denominators. Inspect a class-imbalanced case where accuracy appears high despite missing the important class. Do not use text overlap scores as automatic proof of factuality.
3. Track a candidate regression
Run the related offline-evaluator starter on saved baseline and candidate outputs. Add a small human review rubric and retain per-case failures. Label phrase checks, reviewer scores, latency and missing measurements separately.
Check what you learned
- Metric definitions and denominators are visible.
- At least one failure case accompanies every summary score.
- No unmeasured model quality or cost metric is presented as observed.
Keep your artifact, observed checks and remaining limitations in a short README. Mark any exercise you only designed rather than executed.