L5b: CI/CD Pipeline

Module: M5 Production
Files: starter.py, solution.py

Objective

Create golden dataset + automated regression gate.

Concepts

Refer to the corresponding module for full concept explanations.

Starter

cd course/labs/l5-cicd/
# Open starter file and fill in the TODOs

Checkpoints

1. Define golden test cases (input, expected tools, expected output)
2. Implement score_case evaluation function
3. Run pass@k evaluation (k=1, 3, 5)
4. Gate deployment on pass rate threshold

Solution

Compare against the solution file after attempting the starter.