Skip to content

# L6a: Eval Harness ​

Module: M6 Economics
Files: starter.py, solution.py

## Objective ​

Build golden Q&A pairs + pass@k scoring system.

## Concepts ​

Refer to the corresponding module for full concept explanations.

## Starter ​

bash

    cd course/labs/l6-eval-harness/
    # Open starter file and fill in the TODOs

## Checkpoints ​

1. Define 10+ test cases with tool and output expectations
2. Implement EvalHarness class with score_case
3. Compute pass@1, pass@3, pass@5 metrics
4. Support weighted scoring by case importance

## Solution ​

Compare against the solution file after attempting the starter.