Teaching deck · Seven slides
Rules, purpose, and release evidence
Read at your own pace, or download the deck for a short facilitated introduction.
01 / 07
Rules & Purpose Lab
Deterministic testing tells us whether the system obeyed a rule. AI evaluation helps us determine whether the result fulfilled its purpose.
A trustworthy release process needs both. This short introduction leads into a hands-on customer-support workshop.
Brad Guider · Fictional authored examples
02 / 07
Contracts and purpose
| Deterministic contracts | Purpose evaluation |
|---|---|
| Did the response obey the explicit rules? | Did the response address the customer’s need? |
| Valid JSON, required fields, allowed action, existing policy IDs | Relevant advice, supported claims, a feasible next step |
Deterministic checks are one kind of evaluation. Code, models, and people can contribute evidence about purpose.
03 / 07
One customer, four outcomes
“My unused lamp arrived 10 days ago. How do I return it?”
| Candidate response | Contract | Purpose | Decision |
|---|---|---|---|
| Valid JSON with irrelevant catalog advice | Pass | Fail | Block |
| Helpful return advice in plain text | Fail | Pass | Block |
| Plain text claiming a completed refund | Fail | Fail | Block |
| Valid JSON with supported return advice | Pass | Pass | Eligible |
Authored teaching fixtures. Eligible applies only to this scenario and its criteria.
04 / 07
Evidence can become stale
An original answer
“Please provide your order number so support can help start a return request.”
An added claim
“Your refund has already been processed.”
Old quotations still match, but the evaluated input has changed. The assessment is stale, so the result needs Review.
This detects stale evidence. It does not establish a fresh semantic assessment of the added claim. A defensible reassessment may block the response.
05 / 07
The release decision
| Decision | Evidence required |
|---|---|
| Block | A known contract failure, or a valid uncontested purpose failure |
| Review | Missing or stale evidence, invalid grading, or a material disagreement |
| Eligible | All required evidence is valid and every criterion is satisfied |
A high average cannot cancel a failing requirement. A known contract failure still blocks when grading is unavailable.
06 / 07
The hands-on workshop
- PredictWhich responses should be eligible?
- InspectTrace a decision to its checks and excerpts.
- Repair and reassessChange an exercise copy while preserving the baseline.
- Challenge the judgeExplain why confident grading may be wrong.
07 / 07
The teaching trial
The next evidence comes from learners.
Can they complete the exercise and explain why a passing contract can still lead to a blocked release?
Automated acceptance checks concern lab behavior. Teaching effectiveness remains to be observed. No provider-backed model-quality experiment has been claimed.
python -m rules_purpose demoAdapted from the editable deck and facilitator notes. Browser text reflows for smaller screens. The PDF preserves the original slide design.