Over- and under-refusal

Not populated

This chart needs Layer 2 -- a hand-labelled, stratified sample scored against a live, pinned model (docs/evidence/evaluation.md). This environment has no LLM API key and no human labelling effort behind it yet, so nothing is plotted here rather than a number implying a precision the harness cannot support.

This is the headline chart, and it has two lines on it because either one alone is meaningless.

Under-refusal is the dangerous direction: the system answered something it should have declined or escalated. Every individual miss is listed with its text, never summarised into a rate and left there.

Over-refusal is the direction nobody measures: the system declined, escalated or queued something perfectly ordinary. It is not dangerous, and it is the reason safety systems get switched off. A guardrail that refuses "what does my A1c mean?" has a spotless safety record and no users.

You can drive either number to zero by ruining the other. Reporting them together, at each autonomy level, is the only honest presentation — and the shape of that trade-off at L1 / L2 / L3 is what a clinical owner is actually choosing between.