Triage confidence thresholds
Thresholds turn Jev answers into act / review / abstain. They are product policy, not a model hyperparameter TypeSafe ships. This unofficial page is the threshold slice of the triage pack. Official starting pattern: confidence. We do not sell keys.
Two axes (do not collapse them)
| Axis | Where it lives | Use in triage |
|---|---|---|
choice / score / noul |
answer payload | What queue or how hot |
confidence |
Choice & Score only | Whether to trust the argmax |
| Distance from 0.5 | Noul | Whether to treat yes/no as decided |
Official intent-routing sketch: if intent.confidence < 0.5 → human; if complaint and (complexity.score > 1 or complexity confidence < 0.5) → human. Those 0.5 / 1 values are examples. Fit on offline eval.
Risk-scaled floors
TypeSafe’s confidence page: read-only actions can use a lower bar than irreversible ones.
FLOORS = {
"route_to_queue": 0.55, # illustrations — replace
"page_oncall": 0.85,
"issue_refund_flag": 0.90,
}
def allow(ans, action):
return ans.confidence >= FLOORS[action]
Noul urgency: pick URGENT_TAU (many teams start above 0.7) and a band around 0.5 that always reviews. Do not treat 0.5 as “medium frustration.”
Cascade abort: a separate ABORT on each taxonomy Choice — see workflow.
How to fit (production gate)
- Pin
jev-1.13.0. - Replay labeled tickets with frozen questions.
- Plot accuracy vs confidence; pick a floor where auto-act error rate ≤ your SLA.
- Raise floors after a criteria edit; aliases can move (
jev-latest). - Never copy a Noul τ onto Choice confidence (jaggedness: they are not interchangeable).
FAQ
Should every action use 0.9? No. Over-gating sends the whole queue to humans and hides calibration.
What if I only care about the best option? TypeSafe agent-skill note: you can take the max without a floor. Floors exist for risk.
Hub: Use cases. Siblings: workflow, handoff. Official: docs.typesafe.ai.
Sources
Public TypeSafe or adjacent documentation only. No private claims.