Incident response confidence thresholds with Jev
Prometheus rules and PagerDuty already page on numeric thresholds. Jev is optional on messy customer-reported incidents or multi-alert narratives: which SEV band, which runbook class? It does not roll back deploys or page people.
This unofficial page is the confidence thresholds slice of the incident response classification pack. Intent: apply the Jev (TypeSafe System One) decision model to incident response classification confidence thresholds. Primary search language: Incident response Jev confidence thresholds. Confirm patterns on docs.typesafe.ai. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Independent angle (cover ≠ clone): Runbooks and pagers stay; Jev classifies messy alert/customer text into SEV / runbook class, then code pages. Not a runbook-clone or status-page IA photocopy. Treat floors as production gates and price the false-reject cost — official 0.5/0.9 sketches are illustrations.
Incident response use-case context
Thresholds turn incident response classification answers into act / review / abstain. They are product policy, not a hyperparameter TypeSafe ships. Official 0.5 / 0.9 sketches are illustrations. This slice also carries the false-reject discussion: over-gating incident response classification hides calibration.
Hub: Use cases. Compare, when the other tool is the real job: incident runbooks.
Confidence Thresholds inputs
You need (1) pinned answers on a frozen contract and (2) labels for SEV gold from ICs, runbook-class gold, and whether a page was warranted. State shape:
{
"report": { "id": "INC-44", "text": "Checkout 500s since 14:02 UTC after the payments deploy. Status page still green." },
"signals": { "error_rate_bucket": "high", "payments_deploy_recent": true },
"policy": { "sev1": "SEV1 = complete checkout loss or safety." }
}Decision signals and actions
| Axis | Where it lives | Incident response use |
|---|---|---|
choice / score / noul |
answer payload | What to do with the alert/customer incident narrative |
confidence |
Choice & Score only | Whether to trust the argmax |
| Distance from 0.5 | Noul | Whether customer_impact is decided |
FLOORS = {
"open_ticket": 0.50, # illustrations — replace
"page_sev1": 0.85,
}
NOUL_TAU = 0.70 # for customer_impact
def allow(ans, action):
return ans.confidence >= FLOORS[action]Do not treat a Noul of 0.5 as a “medium” incident response classification score — it means yes and no are equally likely. Conjunctions stay in your code.
Guardrails and escalation
TypeSafe’s confidence-gated examples use a lower bar for recoverable reads than for irreversible actions. Those numbers are illustrations. For incident response classification, treat page_sev1 as the high bar (paging SEV1 / rolling back via automation). Tune on labels — see offline evaluation.
Band around 0.5 on customer_impact always reviews. Do not copy 0.70 onto Choice confidence.
Evaluation and rollout notes
- Missed SEV1 language on planted checkout-down reports
- False SEV1 pages (pager fatigue)
- IC override rate
Fit loop: pin jev-1.13.0 → replay → plot error vs confidence → pick floors where auto-act error ≤ your SLA. Pin jev-1.13.0 (the versioned id) after you fit thresholds. jev-latest and the marketing line jev-1.13 can move. Log the response model. TypeSafe’s published list price for jev-1.13 is $0.042 per million input tokens (vendor claim — confirm on the models page); output tokens are free on that same page. Unused distractors still bill as input.
Official Python and JavaScript SDKs read TYPESAFE_API_KEY and retry documented 429/529. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Pack map
| Slice | Page |
|---|---|
| Graph and primitives | decision workflow |
What may enter state |
input contracts |
| What to gather first | evidence collection |
| Atomic rules | policy checks |
| Act / review / abstain | you are here |
| Reviewer payload | human handoff |
| What to persist | audit trail |
| How it breaks | failure modes |
| Labeled replay | evaluation |
| Shadow → canary | production rollout |
FAQ
Should page_sev1 use 0.9 everywhere? No. Over-gating hides calibration and dumps the queue on humans. Fit per action.
Can I reuse a Noul τ as Choice confidence? No. Jaggedness: they are not interchangeable. See confidence.
Where is the rest of the Incident response pack? Start with Incident response decision workflow and Incident response human handoff. Cluster hub: Use cases.
If metrics already say critical, should we wait for Jev? No. Metrics page now. Jev is for leftover messy text. See safe defaults.
Can Jev write the status-page update? Not in this workflow. Classification only. Generation is a different job.
What this page does not claim
- Not a pager, status page, or IR retainer.
- No MTTR or uptime claims.
- Not official TypeSafe.
- Official TypeSafe status, or that jev.pro issues API keys.
- That a schema-constrained answer is automatically factually correct.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai.
Primary documentation: https://docs.typesafe.ai. Hub: Use cases.
Sources
Public TypeSafe or adjacent documentation only. No private claims.