Compliance failure modes with Jev
SOC 2 / ISO rows have owners, cadence, and pass/fail. Jev can flag “this evidence blob never mentions production.” The control owner still attests. Jev is not the auditor and not the GRC system of record.
This unofficial page is the failure modes slice of the compliance evidence pre-score pack. Intent: apply the Jev (TypeSafe System One) decision model to compliance evidence pre-score failure modes. Primary search language: Compliance Jev failure modes. Confirm patterns on docs.typesafe.ai. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Independent angle (cover ≠ clone): Checklist is the control; Jev pre-scores whether evidence language looks complete. Owner still signs — not a GRC-clone or rival checklist-IA photocopy.
Compliance use-case context
Compliance evidence pre-score breaks in product-specific ways. This page lists those modes so you can write tests — not a generic “AI can be wrong” essay, and not a rival limitations-page clone.
Hub: Use cases. Compare, when the other tool is the real job: compliance checklists.
Failure Modes inputs
Many failures start as contract violations (distractors, missing control prompt + evidence blob text). Canonical shape:
{
"control": { "id": "CC-6.1", "prompt": "Evidence must describe production access reviews this quarter." },
"evidence": { "title": "Access review export", "text": "We reviewed staging users in January." },
"policy": { "env": "Staging-only language is a gap for production controls." }
}Decision signals and actions
- Marking a control passed because Choice said ready_for_owner (compare).
- Legal interpretation from a Noul — counsel owns that.
- Dumping the whole GRC workspace into state (distractors + vendor token bill).
- Schema-safe
gap≠ an audit finding. - Invented “weeks saved on SOC 2” claims.
HTTP vs application:
| You see | Class | Compliance move |
|---|---|---|
| 401 / 422 / 429 / 529 | Documented HTTP | Fix key/body or back off — errors |
| 200 + flat confidence or Noul ≈ 0.5 | Low confidence | Hold; do not mark the control passed |
| Empty gather | Missing evidence | Skip Jev or ask “is enough information present?” |
Do not treat a Noul of 0.5 as a “medium” compliance evidence pre-score score — it means yes and no are equally likely. Conjunctions stay in your code.
Guardrails and escalation
Fail closed: do not mark the control passed. Schema-safe answers are not factual correctness. TypeSafe’s confidence-gated examples use a lower bar for recoverable reads than for irreversible actions. Those numbers are illustrations. For compliance evidence pre-score, treat mark_control_passed as the high bar (marking a control passed or signing attestation). Tune on labels — see offline evaluation.
Evaluation and rollout notes
Your canary set should include each bullet above.
- Missed staging-only blobs on a planted set
- Owner override rate
- Drift after prompt edits (replay; do not “feel” it)
Pin jev-1.13.0 (the versioned id) after you fit thresholds. jev-latest and the marketing line jev-1.13 can move. Log the response model. TypeSafe’s published list price for jev-1.13 is $0.042 per million input tokens (vendor claim — confirm on the models page); output tokens are free on that same page. Unused distractors still bill as input.
Official Python and JavaScript SDKs read TYPESAFE_API_KEY and retry documented 429/529. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Pack map
| Slice | Page |
|---|---|
| Graph and primitives | decision workflow |
What may enter state |
input contracts |
| What to gather first | evidence collection |
| Atomic rules | policy checks |
| Act / review / abstain | confidence thresholds |
| Reviewer payload | human handoff |
| What to persist | audit trail |
| How it breaks | you are here |
| Labeled replay | evaluation |
| Shadow → canary | production rollout |
FAQ
If the API returns 200, is the decision good? 200 only means the call parsed. Low confidence, Noul ≈ 0.5, or a policy miss are application failures.
Where do official weaknesses live? TypeSafe’s jev-1.13 jaggedness note — distractors, arithmetic, adversarial content. We do not invent more.
Where is the rest of the Compliance pack? Start with Compliance evaluation and Compliance decision workflow. Cluster hub: Use cases.
Can Jev be our auditor? No. It pre-scores language. Owners and auditors sign. See governance.
May we send screenshots? Not as images. Transcribe what the screenshot shows, then ask snap questions.
What this page does not claim
- Not an auditor, GRC, or certification.
- No readiness or pass-rate claims.
- Not official TypeSafe.
- Official TypeSafe status, or that jev.pro issues API keys.
- That a schema-constrained answer is automatically factually correct.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai.
Primary documentation: https://docs.typesafe.ai. Hub: Use cases.
Sources
Public TypeSafe or adjacent documentation only. No private claims.