RAG audit trail with Jev
Retrievers hope. After retrieval, Jev marks relevance, contradiction, or injection; code keeps, flags, or drops passages before a generator sees them.
This unofficial page is the audit trail slice of the RAG passage decisions pack. Intent: apply the Jev (TypeSafe System One) decision model to RAG passage decisions audit trail. Primary search language: RAG Jev audit trail. Confirm patterns on docs.typesafe.ai. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Independent angle (cover ≠ clone): Filter/rerank recipes with failure modes; cite vs generate boundary. We cover the intent, not a rival rerank-passages-score URL tree.
Log passage ids and keep/drop, not bibliography formatting. Citation publish/strip traces live on the citation audit page.
RAG use-case context
An audit trail for RAG passage decisions is a decision trace: replayable inputs, typed answers, floors, and the action the retrieval filter took. It is not a chat log and not a clone of a SIEM product page.
Hub: Classifying RAG passages. Compare, when the other tool is the real job: RAG pipelines.
Audit Trail inputs
Persist the filtered payload (the contract), not whatever arrived at the edge:
{
"query": "What is the refund window for pro plans?",
"passage": { "id": "doc-88#p3", "text": "Pro subscribers may request a refund within 14 days." },
"corpus": { "trust": "internal_kb" }
}Redact secrets before the object hits cold storage.
Decision signals and actions
Minimum fields:
- query hash
- passage ids considered
- per-passage answers
- keep/drop decision
- answerer model (separate from Jev)
Also store usage.input_tokens (vendor meter) and the full probabilities map — argmax-only logs cannot explain a close relevant.
Do not treat a Noul of 0.5 as a “medium” RAG passage decisions score — it means yes and no are equally likely. Conjunctions stay in your code.
Guardrails and escalation
If you cannot explain showing a passage to a customer-facing answerer from the trace, you are not ready to auto-act. TypeSafe’s confidence-gated examples use a lower bar for recoverable reads than for irreversible actions. Those numbers are illustrations. For RAG passage decisions, treat feed_to_answerer as the high bar (showing a passage to a customer-facing answerer). Tune on labels — see offline evaluation.
Evaluation and rollout notes
Traces are the eval warehouse. Replay against passage relevant / not, plus injection gold on a hostile slice after criteria or alias changes. Pin jev-1.13.0 (the versioned id) after you fit thresholds. jev-latest and the marketing line jev-1.13 can move. Log the response model. TypeSafe’s published list price for jev-1.13 is $0.042 per million input tokens (vendor claim — confirm on the models page); output tokens are free on that same page. Unused distractors still bill as input.
Official Python and JavaScript SDKs read TYPESAFE_API_KEY and retry documented 429/529. This site does not sell, issue, or proxy TypeSafe keys. Use a credential you already have from the console or a documented gateway.
Pack map
| Slice | Page |
|---|---|
| Graph and primitives | decision workflow |
What may enter state |
input contracts |
| What to gather first | evidence collection |
| Atomic rules | policy checks |
| Act / review / abstain | confidence thresholds |
| Reviewer payload | human handoff |
| What to persist | you are here |
| How it breaks | failure modes |
| Labeled replay | evaluation |
| Shadow → canary | production rollout |
FAQ
Is the HTTP log enough? No. Persist the filtered state, full probabilities, floors, and downstream action as a decision trace.
May I log raw secrets? Redact in code. Jev will not be your DLP layer.
Where is the rest of the RAG pack? Start with RAG human handoff and RAG evaluation. Cluster hub: Use cases.
Should Jev generate the RAG answer? No. Classify or score passages; another model (or extractive code) writes. That is the cite-vs-generate boundary.
Do we publish rerank lifts? No. TypeSafe’s cookbooks may show measurements — treat those as vendor figures and re-run on your corpus.
What this page does not claim
- No invented top-1 / top-10 lifts.
- Not a vector database.
- Not official TypeSafe.
- Official TypeSafe status, or that jev.pro issues API keys.
- That a schema-constrained answer is automatically factually correct.
Disclaimer
This is an independent unofficial site and is not affiliated with TypeSafe AI; official documentation is available at https://docs.typesafe.ai.
Primary documentation: https://docs.typesafe.ai. Hub: Use cases.
Sources
Public TypeSafe or adjacent documentation only. No private claims.