one canonical IR typed: choice · boolean · score abstention is a response, not an error

The deep-dive docs.

How the runtime works when you want more than a landing page: the decision IR field by field, the three question types, the calibrated confidence gate, the seven-rung escalation ladder, declarative graphs, the browser SDK, and the self-hosted binary. Written against the crate sources — where a rung needs a backend your build does not ship, this page names the refusal in the same breath. The normative wire reference is api.html; this page explains the machine it documents.

Overview

A decision is a typed value, not a sentence.

OpenCodifier is a local-first, deterministic-first decision runtime. You hand it unstructured state and a question; it returns a machine-actionable answer with a calibrated confidence, a distribution over the possible answers, and the execution trace that produced it. It is not a chatbot, not an LLM wrapper, and not a decision-tree classifier — it is the substrate an AI system calls before it pays for a large model.

The three decision types

The three canonical question types
TypeYou supplyYou get back
choice a question and a candidate list — ids plus descriptions, defined per request the winning candidate, a full distribution over the candidates, and a confidence
boolean a yes/no question as text true/false with its probability and a confidence
score a question and at least two ordered level labels the level, the full ordered distribution, and an expected value

The candidate list is runtime-defined, not baked into a model's label set. That single choice is what makes the whole system candidate-conditioned: the same engine routes a ticket, picks a model, and grades a severity, because the answer space arrives with the request.

The two rules everything else follows

Deterministic first. The engine always spends the cheapest mechanism that can decide with sufficient confidence — exact rule, cached decision, metadata filter, lexical match, then, only if it must, embedding, decision model, verifier. It never runs a more expensive layer than the question requires, and it never silently discards a candidate on weak evidence.

Refusal is first-class. Low confidence yields an explicit abstain or escalate, never a forced guess — and an abstention ships as HTTP 200, because the runtime answered the question you actually asked ("can this be decided?") honestly. Every hard failure has a stable, machine-readable code instead of a message to grep.

60-second quickstart

This is the HTTP shape — the same envelope every surface (WASM, MCP, CLI) normalizes into. Build the binary, serve it on loopback, and decide something:

That request is the minimal legal payload: policy and metadata are optional on the wire, and omitting them means the engine's default posture (accept at ≥ 0.80, verify below 0.65, abstain below 0.50, risk low). A quickstart should not have to invent housekeeping fields to ask one question.

No server? The same request from the terminal: opencodifier decide --input request.json. No install at all? The Decide console runs the identical engine compiled to WASM, in your tab, with no network.

What is shipping, and what is not

The binary is the product today: decide, graph validate, serve, mcp serve, and the model/ladder/recipe tooling, Apache-2.0, fully offline. The hosted API is a waitlist — planned at $1 per million decisions, no endpoint yet. Nothing on this page requires it, and nothing here describes an endpoint you cannot already run yourself.

The decision IR

One request shape, one response shape.

Native JSON here; OpenAI, Anthropic, and Jev shapes are adapters that normalize into this same canonical form before the pipeline sees them. Vendor formats live at the edges — this is the internal truth, and it validates itself: the IR types refuse to exist in an invalid state, so the engine never re-checks an invariant.

Request — POST /v1/decide

Request fields
FieldTypeRequiredNotes
state.textstringyes The unstructured state to decide on. Non-empty; ≤ max_input_bytes (1 MiB default). JSON pasted here is sent as text — the engine reads text.
state.factsmap of typed factsno Structured facts the rule engine and filters can act on. Each fact is tagged: {"kind":"text"|"integer"|"float"|"boolean"|"list", "value":…}. Floats must be finite — NaN is refused, so cache keys stay deterministic.
questionsarrayyes 1–32 questions, decided in order. An empty array is ir.empty_questions — deciding nothing is not a decision.
policyobjectno The confidence/risk policy. Omitted means the engine default posture (§4).
metadata.request_idstringno Carried for your own correlation; never interpreted.
metadata.limitsobjectno Per-request resource ceiling. A request may declare anything at or under the default, never above.
metadata.limits (defaults)
max_input_bytes 1048576 · max_questions 32 · max_candidates 256 · max_graph_nodes 128 · max_execution_time 120 s · max_retrieval_results 64

The three question shapes

Question shapes
ShapeFieldsRefused when
{"type":"choice",…} id, text, candidates[] — each {id, description} the candidate set is empty (ir.empty_candidates), an id repeats (ir.duplicate_candidate), the text is empty (ir.empty_field), or the set exceeds max_candidates (ir.invalid_policy)
{"type":"boolean",…} id, text the text is empty (ir.empty_field)
{"type":"score",…} id, text, levels[] — each {label}, in order fewer than two levels (ir.too_few_levels), a repeated label (ir.duplicate_level), or empty text

Candidate description fields may be empty but must be present — and you should write real ones: they are the primary signal the semantic layers score against. Identifiers accept a-z A-Z 0-9 _ - . and carry a length cap; anything else is ir.invalid_id.

Policy

Decision policy fields
FieldDefaultMeaning
min_confidence0.80 calibrated confidence at or above which a Low/Medium-risk answer is accepted
verify_below0.65 below this, the answer goes to a verifier instead of being accepted
abstain_below0.50 below this, the runtime refuses to answer at all — see §4
risk"low" low | medium | high | critical. High and Critical are verified even when confident — this is policy you declare, never something inferred from model output.
entropy_ceilingoff entropy at or above this may not be accepted outright (§19 gate)
min_marginoff a margin below this (a near-tie) may not be accepted outright
ood_ceiling / ood_modeoff / answer_entropy an out-of-distribution score above the ceiling demotes the decision; ood_mode chooses what is measured — answer-entropy, or lexical_band input-likeness where a lexical rung ran
boolean_thresholdoff (argmax) answers a boolean true when p_true ≥ threshold instead of by argmax — a fitted boundary, never a hard-coded 0.5
abstain_candidateoff a synthetic choice candidate {id, description} appended to every choice question, so the deciding rung can elect "none of these" instead of guessing (§4)
max_label_overlapoff ceiling on label-set overlap between candidates, for policies that want to measure ambiguity in the candidate list itself

The three gates must satisfy abstain_below ≤ verify_below ≤ min_confidence; anything else is a contradictory cascade and is refused with ir.invalid_policy. The opt-in gates serialize as absent when disabled, so a policy that does not use them is byte-identical to the pre-gate form and cache keys do not shift.

Response

Response fields
FieldTypeNotes
answers[]per-question answers, request order A decisive outcome (accept, verified) always carries at least one; an abstention may legitimately carry none.
outcomeenum accept · verified · verify · abstain · escalate · no_valid_candidate. Only the first two carry a usable answer.
confidencereport Six dimensions, one per question request — §4.
traceordered entries {trace_version, entries[]}, each {node, detail}. Deterministic facts about execution — never chain-of-thought. Details are sorted maps, so traces diff cleanly.
metricsobject candidates_in, candidates_out, cache_hit, verification_triggered — how hard the narrowing pipeline worked and whether caching or verification engaged.

Answer shapes mirror the questions: a choice answer carries {question_id, choice, distribution, confidence}, a boolean carries {question_id, value, probability, confidence}, and a score carries {question_id, expected, level, distribution, confidence}. Every distribution is a complete categorical one — unique keys, each probability in [0,1], total mass 1 within 1e-6 — or the runtime refuses to emit it (ir.distribution_not_normalized). A score's expected value uses positional weights 0, 1, 2, … and its level is the floor of that value, clamped to the range.

A full example, annotated

This is the repo's committed fixture (fixtures/native/request.json): one request, three question types, a policy, and explicit limits.

answers[1].level = "difficult"
expected 2.53 floors to index 2 — but the level list is ordered, and level here is the level at that position: the runtime resolves the expected value into a label for you, so consumers never re-derive it.
confidence.margin = 0.82
top probability minus runner-up (0.91 − 0.09). A small margin is the near-tie signal that drives verification.
confidence.calibrated_confidence = 0.91
the only number a policy gate reads. Here the identity calibration is in effect, so it equals the raw top probability — and the engine identity reports calibration_version: 0 so an un-calibrated run stays visible as such.
verifier_agreement = null
no verifier ran. true means one ran and agreed, false means it disagreed.
trace.entries
this fixture decided through the default pipeline, so the trace opens with the schema normalization entry; a graph run records one entry per node it executed.

The response is always the canonical native JSON projection — the CLI's --format flag and the HTTP x-opencodifier-format header choose how requests come in, never how answers go out.

Question types & criteria

Choosing between three shapes — and wording them so they decide.

Every question is one of three shapes. Picking the right one is most of the integration work; wording it is the rest, because the cheap rungs read text.

choice — pick one from a list you define

Use it whenever the answer is one of a known set: routing, selection, triage. Descriptions do the work — the lexical rung scores your question text against them, the filter rung eliminates provably-out candidates on facts, and the model rung reads them as its candidate conditioning. A candidate list of bare ids with empty descriptions is a request the semantic layers can only guess at.

boolean — yes/no, with a probability

The wire carries a boolean question as text: {"id","text"}. The verdict is argmax over {true, false} unless the policy sets boolean_threshold, in which case it is p_true ≥ threshold — a fitted boundary from labeled evidence (opencodifier ladder fit-boolean emits one), never a hard-coded 0.5.

Criteria are composed into the question text. There is no separate criteria field, because the zero-ML rungs read text — so "yes when … / no when …" is composed into a single string and sent as the question. The Decide console's builder shows the exact composition in its JSON view; nothing is hidden.

Composed criteria must be negator-free

The lexical rung determines a boolean's polarity by counting negator tokens in the question text and flipping the verdict when that count is odd. The list is closed and pinned by a test on both sides: not, no, never, without, cannot, neither, nor. A criteria template that contains one bare "no" therefore inverts the answer it was trying to state — a parity flip, not a parse failure, so nothing warns you. Write criteria as affirmative statements ("satisfied when the invoice is a duplicate charge") and keep the composed text at zero negators; an even count restores the intended reading, but zero is the only count you can reason about at a glance.

score — ordered levels, distribution, expected value

Use it for gradations: severity, difficulty, priority. Levels are ordered by position and must be unique; two is the minimum. The response keeps the full distribution over levels plus the expected value and its resolved level, so a caller can threshold on the distribution ("expert with p 0.12 is not worth waking anyone") rather than on a single label.

One request, many questions

A request can mix all three types — the fixture above does. Each question is decided independently, in order, with its own answer; the request-level outcome is the cascade's verdict for the request as a whole. For independent workloads sent together, use the batch envelope instead (POST /v1/batch, ≤ 16): it shares the decision cache across the batch and isolates failures per item, so one malformed request never fails the other fifteen.

Confidence, calibration & abstention

Confidence is six numbers, and one of them gates.

A single confidence: 0.91 hides everything the verification cascade needs. The response carries the full picture; the policy reads exactly one field of it — the calibrated value — plus the opt-in gates you turned on.

Confidence report fields
FieldMeaningGated on?
top_probabilitythe winning answer's probability before calibrationno
margingap between the top two probabilities — the near-tie detectorvia min_margin
entropyShannon entropy of the answer distribution, in bits (0 = certain, log2(n) = uniform)via entropy_ceiling
calibrated_confidencethe calibrated value — the only field the cascade reads for acceptanceyes
ood_scoreout-of-distribution signal in [0,1]; higher means the input looks unlike anything the rung was fitted onvia ood_ceiling
verifier_agreementtrue when a verifier ran and agreed, false when it disagreed, null when none randrives the verified/abstain tail

Raw softmax probability is never exposed as confidence. Calibration is fitted per rung — an exact relational proof is p = 1.0 by construction, a lexical score is not a probability at all, and a model rung carries a fitted temperature — and the calibration version rides the engine identity and every cache key, so swapping an artifact re-keys rather than silently re-scoring. Where no calibration is fitted, the identity mapping is used and calibration_version: 0 says so honestly.

The cascade

A verifier that agrees promotes the answer to verified; one that disagrees demotes it to abstain or escalate. Verification is confidence-gated — the runtime never runs two classifiers on every request, because "check everything" is how a cheap runtime stops being cheap.

Abstention is a response

When calibrated confidence lands below abstain_below, the answer ships as an HTTP 200 with outcome: "abstain" — not a 4xx, not an error envelope. The runtime answered the question you asked ("can this be decided from this state?") and the answer was no. An abstention never masquerades as a failure, and a failure never masquerades as a low-confidence decision.

There is also an elicited abstention: set policy.abstain_candidate and a synthetic choice candidate is appended to every choice question, giving the deciding rung somewhere to say "none of these" instead of picking the least-wrong option. A decision that elects it becomes an abstain, the answer is withheld, and the trace discloses both the candidate and the elicitation (abstain_candidate / abstain_elicited). The mechanism exists because abstention is not emergent — in the calibration survey recorded in the research notes, a model that faced items it could not know abstained on only 10.5% of them, and was right 17 times out of 18 when it did. The behavior is usable; it has to be offered.

Refusal codes

One envelope everywhere — {"error": {"code": "…", "message": "…"}} — and the same codes across WASM, HTTP, MCP, and CLI. Messages are typed, never an echo of your payload. These are the codes the runtime emits today:

ir.* — canonical IR validation

empty_field · empty_candidates · duplicate_candidate · too_few_levels · duplicate_level · invalid_policy · invalid_id · invalid_probability · distribution_not_normalized · empty_questions

schema.* — adapter layer

unsupported_construct · unsupported_generation_field · missing_field · invalid_type · invalid_value · limit_exceeded · empty_questions · invalid_json

graph.* — graph engine

cycle — the validator reports the first violated invariant it meets, so a malformed DAG may also surface a structural code instead.

engine.* — execution

missing_backend

http.* — the HTTP layer itself

remote_bind_forbidden · bind_failed · serve_failed — the only three codes the web layer owns; everything else is relayed verbatim from below.

cli.* / mcp.* / model.*

The terminal surfaces add their own: cli.escalated, cli.bind_requires_allow_remote, cli.model_rung_unavailable, mcp.session_failed, and the model-artifact family (model.invalid_manifest, model.unreadable_artifact, …).

Two names that appear in friendlier summaries are not codes this runtime emits: gate.below_threshold is not an error at all — that is an abstention, shipped as HTTP 200 with outcome: "abstain" — and an empty candidate set refuses as ir.empty_candidates. If a code you saw somewhere does not appear above, it is not one the binary produces; the list here is checked against the crate sources.

Status codes

Decisions — including abstentions — are 200. Refusals are 400 with the error envelope; oversize bodies and unknown routes are the web framework's own errors. The CLI expresses the same distinction as exit codes: 0 decided, 2 answered but not decisively (abstain, escalate, verify, no valid candidate) unless --abstain-is-success, 1 input error, 3 internal error. The response is printed either way.

The decision ladder

Cheap mechanisms decide. The model pays only for the hard tail.

A decision cascades down the rungs until one of them is confident enough to accept — and stops there. The engine never decides anything a cheaper rung already decided, and it never runs an expensive layer on a question a cheap one settled.

The seven rungs, what they do, and what they cost
RungMechanismWhat it doesMeasured costWhere it runs
1Exact rule relational proofs over extracted facts — accepts at p = 1.0 by construction ~0 mseverywhere, including WASM
2Cached decision versioned, byte-stable replay of an identical prior decision ~0 mseverywhere, including WASM
3Metadata filter removes provably-out candidates on facts — narrowing, never deciding sub-mseverywhere, including WASM
4Lexical / BM25 hand-written scorer over candidate descriptions, gated accept 0.4 mseverywhere, including WASM
5Embedding static-table semantic scoring; annotates, never removes 7.9 msneeds an embedding backend
6Decision model candidate-conditioned parallel readout — the tail, reached only when the cheap rungs did not accept 0.3–2.3 sneeds an inference backend (CPU-only)
7Verifier one gated shot when the gate or the risk policy demands a second opinion on demandneeds a backend

Costs are the measured figures from the ladder runs behind the measured board — CPU-only, consumer desktop hardware — not estimates. Most requests never reach rung 5, which is the entire point.

Why cheap first is a correctness property, not just a cost one

Every rung has its own confidence character: a relational proof is certain by construction, a lexical score is a similarity, a model rung's output is a calibrated distribution. Letting an expensive rung re-decide a settled question would not add information — it would add a second opinion with more ways to be wrong. So the ladder is a one-way valve: a rung fires only while the previous rung's gate did not accept, an accepted question runs exactly one classifier, and the trace records which rung answered, so every decision cites its own provenance.

Which rungs your build actually has

The WASM build — this site's playground and everything the browser SDK embeds — ships rungs 1–4 and the graph engine. Rungs 5–7 need inference backends a browser tab deliberately does not ship; a graph that needs one returns engine.missing_backend instead of pretending. On the server, the same zero-ML stack is the default and the model rung is opt-in: the llama.cpp loopback backend sits behind the llamacpp cargo feature and its --llama flags, and a build without the feature refuses --llama with cli.model_rung_unavailable rather than silently ignoring it.

Graph mode

Declarative DAGs, no scripting language.

A graph chains steps — normalize, decide, gate, output — into one serializable document. It is data, not code: there is no embedded scripting language to sandbox, and the whole document validates before a single node runs.

Node kinds

Graph node kinds
KindRole
normalizecanonicalizes the request into the working state — conventionally the entry node
ruleruns the deterministic rule engine
cacheprobes the exact-decision cache
filternarrows candidates deterministically
lexicalscores surviving candidates lexically with BM25
choice / boolean / scoredecides the question of that type
thresholdapplies the confidence gate and verifier cascade; carries the graph's threshold knob
branchconditional short-circuit: dependents are skipped when when is false
embeddingembeds the state once, scores candidates by cosine — annotates only, never removes or reorders
retrievenarrows each choice question to the top-N semantically scored candidates above a floor; every dropped candidate is named in the trace, and the floor can never empty a question
rerankreorders survivors by a named reranker; never removes
outputterminal — exactly one per graph

Validate, then run

Validation enforces the DAG contract locally — ids, edges, acyclicity, a single output, the node limit, knob kinds — and returns a summary of what validated, so a client can confirm the shape it is about to run. Execution then feeds the graph and the request through the same engine; the graph's cache identity is derived from its own content, so changing the graph re-keys its decisions.

Two refusals are part of the design, not edge cases. A cyclic graph is refused with graph.cycle — break the loop, do not expect the engine to guess an order. And a node whose kind needs a backend the build lacks is refused with engine.missing_backend; the runtime never silently substitutes a cheaper rung and presents the answer as if the graph you wrote had executed.

Per-node gate overrides live outside the graph: a ladder profile (serve --ladder profile.json) carries full policies per node id or per node kind, because a relational proof and a model rung's calibrated confidence are not commensurable and one request-level policy cannot gate both well. An empty ladder runs byte-identically to no ladder at all.

The browser SDK

One global, no dependencies, DOM-free by default.

sdk/decisions-sdk.js is the shared runtime behind every playground game on this site. It exposes a single DecisionsSDK global, defines no dependencies, and touches the DOM only in its explicitly-marked shell section — the headless parts run under Node's test runner.

DecisionsSDK surface
GroupMembersWhat it is for
rnghashSeed, mulberry32, makeRng, pick, randomSeed seeded determinism — a string seed through FNV-1a into mulberry32. No Math.random in game logic, ever: the same seed replays the same match.
configclamp, clampf, normalize spec-driven config normalization for game controls — integer clamps, float clamps, strict booleans, enums with defaults.
rubricnormalize, score, resolve weighted criteria over normalized metrics, a per-run breakdown for auditability, and tie-break-aware winner resolution.
boards / meterssessionBoard, meters, nowMs a memory-only scoreboard that dies with the tab, and EMA/worst/count meters for the stats overlay. Nothing a visitor does leaves the tab.
engineengineBridge the one sanctioned path to the WASM engine — below.
formatfmtMs, fmtPct, fmtTime, fmtNum shared number and duration formatting.
shellh, accordion, tabs, kvRow, fullscreen, numericSeed, resetParams the shared game chrome: element factory, single-open accordion, tabs, key/value rows, fullscreen toggle, seed box, restore-defaults.

engineBridge — the engine call path

Four honesty rules are enforced by construction rather than by convention. The boundary speaks JSON strings both ways — an object into decide() hangs the engine, so the bridge stringifies for you and parses back. An abstention from the engine is a valid non-null result; only a decode or execution failure yields null, and call sites must keep a local fallback for it. Every call is metered. And after failLimit (default 5) consecutive failures the bridge turns itself off — a game never stalls, and never pretends it asked the engine when it did not.

Live: the same seed, the same numbers

This box runs the SDK's own RNG in your tab — the same function every game seeds its match with. No engine, no WASM, no network.

loading…

Change the seed and the sequence changes; restore it and the sequence is bit-identical. That is the whole determinism contract, running on this page.

Deciding from a page

identity() returns the exact versions that decide cache keys and reproduce decisions; decide_batch() takes the same ≤16 envelope as POST /v1/batch and isolates failures per item. Failures throw the standard error envelope as the exception value — there is no third channel. The playground is the working example, and api.html documents the module exhaustively.

Self-hosting

One binary. Loopback by default. No config file.

Everything ships in opencodifier-cli: the decision engine, the wire adapters, the HTTP server, the MCP server, and the model/ladder tooling. The base binary is useful with zero models installed.

Subcommands

CLI subcommands
CommandDoes
decidedecide one request and print the canonical response. Flags: --input (file or stdin), --format (native|openai|anthropic|jev), --policy (override JSON), --trace, --focus-budget, --ladder, --llama*, --abstain-is-success.
graph validatevalidate a graph document against the DAG contract and print a one-line verdict.
servethe HTTP surface on loopback. Flags: --bind (default 127.0.0.1:8177), --allow-remote, --graph, --ladder, --focus-budget, --llama*.
mcp servecodify_decide, codify_batch, codify_graph, codify_validate, codify_verify, codify_explain — stdio only, so the surface is local by construction.
models verifycheck a model artifact's SHA-256 against its manifest before the engine ever loads it.
ladder fit-booleanfit a boolean verdict boundary from {"p_true","label"} JSONL evidence and emit the ladder profile that carries it.
recipe list / installlist the built-in recipe fleet, or write a recipe's graph, request, and captured expected response into a directory ready for serve --graph.

Exit codes are a contract, printed in --help: 0 a decision was produced, 1 input error, 2 the runtime answered but not decisively (abstain, escalate, verify, no valid candidate) unless --abstain-is-success is passed, 3 internal error. Diagnostics go to stderr as opencodifier: <code> <message> with the same stable codes the HTTP API reports — so a pipeline can branch on a refusal without parsing prose.

Configuration, such as it is

There is no config file, and that is deliberate: the binary takes flags, and policy lives on the request in the canonical IR — which is why the same request decides the same way no matter which door it came in through. serve --policy is accepted and validated for symmetry with decide, and then reported on stderr as inapplicable, because silently ignoring a flag would be a false promise. Resource limits ride the request's metadata.limits, never above the engine's ceiling.

Local-first is enforced twice, independently: the CLI refuses a non-loopback --bind without --allow-remote (cli.bind_requires_allow_remote), and the HTTP layer checks the same rule again at construction. The posture does not depend on one module remembering it.

Observability

Every response carries its own metrics and trace, and GET /v1/healthz returns liveness plus the engine identity decisions are cached under; GET /v1/capabilities reports the decision kinds, endpoints, limits, and cache state this build actually has. There is no /metrics scrape endpoint in the binary today — telemetry is per-response and per-decision, by design, and the hosted tier's dashboards are built on the same fields. If you need a scrape endpoint today, run the decisions through the HTTP API and aggregate the responses; the fields are already structured for it.

Running it as a service, and in a browser

A hardened user-level systemd unit, the wasm-pack build for embedding the engine in your own site, and the release matrix with attestations are all on the main page's build section — same binary, same flags. The short version for a service:

Methodology

One number per surface. Never a universal score.

Every figure this project publishes traces to a committed, deterministic run. This page states the rules; the measured board holds the numbers, and board.csv is the raw data behind them.

  • Per-surface, always. A JevBench split score, a locked internal-suite score, and a leaderboard entry are different measurements of different things. The site never averages them, and neither should you — the divergence is published, not hidden.
  • Two boards, two axes. Board A is the JevBench public split (231 items, run with the authors' own harness so scoring cannot drift). Board B is a locked 120-item internal suite that isolates what the field's suite does not — constraint checking, paraphrase routing, relational composition. Internal context, explicitly not comparable to Board A.
  • Derived columns are labeled derived. Trust score and accuracy-per-GiB are presentation-layer formulas computed in your browser with a disclosed formula. They are not project measurements, and the board marks them as such.
  • The CSV is the source of truth. The table on benchmarks.html is a rendering of board.csv, not a separate dataset. Cite the CSV or docs/BENCHMARKS.md — never a screenshot.
  • Determinism is testable. Same request, same engine identity, same answer — bit-identical predictions and probability vectors. Cache keys include model, calibration, policy, and graph versions, so an artifact update invalidates cached decisions instead of quietly re-scoring them.
  • Claims we refuse to make stay refused. "Beats hosted Jev on accuracy" is not claimed and is not true yet; "one accuracy number for every benchmark" is refused on principle. The full list lives on the main page, and it is product, not boilerplate.

Record date for the current board: 2026-09-29, pinned by SHA-256 per model artifact in benchmarks/decision-model/results/models.manifest.json. Method of record for each board is in the repo's docs/BENCHMARKS.md.

FAQ

The questions this page exists to answer.

Is the hosted API available?

No — it is a waitlist, planned at $1 per million decisions. Self-hosting is the product and ships today: same engine, same IR, Apache-2.0, no API keys. The hosted tier's terms are published so you can see what is planned, not what exists.

Do I need a GPU, or any model at all?

No. The default stack is zero-ML — rules, cache, metadata filters, lexical scoring — and it is what decides most requests. Rungs 5–7 are opt-in backends: the model rung runs CPU-only behind the llamacpp feature and its --llama flags, and a build without the feature refuses the flag rather than ignoring it.

Why did I get outcome: "abstain" instead of an answer?

Because calibrated confidence landed under your policy's abstain_below, and the runtime does not guess. It is a 200 response, not an error, and it is the product working: a wrong answer you would have acted on is worse than a refusal you can route to a human or a bigger model. Raise abstain_below's neighbors or add evidence to the state if you are seeing too many.

Can it write prose, or answer free-form questions?

No, and it refuses the attempt: free-form generation fields in an OpenAI-shaped request come back as schema.unsupported_generation_field. Pretending to generate prose is how wrappers lie. If you want prose, call a language model — and use this runtime for the decision about which one.

How is this different from asking an LLM and parsing the reply?

The reply is not the product; the typed answer, the calibrated confidence, and the deterministic trace are. A parsed sentence has none of those: no distribution, no calibration you can audit, no replay. Most software decisions are small and repetitive — routing, flagging, grading — and paying a frontier model to generate a sentence you immediately parse back into an if-statement is the wrong shape.

Can different questions in one deployment use different gates?

Yes. risk on the request policy makes High and Critical verify even when confident, and a ladder profile (--ladder) overrides gates per node id or per node kind — because a relational proof and a model rung's confidence are not commensurable. An empty ladder runs byte-identically to no ladder, and a non-empty one re-keys the cache by its id.

Does it work in the browser?

Yes, for the deterministic stack: rungs 1–4 and graph mode compile to WASM, and every game on the playground runs on it. Rungs 5–7 are refused there with engine.missing_backend — a browser tab does not ship inference backends, and the refusal is the honest behavior.

Why did the CLI exit 2 when it printed an answer?

Exit 2 is "answered, but not decisively" — abstain, escalate, verify, or no valid candidate. The response is still on stdout, complete. Pass --abstain-is-success when your pipeline treats a refusal as a fine outcome; the default is the stricter posture, because pipelines that silently accept refusals tend to keep them.

Does anything leave my machine?

Nothing. No telemetry, no accounts, no external requests from the runtime or from this site's pages. The WASM playground executes entirely in your tab, and game scoreboards live in tab memory only. The only data that ever leaves is the request you deliberately send to the future hosted endpoint.

Where do I report a doc that disagrees with the code?

The GitHub tracker, please — a doc that drifts from the binary is a bug in the doc, and the API page is generated against the crate sources precisely so the drift is visible. Start from api.html for wire behavior and this page for the reasoning; if they disagree with each other, that is a bug too.