Definitions

The identifier the registry probes, written host/model.

The leading segment is the serving host; the rest names the model. The same open weights served by two hosts are two separate entries, each with its own price.

field: model_key

The share of probed samples whose output both parsed and matched the requested JSON Schema.

Parsing and conforming are separate: output can parse as JSON and still violate the schema. Reported with a 95% confidence interval and its sample count.

field: conform_ratehow it's measured ▸

The share of conformant samples whose field values also matched the known-correct answer.

Conformance checks the shape of the output; value accuracy checks the numbers inside it. A model can pass one and fail the other.

field: value_accuracy_meanhow it's measured ▸

The cost of 1,000 outputs that actually passed the schema on this corpus, repairs included.

Total spend — retries and repairs counted — divided by the number of conformant outputs. A cheap model with a low conformance rate can cost more per passing output than a dearer one that passes. This is the column that ranks the table. Methodology note: cost counts every billed token, and a model's reasoning (thinking) tokens are billed as output — so a model that reasons heavily carries that spend in this figure. Since cycle-003 every model is probed at its provider-default reasoning setting (earlier cycles pinned some Gemini models to thinking level high — retired 2026-08-10). If a figure is ever measured under a non-default setting again, its row carries a disclosure marker.

field: effective_usd_per_1k_conformant

A served model's quality gap to the reference model on the same replayed samples.

Regret is quality(reference) − quality(arm), paired sample by sample and reported with a confidence interval and its sample count. Zero means the arm matches the reference; a negative number means it beats it. It is always measured on your own replayed traffic, never asserted — a regret figure without its n and CI is not a claim.

field: regret_mean

The arm a bake-off treats as the ceiling for the task — every other arm's regret is measured against it.

By default the strongest declared candidate for the task (by probed schema conformance in the registry); a route may name one explicitly. It runs on the same replayed samples under its own cost envelope, because the strongest arm is the dearest by definition. It is a yardstick, not a proposal.

field: is_reference

The per-unit-of-work budget you consider reasonable — the cost bound a proposed model must stay under.

Declared per 1,000 quality-passing outputs or per run. A challenger is only proposed when its cost is within the ceiling, so a better-and-dearer model is proposable while it stays under your budget — near-optimal at a price you set, not the cheapest that passes.

field: price_ceiling

What a bake-off optimizes for: lowest regret under a ceiling (quality-first), a blend (balanced), or cheapest-that-passes (cost-first).

Quality-first — the default once a route declares an objective: or bakeoff: block (a route with neither stays cost-first, unchanged) — minimizes regret subject to cost ≤ your ceiling, so nothing is proposed until you declare a regret bound or a ceiling. Cost-first reproduces the older gate. The objective is declared on the route; the report names the one it ran under.

field: objective.mode

A promotion whose winner costs more than the incumbent — better-and-dearer, proposed under your ceiling to cut regret.

The mirror image of the shipped down-swap ("cheapest that passes"). An up-swap raises your effective cost to buy quality, so it is presented as a quality investment, never as a saving: the PR body states the cost increase and the regret cut (with n and CI), and the verified-savings odometer excludes it entirely. Like every swap it never auto-executes — a human approves the PR.

field: evidence.swap_direction

A route's regret over time — a recent window versus a trailing baseline, each with its sample count and confidence interval.

The quality twin of the cost odometer's savings band. It is read back from the regret monitor's sampled bake-off cycles, so every number on it is measured on your own replayed traffic. A band without its n and CI is not a claim.

field: regret_mean

When the recent regret band's upper confidence bound exceeds ε (your max_regret), or drifts materially above the baseline.

ε is the regret bound you declare on the route (objective.max_regret). A breach — with enough paired samples on both windows to trust the number — files one regret_breach bake-off proposal for a human to review; on thin data the monitor refuses rather than propose. The alarm is the sampled band; the bake-off is the evidence.

field: objective.max_regret

A scheduled, envelope-bounded bake-off that replays a sample of recent captures against your reference to keep regret measured over time.

Each cycle is a two-arm sampled bake-off (your served config versus the reference) under the reference cost envelope; its result mirrors as a marked monitor cycle so the band reads back and never seeds the odometer. It watches for the ceiling rising or your regret drifting up, and files a proposal when you cross ε — quality's standing measurement, the way the odometer is cost's.

field: bakeoff.regret_monitor

The share of grounding-probe answers that carried at least one citation.

Measures whether a model searched when asked rather than answering from memory. Citations are counted as distinct URLs.

field: grounded_ratehow it's measured ▸

The share of repeat calls that reported cached input tokens.

Measures whether declared prompt caching is realized on the serving path, on an identical repeat call.

field: cache_hit_ratehow it's measured ▸

The share of a route's input tokens the provider served from cache over a trailing window, from its own reported usage.

The production-traffic counterpart to the leaderboard's cache-hits column, which probes identical repeat calls. Measured per candidate model on live traffic with replays excluded; a route reusing a stable prompt prefix trends high, while one at 0% is paying full price and may not know it.

field: cacheHitRate (route cache-stats rollup)

Whether a row's cache key was generated by ModelRig or supplied by the customer, read from the row rather than stored in a column.

ModelRig stamps its generated key equal to the prefix fingerprint, so origin is derivable with no extra column. A cache_key equal to prefix_fingerprint is a ModelRig-generated key (the route's cache: auto stamping); a non-null cache_key that differs is a customer-supplied handle passed through verbatim; a null cache_key means no key was sent. A customer handle names the provider-native cache resource the customer owns — ModelRig carries it and prices the hits but never manages its lifecycle.

field: cache_key vs prefix_fingerprint (derived, no column)

What an input token really costs on a route once cache reads and write premiums are counted, per million tokens.

The list input price assumes every token is uncached. The effective rate blends the cached-read rate, the cache-write creation premium, and the uncached list rate by their observed shares, so it is the honest marginal input cost rather than the sticker price.

field: effectiveInputPerMTok (route cache-stats rollup, spec §4.2)

The one-time cost of moving a route to a new model — writing its prompt prefix into the new cache while abandoning the old warm one.

A cold challenger must re-create cache the incumbent already holds, so a nominally cheaper model can cost more until that write is amortized. Reported on a swap proposal's cache evidence.

field: cache_context.switch_cost_usd (proposals, spec §4.3)

How many calls a route must make after a model switch before the switch cost is repaid by the new model's per-call savings.

Below this count the swap is still under water; above it the cheaper model is genuinely cheaper. A proposal whose break-even exceeds the route's expected volume is labeled cache-marginal, not recommended.

field: cache_context.breakeven_conformant_calls (proposals, spec §4.3)

The cost of 1,000 schema-passing outputs on a route with cache reads and write premiums counted — the cache-aware form of effective cost of conformance.

The north-star for cache decisions: quality-normalized spend that credits realized cache savings and charges realized write premiums, so a swap is judged on cost per successful task rather than raw token price. The cache-blind base is the effective cost of conformance the leaderboard already ranks by.

field: cache-adjusted effective_usd_per_1k_conformant (spec §5.7)

Negative verified savings after a model swap — the change cost more than it saved.

The odometer measures it the same way it measures a gain, from telemetry that actually flowed since the swap, so a negative reading is shown plainly rather than hidden or projected away. Most often it means the new model gave up cache warmth the old one held, so the switch cost outran the per-call saving.

field: odometer.perSwap[].verifiedUsd < 0 (measured, savings/odometer.ts)

The four-value label a swap proposal carries once cache economics are weighed: cache_positive, cache_marginal, cache_negative, or no_cache_signal.

cache_positive clears both an absolute floor and a relative threshold (the hysteresis rule, spec §4.3) and is recommended. cache_marginal is created and labeled but not recommended; cache_negative means the challenger is cache-disadvantaged; no_cache_signal means there is too little cache data to judge.

field: cache_context.verdict (proposals, spec §4.3)

The three layers of a registry entry: what the provider declares, what the probe suite measured, and what routed traffic observed.

Probed facts are preferred over declared claims; where the two disagree, the entry records a discrepancy. The date column shows when the probed layer was last measured.

field: probed / declared / observed

The number of probe samples behind a rate, and the 95% confidence interval around it.

Probed rates are sampled statistics, not deterministic tests. A rerun reproduces a number when it lands inside the interval.

field: samples, conform_ci95

A recorded disagreement between a declared capability and probed behaviour.

For example, a model that declares schema support but was served only via json-mode coaching. Every discrepancy is published, not hidden.

field: discrepancies

The subset of fixtures graded hard, scored separately from the whole corpus.

Strong models saturate the full corpus and stop separating; the hard tier is where their differences show. What earns a fixture the grade — and the design levers behind it — is documented in the probes method.

field: hard_conform_ratehow it's measured ▸

The corpus a row's samples came from — the probe-suite, the authored demo-rig fixtures, or both.

A row probed only by demo-rig is authored-fixture evidence, not field evidence. The counts say how many samples came from each.

field: families, fixture_counts

The share of conformant samples whose values matched the known-correct answer, measured on the selected leaderboard view — a descriptive statistic, not a RigIndex rank.

The interim views slice the corpus by provenance (the authored demo-rig fixtures, the probe-suite) or by difficulty (the hard-tier subset), not yet by task subtype. Selecting one re-ranks the table by its marginal accuracy; it reports what a model scored on those fixtures, not a paired cross-model rank. The RigIndex name is reserved for the paired scoring that earns it.

field: by_family[view].value_accuracy_mean, hard_value_accuracy_meanhow it's measured ▸

A paired cross-model rank per task subtype — Bradley-Terry over same-fixture comparisons, anchored to a dated frontier basket = 100 — published only where every believability gate passes.

Rank comes only from paired comparisons: two models on the same fixture, scored by the same ground truth. Marginal accuracies never produce it — those views stay labeled descriptive, not a RigIndex rank. Every published score is probe-derived (single source) until community pairs arrive, carries a bootstrap 95% confidence interval, and overlapping intervals render as one tie-tier, never a fake order. A subtype that fails a gate shows its honest state — insufficient data with its n, or the all-tie sentence — instead of a rank. Scores above 100 mean above the frontier-basket average on that subtype; the basket is dated and re-basing is a loud, versioned event.

field: layers/rigindex.json — subtypes[subtype].models[model_key]

The authored demonstration corpus — fixtures we wrote for the demo rig, not field traffic.

A row whose schema samples all come from demo-rig is authored-fixture evidence and is labeled that way. The raw key is kept on the board so a reader can trace a row straight to the fixture files.

field: families, by_family

The standing probe corpus — the versioned fixtures every registered model is probed against.

Fixture families are provenance: they say which corpus a sample came from, not which task subtype it exercised. Task-subtype views arrive with the next corpus expansion.

field: families, by_family

A real task-type grouping of fixtures — extraction, classification, numeric reasoning — tagged per fixture and scored from the same published samples.

Unlike a fixture corpus (provenance), a subtype says what the task is, and it is cross-corpus by design: a demo-rig and a probe-suite fixture can share one. Thin subtypes are min-n gated to "—" until more fixtures land. Accuracy on a subtype is a descriptive statistic, not a RigIndex rank.

field: by_subtypehow it's measured ▸

The share of conformant samples served by native strict schema enforcement rather than prompt coaching.

A model coached into JSON through the prompt is a different serving path from one whose API enforces the schema. The rung records which one produced the sample.

field: native_rung_rate

The runtime loop that serves one rig.run call: the route's resolved candidates are tried in order until one returns a schema-valid output or the ladder is exhausted.

It is a different thing from the migration autonomy ladder (the T0–T3 tiers) that shares the word. The full runtime model — resolve, ordering, validate then repair then fall-through, and the boundary of what it will not do — is on modelrig.dev/routing-reliability.html.

field: route.candidates

Advancing from one candidate to the next after the current one has exhausted its budget for a fall-through-triggering failure.

It is triggered by structure and infrastructure only: a schema miss still invalid after the repair rung, a retryable infra class after its budget, or a missing provider key. A schema-valid but low-quality answer is accepted, never a fall-through trigger.

field: RunMeta.attemptsByClass

One dispatch to one model, which is one telemetry row.

A single rig.run call can produce several attempts — serving, failed, and repair attempts each get their own row, keyed by id (the meta.inferenceId of the serving attempt). Conformance is recorded as failureClass being null, not a separate column.

field: RunMeta.inferenceId

The optional schema-repair step the ladder runs on a schema-invalid output before it retries the same candidate or falls through.

Off unless a repair block is declared. Attempt 1 re-asks the same model with the validation errors; attempt 2 hands the invalid output, errors, and schema to a declared repair model. It draws on its own budget, created once per run, and repaired rows carry repaired_by.

field: repair.max_repairs

The typed reason an attempt failed — one of nine, each with its own non-fungible retry budget.

content_invalid, capacity_shed, network, refusal, cache_invalid, and timeout are retryable; config_auth advances to the next candidate; budget_exhausted and invariant_violation are terminal and abort the run. The class rides RigFailure.class and RigFailureError.failure.class.

field: RigFailure.class

What the candidate ladder actually enforces: a JSON-Schema conformance check, not a judgment of whether the answer is good.

The opposite term is a quality gate — a builder-defined judgment of whether the answer is good. On the SDK lane this exists: a `qualityGate` predicate can reject a schema-valid output at run time (recorded quality_rejected). A route-DECLARED judge that does the same on the hosted lane is not built yet; either way, offline bake-offs also score value accuracy. See modelrig.dev/routing-reliability.html.

field: RunMeta.validated

The versioned, metadata-safe record of what happened on one model call — finish reason, refusal, hashes, cost components — never the content.

AttemptMeta@v1 on inferences.meta: finish_reason, refusal, model_snapshot, context_bucket, reasoning_tokens, http_status, error_code, region, cost.* components, response_chars, and sha256 input/output hashes. Every field is an enum, number, boolean, hash, or identifier — the classification rule guarantees no free text. It is the substrate the self-describing analytics API (GET /v1/analytics/meta, POST /v1/analytics/query) queries: metadata only — enums, numbers, hashes, identifiers; never content.

field: inferences.meta

Five well-known tag keys — subject, feature, user, session, cost_center — that become first-class group-by columns in the analytics grammar.

They stay ordinary tags in storage (GIN-indexed, no schema change) but are documented, lint-nudged (modelrig validate warns on near-misses like customer→subject), and queryable as bare columns. subject is the customer/tenant the work is for — an opaque customer-side id, hashed by the caller if it looks like PII. A row with no subject is always shown as its own 'unattributed' group.

field: inferences.tags

A sha256 of the rendered request (system + user + schema) recorded in place of the request itself — a fingerprint, never the text.

input_hash on the attempt envelope lets you count distinct prompts, detect drift, and join a run to its request without ever storing the prompt. output_hash does the same for the response. Because it is a hash it is ≤128 chars, whitespace-free, and passes the metadata-safe rule by construction. This is why the analytics API can say 'metadata only — never content': what would be content is a hash.

field: inferences.meta.input_hash

A per-customer statement — attempted · verified · billable, with the true cost we measured — derived on read from your telemetry, never stored.

A tenant is one of your end customers. A request carries its tenant with a subject tag (an opaque customer-side id) or a project-scoped key; everything keys on that tenant node. The statement is computed on every read from the same rows the cost side settles on — tokens, cache reads/writes, the deterministic outcome — so there is one graph and no second store. It is a meter of record, not a billing system: it never invoices, taxes, duns, or moves your customers' money. It carries counts, cents, and hashes only — never a prompt or a completion.

field: GET /v1/tenants/:project_id/statement

A run whose result passed deterministic conformance — a run-outcome@v1 pass or a schema-conformant artifact — and nothing weaker.

Verified means measured, not judged: a model judge or a human thumbs-up appears on the statement as quality context but never makes a run verified or billable. This is the same 'deterministic before judge' rule the near-optimal loop uses. billable is derived from verified — so you can bill only outcomes you can prove.

field: statement.volume.verified

Usage with no resolvable tenant — no subject tag and no project-scoped key — shown as its own row, always, on every tenant surface.

The unattributed row is never hidden and never zero-suppressed: it is the honest home for traffic you have not tagged yet, and for the overflow when an org exceeds its tenant cap (a counted cap_overflow). modelrig validate warns when a route has a rate card but no tenant source, so unattributed spend does not surprise you.

field: statement (unattributed row)

The honest label for cost we could not price exactly — an unpriced model's rows are shown at list, never folded into the exact total.

When a model has no registry price, its rows are summed into a separate 'priced at list' line rather than silently into the true-cost total, so the number you read is never rounded in our favour. In TB-1 every statement is 'priced at list' until rate cards land — price and margin read null until then.

field: statement.cost_cents.pricing_missing_rows

A signed, published record of what was measured on one bake-off, run, or route — counts, scores, and hashes only, never content.

Receipt@v1: kind, issuer alias, the measured table (n and CI where present), model list, honest-limit lines verbatim from the producing spec, a sha256 hash, and an Ed25519 signature. It is published as a login-free page at /r/<id> that anyone can read and verify without a database ever being on the path. Internal route/task/pipeline names are aliased by default; the mapping never leaves the org.

field: artifacts.type=receipt

Re-fetch a published receipt, re-canonicalize it, re-hash it, and check the signature — anyone can, with no login.

GET /v1/verify/<id> returns { valid, hash, key_id, issued_at }. The published object is fetched from the content store (never the database), its canonical JSON is re-hashed, and the Ed25519 signature is checked against the public key at /.well-known/modelrig-receipts.json. Any byte changed since publication makes valid false. It reads 'measured by ModelRig — verify' — a pointer to the check, not an endorsement.

field: GET /v1/verify/:id

A receipt published for one recipient, reachable only by its 128-bit capability URL — used for statements and disclosures.

The receipt id is the capability: no listing endpoint exposes it, so anyone holding the link can read the receipt until the org revokes it (which writes a tombstone and returns 410). Used to send a statement or a disclosure pack to a single customer without an account. It is a capability URL, with the honest limit that anyone holding the link can read it until revoked.

field: visibility=recipient

Per-route masking that tokenizes detected spans in the request before it leaves your process, then proves both that the masked arm leaks no detected span and that masking didn't change the answer.

A route mask: policy runs deterministic detectors plus your dictionary and field rules over the outgoing request, substitutes run-scoped tokens, and rehydrates the answer after. Two proofs back it: the leak proof re-runs the detector over the masked request and finds nothing; and a masked-vs-unmasked bake-off replays your captured traffic against the same model and reports the value-accuracy Δ with a confidence interval, gating promotion of the policy. Deterministic + dictionary + field rules — free-text names you didn't list need a v2 detector sidecar; the Δ is measured on the replayed corpus, on the stated n.

field: routes/*.yaml mask:

The counts-only record of what one step masked — findings by type, tokens issued, rehydrated, unresolved — never a matched value.

Persisted per inference and rolled up per run; the same single-producer, counts-never-values discipline as the scrub summary. It answers what a route masks without moving a secret into the record.

field: inferences.mask

Replacing the run-scoped tokens in a model's response with their originals, after schema validation, inside your process.

The response is validated as the model produced it (tokens and all), then every string leaf is walked and known tokens are restored. Unknown tokens the model invented are left verbatim and counted as unresolved.

field: MaskReceipt.rehydrated

The in-process map from token to original value, scoped to one run and never exported — the key that stays with you.

The same value maps to the same token across every step of a run, so a token denotes the same person throughout. The vault lives in your process, is dropped at run end, and is never reachable from telemetry, exports, or receipts.

field: RunVault (in-process)

The published per-identifier table showing, for each HIPAA Safe Harbor class, whether ModelRig catches it deterministically in-process, needs the customer-run sidecar, or leaves it out of scope.

Deterministic detectors cover the classes with a shape or checksum — dates, ages over eighty-nine, fax, URLs, IPs, ZIPs, account and license numbers, VINs and device ids — beside scrub v1's SSN, email, phone and MRN. Names and street addresses need the sidecar; biometrics and photos are out of scope. Recall is measured on synthetic PHI by the SC-3 probe, marked pending until it runs — never a guessed number, never a compliance claim.

field: ScreenReceipt.findings[].type (pii:*)

A customer-run detector container ModelRig calls over a tiny HTTP contract for the free-text identifiers deterministic rules miss, returning span counts only — never the text back.

ModelRig ships the adapter and the contract, never a bundled model. The route POSTs the segment to your own endpoint and reads back spans (offsets, type, confidence); only per-type counts reach the receipt, under the id sidecar:<digest>. An unreachable sidecar fails closed: degraded under detect, pre-dispatch block under enforce on a PHI route.

field: ScreenReceipt.findings[].detector (sidecar:<digest>)

The load-time gate that refuses a PHI-classified route unless it masks its content and either retains nothing or runs a de-id sidecar.

A route with classification phi must have mask mode at least redact, and must either require zero_retention or configure a sidecar detector. Otherwise it refuses to load with a teaching error naming the unmet requirement. It is deterministic and dependency-free; it does not claim de-id, it enforces that PHI never rides an unmasked, content-retaining route.

field: RouteBundle.classification