ModelRig is the open-source operating layer for AI workstreams: your model calls become routes — YAML in your repo — every run lands in an inspectable artifact store, and each route serves the cheapest candidate model that passes your output schema, with probed evidence behind the choice.
Point your coding agent at it.
The fastest migration is the one your agent performs — that path comes first. Prefer your own hands? The steps, spelled out. Either way, Step 0 is the same.
Step 0 — create your org (two minutes, human-only)
Hosted sync — the console, synced telemetry, grades, the artifact store — needs one human prerequisite before either path below. It stays human by design: your coding agent never creates accounts, signs in, or holds credentials.
Show me how (stay on this page) →
- Open the console signup — it opens in a new tab, so this page stays put.
- Enter your email and a name for your organization. That's the whole form.
- In the console, issue an API key (the rig_sk_… value shown once — copy it).
- Back here, export it beside your provider keys: MODELRIG_API_KEY=rig_sk_…
No account needed for the open lane: with only your provider keys set, routes, probes, and telemetry run entirely on your machine — the hosted console and synced telemetry are what the key unlocks. You can add it later; nothing below requires it.
step 1 — paste one prompt
Paste this into Claude Code, Codex, or Cursor from inside your project. Your agent reads the docs, finds your LLM call sites, and turns them into routes:
Migrate this project's LLM calls to ModelRig (https://modelrig.dev).
Work the details from the docs yourself; STOP and present to your human at each ⏸ checkpoint.
OPERATING RULES (read first; they override anything you remember):
R1. Ground in live surfaces, never memory: which models are probed, which knobs a
provider honors, what a claim means — ask https://modelrig.dev/llms.txt, the
raw-lane support matrix (modelrig.dev/route-bundles.html), rawKnobSupport(),
or the MCP oracle. What the routing engine does and does not do at run time
(fall-through triggers, the repair rung, what is stored, what is NOT built) is
https://modelrig.dev/routing-reliability.html — read its "Not yet implemented"
section before proposing any behaviour that depends on a route-declared judge or
raw-lane structure. Declared capability and probed behaviour differ — that gap
is ModelRig's entire thesis.
R2. Key custody: never create accounts, sign in, or read/transmit/store provider
key VALUES. Existence checks by NAME only; humans place keys.
R3. Behaviour-identical, or a NAMED change presented at a checkpoint. Never a
silent change — not sampling, not caching, not grounding, not enforcement.
R4. A ModelRig knob that is absent is byte-identical to before; a declared knob a
provider cannot honor FAILS CLOSED with a classified error. If you hit one,
that is the product working — consult the matrix, don't work around it.
R5. If the human's answers conflict with each other or with the code you read,
STOP and surface the conflict. If they ask for something ModelRig does not
support, say so directly and propose the closest supported alternative —
never improvise.
R6. The rig has a lifecycle: create it ONCE (module singleton for a server;
per-invocation for a one-shot job) and await rig.close() before exit, or
telemetry is lost and timers hang the process.
R7. Run the project's own tests before proposing any merge; show the diff.
R8. Every pipeline execution is a run. Wrap each execution in run.start /
run.scope, name one step() per model-call family, and artifact.save the work
products; tag subject and feature (inside a run context the SDK stamps run_id
AND step for you — a value you pass yourself always wins). subject is your
end-customer's opaque id — it powers the per-customer tenant statements on
/tenants; a tag named client/tenant is only a cost dimension. Acceptance: the run
appears on /runs with its steps and artifacts — a migration is not done until
one real execution shows a run with steps there.
0. PERSISTENT GROUNDING (do this before anything else):
a. Add to this repo's agent instructions (CLAUDE.md, AGENTS.md, .cursorrules —
whichever exists; create CLAUDE.md if none):
"Before writing ModelRig code, fetch https://modelrig.dev/llms.txt —
probed facts change; do not rely on memorized models or parameters."
b. If your tool supports MCP servers, register the ModelRig oracle and ASK it
rather than recalling (query_registry, get_leaderboard, explain_pricing,
get_call_notes):
{ "mcpServers": { "modelrig": { "command": "npx", "args": ["modelrig-oracle"] } } }
c. Keys: if MODELRIG_API_KEY is not set and the human wants hosted
telemetry/console sync, STOP and ask them to create an org and issue a
rig_sk_ key at https://app.modelrig.ai (about two minutes, human-only).
Without it, proceed in local-only mode; that is fully supported.
1. DISCOVERY-LITE (ask, don't infer, these three; the code answers the rest):
- What is this project, and is ModelRig replacing existing LLM calls or
starting fresh?
- Which providers does it (or should it) use, and which provider keys exist?
- Hosted telemetry (app.modelrig.ai) or local-only?
Then read https://modelrig.dev/routing-reliability.html (how a call actually
behaves and what is not built), https://modelrig.dev/migration-playbook.html
(this prompt follows its T0-T2 autonomy ladder and checkpoints) and
https://modelrig.dev/quickstart.html.
2. START AT THE BOTTOM OF THE LADDER, not at routes. T0: point an existing
OpenAI-compatible client at the ModelRig raw lane by swapping its base URL —
transport only, nothing about the request or model changes, telemetry lights
up. T1: run `modelrig observe` to see runs without migrating anything. T2
(routes, the steps below) is a per-call-site graduation you earn AFTER T0/T1
prove the plumbing — not the opening move.
3. ⏸ CHECKPOINT — fit and lane. Inventory the LLM call sites (generation calls
incl. JSON-extractor followups; embeddings and web-search tool calls are out
of scope). If the codebase already has a router layer, runtime-assembled
prompts, or env-resolved model choice, do NOT externalize it into routes —
migrate ONE naive call site as a reference route and integrate the router at
the rig.runRaw seam instead. Present this as a MIGRATION RECOMMENDATION and
get explicit approval before writing any code:
Use case: <one sentence> Lane: <T0 / T1 / T2 routes / runRaw seam>
Call sites: <count, by provider> Keys present: <names only>
Knobs to carry (each with why): <sampling / cache / grounding / tier / reasoning …>
Named behaviour changes (if any): <e.g. provider-strict JSON -> own extractor>
Open questions / assumptions: <what you inferred that the human should confirm>
Ready to proceed?
4. ⏸ CHECKPOINT — caching, grounding AND SAMPLING inventory, BEFORE writing any
route or seam call: search for provider caching (cachedContent, cache_control,
prompt_cache_key, cached-token usage fields), provider-native search/grounding,
and explicit sampling params (temperature, top_p, max_tokens/max_output_tokens).
Caching must be carried through (the cache handle on rig.run / rig.runRaw —
modelrig.dev/caching-lifecycle.html); grounding gaps must be REPORTED as named
behaviour changes; sampling must be PRESERVED by declaring policy.sampling
{ temperature, top_p, max_output_tokens } on the route (or the same fields on
runRaw) — a classifier at temperature 0.1 is a deliberate choice, so carry it
over exactly (absent, adapters use their own defaults; Gemini runs at 1.0).
Never migrate past any of the three silently.
5. Write routes (T2; install with the project's own package manager — pnpm/npm/
yarn; Node >= 20): modelrig/routes/<task>.yaml with the prompt in a FILE
(prompt.system: ./prompts/<task>.md, never inline), schema, candidates,
policy. Replace call sites with rig.run("<route>", { input, tags }) — tags is
REQUIRED and every tag key must be declared as a dimension in rig.yaml.
result.output arrives parsed AND schema-validated — delete your manual
JSON.parse / fence-stripping. Routes resolve from ./modelrig/routes relative
to the process CWD: in a monorepo put modelrig/ at the app root and run from
there, or set MODELRIG_ROUTES_DIR. A runRaw-only seam needs no routes at all —
construct with routesDir: null. Keep prompt and schema semantics unchanged; if
the original was JSON-mode + app-side validation, adding schema enforcement is
an UPGRADE, not identity — present that choice at the prove-it checkpoint, or
set policy.json: json_mode for closest identity. To load-check the bundle
WITHOUT any provider key, run `modelrig validate` — it structurally loads
rig.yaml + every route and reports per-route OK / config error.
6. ⏸ CHECKPOINT — candidates. Any model an adapter can reach may be a candidate —
probes gate CLAIMS, not serving. Pin what production resolves to today as
candidate #1, always. Probed models (llms.txt or the public registry) carry
evidence; an UNPROBED candidate is flagged as such and costs exactly this: no
json: native claim (the emulation ladder serves structure), no leaderboard/
bake-off priors, and possibly conservative envelope pricing until the next
sweep. Never silently substitute a probed sibling for the model production
runs; offer a probe request or a probed sibling as candidate #2. Present the
list for the human to ratify.
7. ⏸ CHECKPOINT — prove it: run the project's tests, show the diff and every
changed file, and after first traffic compare the route's cache-hit rate and
cost against the pre-migration baseline in the console (this metric needs a
REAL run — a single probe cannot produce it; say so if it must wait). A zero
hit rate on a route that cached before is a regression; say so.
8. ⏸ CHECKPOINT — see the run (R8). Open https://app.modelrig.ai/runs: your
execution is listed, expands to its steps in order, and each step carries its
inference and saved artifacts; /projects groups the same run by its tags. An
empty /runs after a real execution means it was not wrapped in run.start /
run.scope — fix that before calling the migration done. `modelrig status`
prints "runs recorded: N (last: …)" as the local mirror of the same fact.
9. ATTACH A GRADE, THEN PROVE A CHEAPER MODEL. Capture is not the whole
loop — a grade is what makes /coverage readiness and /insights findings
meaningful. A grade is your judgment — from your code, a human, or an AI — a
0–1 score with an optional comment. If the pipeline already has a quality
signal (a validator, a downstream accept/reject, a human QA gate), emit it as
a grade — score 1 on a clean pass, 0 on a rejection:
rig.grade({ kind: "inference", id: result.meta.inferenceId }, { score: 1, kind: "human" }); // one call
rig.grade({ kind: "run", id: run_id }, { score: 0, kind: "human", comment: "…" }); // whole run
The write is synchronous and local, mirrored by the exporter, and never blocks
or throws into your path. A route with ten attached grades reaches Rung 1 on
/coverage; a route with no schema to conform to (a prose output) gains proposal
confidence only this way. Then prove a cheaper model on YOUR own traffic, on the
lane that fits: a DECLARED route uses `modelrig bakeoff --route <route>
--replay-last N`; a RAW/run-based pipeline (a rig.runRaw seam, no routes)
promotes a real output to a golden (rig.artifacts.artifact.promoteToEval(handle,
{ task }), or the console's one-click "make golden") and then runs `modelrig
bakeoff --from-eval-cases <task> --variants default,<challenger>`. A bake-off
measures; it never switches what serves a route — that is still a human-approved
PR.
10. CONTENT CUSTODY IS ON BY DEFAULT — and Optimization on is the recommended
default. Set `capture: true` on your routes so ModelRig keeps that route's
telemetry rows and the bounded review samples your bake-offs replay — row-scoped
in your organization, visible and deletable in the console — repaid with community
priors and evidence on your own routes at the same flat 2%. Your organization's
content custody posture is managed by default, so a route that captures content
lands its prompts, model outputs and evidence in your managed store, scrubbed for
the PII/PHI you classify, yours to see and export in the console. Opting out is one
setting, not a default you have to escape: from your account settings (the console or
the `set_org_settings` MCP tool) set posture `off` to capture nothing, or posture
`metadata` to keep hashes only, or require `zero_retention` on a route to keep
nothing there.the agent will
- read the quickstart and route-bundle docs
- write route YAML into your repo and swap each direct LLM call site to rig.run()
- run your existing tests and report every file it changed — you review the diff
it won't
- create accounts or sign in anywhere — Step 0 is yours, and takes about two minutes
- touch your API keys — those stay in environment variables you set yourself
- change prompt or schema semantics — call sites are swapped, not rewritten
step 1 — install & connect
Add the package, then put a provider key for each candidate in an environment variableⓘ so it never lands in code:
shellpnpm add modelrig export OPENAI_API_KEY="your-key" # candidate: gpt-5.4-mini export GEMINI_API_KEY="your-key" # candidate: gemini-3.1-flash-lite export DEEPSEEK_API_KEY="your-key" # candidate: deepseek-chat export MODELRIG_API_KEY="rig_sk_…" # hosted sync (Step 0) — omit for local-only
One key is enough to start: a candidate whose provider has no key goes inert — the SDK warns at startup and skips it at dispatch, serving the next candidate. A route only fails when none of its candidates has a key, with a typed error naming the env vars to set.
modelrig 0.8.1 is on npm (Apache-2.0, Node >= 20). It builds a native module (better-sqlite3) on install, so give it a toolchain. The probe suite, the registry and the leaderboard stay public and reproducible. If the native build fails →
step 2 — define your first route
A route is a file of YAMLⓘ in your repo bundling the candidate models, a JSON Schemaⓘ for the output, and retry/budget policy. This one is complete as shown — or let your agent write it (first tab):
modelrig/routes/example.support_summarize.yamlroute: example.support_summarize version: 1 schema: ./schemas/support_summarize.schema.json candidates: # THE candidate set — nothing else can serve - provider: openai model: gpt-5.4-mini - provider: gemini model: gemini-3.1-flash-lite - provider: deepseek model: deepseek-chat require: [schema_conformant] prefer: [cost] prompt: system: ./prompts/support_summarize.system.md variables: [ticket, product, priorContext] policy: retries: { content_invalid: 2, network: 4, capacity_shed: 3 } timeout_ms: 60000 tier: flex json: nativekey reference: candidates · require · prefer · prompt · policy
step 3 — wrap the execution in a run, then call the task
Every pipeline execution is a run — that is the standard path, not an appendix. Open a run context around the execution, name a step per model-call family, and
artifact.savethe work products; tagsubjectandfeature.subjectis your end-customer's opaque id — it powers the per-customer tenant statements on/tenants; a tag namedclient/tenantis only a cost dimension. Inside the context the SDK auto-stamps bothrun_idand thestepkey for you (name the step withstepKeyfor a specific label — a value you pass always wins), so the run fills/runsand/projectswithout a hand-written id. Output is schema-validated no matter which candidate serves it. Full artifact API (lineage, grades, the reusable seam): the instrumentation guide.app codeimport { createRig, loadConfigFromEnv } from "modelrig"; const rig = createRig(loadConfigFromEnv()); // Every pipeline execution is a run. Wrap it once: run_id auto-stamps inside the // context, so this run fills /runs AND /projects with no hand-written id. Name // the step explicitly with stepKey — auto-stamped inside a run context, and a // value you pass always wins — and artifact.save the work products. await rig.artifacts.run.scope({ pipeline: "support", episodeKey: ticketId }, async () => { const result = await rig.run("example.support_summarize", { input: { ticket, product, priorContext }, tags: { subject: customer, feature: "summarize" }, // run_id auto-stamped stepKey: "summarize", // names this step on /runs }); rig.artifacts.artifact.save(result.output, { name: "summary", type: "step_output" }); // result.output — schema-validated JSON, whichever candidate served it // result.meta — served model, tokens, cost, validation status }); rig.close(); // stops the telemetry exporter, closes the local store — call at shutdownstep 4 — see it on /runs
Open app.modelrig.ai/runs: your run is listed, expands to its steps in order, and each step carries its inference and saved artifacts — then
/projectsgroups it by tag and/coverage//costsshow the rollups. Locally,modelrig statusprintsruns recorded: Nas the same fact. An empty/runsafter a real execution means it was not wrapped in a run — the one thing to fix.
Your routes are YAML in your repo. Leaving is a git rm. That's why you'll stay.
One flat 2%, your keys or ours. On your own keys, your first million requests each month are free. Nothing is chargeable until prepaid balances ship. The pricing approach.
BYOK is free during launch — no platform fee on usage routed on your own provider keys while launch terms are in effect. Launch terms end when we announce pricing. You get 60 days’ notice, and organizations created during launch keep launch terms for 6 months after the announcement.
How it all fits together
One map: routes live in your git, your app calls the task, and the probed registry is what routing decisions read. What the console holds is the lane you choose, per route — two lanes, and you pick per route:
| lane | what's kept | where it lives | what you get back |
|---|---|---|---|
| Pure router(capture off) | Nothing retained. Real, supported, selectable, and the honest answer for a ZDR-strict or data-residency-bound workload. | ||
| Optimization on(recommended) | Per-inference telemetry rows, and bounded review samples for the bake-offs you create. | Your organization, row-scoped by the database, not by our application code. | Community priors on your very first bake-off and RigIndex-grade evidence on your own routes. Same flat 2% either way — the corpus earns its place by being useful, not by being a discount. |
One flat 2%, your keys or ours. On your own keys, your first million requests each month are free. See it all in the console, delete it, export it, take your routes and go — they were always YAML in your git.
Where things stand
Current state, stated plainly. What works today: the gateway installs from npm (modelrig 0.8.1, Apache-2.0), accounts and keys are live at app.modelrig.ai, runs, steps and artifacts flow into the console's inspectable store (content custody is early access; metadata and lineage are live), and the probe suite, registry, leaderboard and contribution kits are public and reproducible. What is not out yet: prepaid balances, which are what puts the fee on rails — so nothing is chargeable and we would rather say that than have you find it out. Where this goes: the hosted path is the short one — sign up, issue a rig_sk_ key, point the exporter at us, and your routes stay YAML in your git. Self-host/BYOK stays the open lane: real, supported, open source, and the honest answer when a workload can't have a vendor holding payloads. On either one you pick a lane per route — Pure router keeps capture off and retains nothing; Optimization on keeps that route's telemetry rows and the review samples your bake-offs need, and repays them with community priors and evidence on your own routes — at the same flat 2% either way. How we charge, and what we keep.
Hosted setup docs land with billing.
The probe suite's verify command is fully standalone today — every published stat can be independently recomputed from its raw samples. See Reproduce our numbers in the public repo.
Prove it before you commit
We don't benchmark models in general. We bake them off on your traffic. Once your routes capture traffic, the bake-off replays your own logged requests against cheaper candidates:
modelrig bakeoff --route example.support_summarize --variants gemini-3.1-flash-lite,gpt-5.4-nano --replay-last 50
No declared route — a rig.runRaw seam, or a run-based pipeline? There is nothing to --replay-last. Promote a real /runs output to a golden (the console's one-click make golden, or promoteToEval) and bake off the eval cases instead:
modelrig bakeoff --from-eval-cases report.section --variants default,gemini-3.1-flash-lite
A bake-off scores conformance; a grade records whether the answer was actually good — your judgment, from your code, a human, or an AI, as a 0–1 score with an optional comment. Attach one with rig.grade from a signal you already have — a validator, a downstream accept/reject, a human QA gate:
rig.grade({ kind: "inference", id: result.meta.inferenceId }, { score: 1, kind: "human" }); // one call
rig.grade({ kind: "run", id: run_id }, { score: 0, kind: "human", comment: "…" }); // whole runThe write is synchronous and local, mirrored by the exporter, and never blocks your pipeline. Ten grades on a route reach Rung 1 on /coverage, and a route with no schema to conform to earns proposal confidence only this way. See the grade protocol.
Validity isn't accuracy: perfect JSON, wrong numbers. We gate on both.
Retention is a per-route choice between the two lanes above — same flat 2% on either. The two lanes.
More than the five steps
Once your runs are flowing, ModelRig also gives you per-customer tenant statements, shareable published receipts, a metadata-only analytics query API, provenance-first assembly & trust, and pairwise, regret-bounded bake-offs. The whole surface, one line each: how it fits →