ModelRig

Monitoring and improving the AI calls your product makes

See. Grade. Improve.

Every model call with its cost, latency and output, grouped into the runs your product actually does. Grade what's working, catch regressions and price changes the morning after, and swap to a cheaper or better model only when the evidence says so.

Start free — create your orgSee your runs first — no migration
app.modelrig.ai — runs
run · weekly-digest-demo4 steps · $0.0231 · 7.15 s
  1. fetch-contextvendor-a/fast$0.00110.64 s0.98
  2. extract-fieldsvendor-b/mini$0.00241.18 s0.95
  3. draft-summaryvendor-c/pro$0.01874.92 s0.81▲ conform rate down 9% since Tue · in Action queue
  4. check-citationsvendor-a/fast$0.00090.41 s0.97
Illustrative — your console shows your own traffic.

What ModelRig does

ModelRig enables better monitoring of the AI calls your product already makes: see every call and run, grade what's working, and improve cost and quality on the ones that matter.

  • See every call and run

    content custody · early access

    Every call with its model, tokens, cost, latency and failure class — grouped into runs with their steps, artifacts and lineage, and spend rolled up by model, route, tag and project.

    Runs, steps and artifacts are live in the console today — metadata, lineage and hashes; managed content custody is early access. The pitch is one sentence: when someone asks why the agent did that, you pull up the run and show them.

    How monitoring works →

  • Grade what's working

    Attach a 0–1 grade from your code, a human or a calibrated AI grader, and see the conform rate per route. A daily check flags quality drops in your Action queue the next morning.

    Set up a grader →

  • Improve cost and quality

    Bake off cheaper-or-better models on your replayed traffic, gated by confidence intervals, then promote the winner — versioned, reversible — behind declared candidates, typed fallback and spend stops. Never a silent swap.

    How bake-offs and routing work →

Point your coding agent at it

Paste one prompt or run modelrig init; your calls become routes in minutes.

Set up by your coding agent, in an environment where isolation is tested, not asserted.

prompt — paste into your coding agent
Migrate this project to ModelRig routes.
… (copy for the full 6-step prompt)

From first call to better model.

Continuous savings, confirmed performance.

  1. 01

    Observe first — no migration

    Point the exporter at us and see your runs before you change anything.

  2. 02

    Replay your real traffic against challengers

    Shadow bake-offs on logged requests; production never sees the test.

  3. 03

    Promote the winner — never silently

    Promote it in ModelRig — versioned, reversible — or merge it as a PR. Never a silent swap.

  4. 04

    The test re-runs itself

    A price or model change triggers a suggested re-test — last quarter's decision doesn't quietly rot.

You found out about the bill from the invoice. You found out about the regression from a customer. Both facts were knowable the moment they happened — if every call carried its cost, its task, its conformance check, and any grade you attach. Ours do.

How monitoring works.

We also probe every model we route to in the open → Leaderboard

We built a 16-step financial research pipeline first. What we couldn't see was the part that hurt: which step cost what, and which model had quietly started returning worse answers. Testing cheaper models came later — first we needed to see the calls. We built ModelRig because we needed it.

Our promises. No leap of faith.

  • Never a silent swap — automated changes (opt-in) are proven before they apply; promote a change in ModelRig (console or MCP; versioned, reversible) or merge it as a PR if you keep routes in git.
  • Your data never leaves without you.
  • Everything we hold, you can see and export.
  • One flat fee — 2% of what you run through us at list prices, plus the standard Stripe cost on the charge (the processor's, not ours). On your own keys the first 1,000,000 requests each month are free, then 2%; managed keys are list price plus 2%, drawn from your prepaid balance.
  • Leave anytime — the SDK is Apache-2.0 on npm.

All hosted data lives on SOC 2 Type II–audited infrastructure, encrypted in transit and at rest, row-scoped to your organization by the database itself. Security →

Every model decision, defensible.

From gambling on model claims to operating on proof — costs governed, conformance enforced, and when the bill comes, you defend it with a chart.

Near-optimal, measured — not best, asserted.

app.modelrig.ai — cost by route, $ per 1,000 calls
  • draft-summary$18.70$7.90
  • extract-fields$2.40$2.40
  • classify-ticket$4.10$1.60
Illustrative — your console shows your own traffic.

One flat 2% at list prices, your keys or ours — on your own keys, the first 1,000,000 requests each month are free.

Start free — create your orgSee your runs first — no migration