Economic Integrity Observatory · seed readings
Pricing pages quote cost per token.
Pipelines pay per attempt.
We measure what production pipelines actually pay for the work they actually use: attempt-level ledgers, Inference Amplification (IAF), Economic Contamination (ECI), and degradation accounting. Every reading below is dated, carries its sample size, and traces to published raw data. Nothing is projected.

New to this research? Start with the plain-language guide.
Economic Integrity in AI: The Hidden Cost Behind the Answer — read it online or download the PDF.
Current readings
DeepSeek (direct API) · 2026-08-10/11 · n=10 reps per cell · 1,244 attempts serving 1,046 jobs · ledger $3.69 / billed $3.07
Kimi (Moonshot AI, direct API) · 2026-08-13 · 562 attempts serving 455 jobs · ledger $33.99
Grok (xAI, direct API) · 2026-08-13 · 455 attempts serving 455 jobs · ledger $14.26
physical attempts ÷ logical jobs
How many times the pipeline actually hit the model for each unit of work it was asked to do. 1.000 is clean: every job took exactly one attempt. An IAF of 1.70 means the pipeline paid for 70% more invocations than the work it delivered — retries and re-asks that never appear on any quality scoreboard.
total spend ÷ spend on used output
How much of the bill bought output that actually shipped. 1.000 is clean: every dollar bought used work. An ECI of 1.96 means the pipeline paid nearly twice what the used output cost — the rest went to attempts whose output was discarded, mostly billed reasoning tokens.
Layers are never pooled: the agent layer and the adversarial validation layer fail differently and pay differently, so every cell below reports them separately.
| Platform | Configuration | Agent layer | Adversarial layer | Spend | Discarded | Attempts / jobs | Starved | ||
|---|---|---|---|---|---|---|---|---|---|
| IAF | ECI | IAF | ECI | ||||||
| DeepSeekdirect API2026-08-10/11n=10 / cell | Proactive headroombudget headroom on, retry off | 1.000 | 1.000 | 1.000 | 1.003 | $0.87 | $0.0010.1% of spend | 246 / 246 | 2 |
| Bundleheadroom on, retry adds headroom | 1.016 | 1.029 | 1.000 | 1.000 | $0.86 | $0.0192.3% of spend | 236 / 234 | 2 | |
| Reactive retryheadroom off, retry at raised budget | 1.087 | 1.023 | 1.698 | 1.576 | $0.93 | $0.12012.9% of spend | 339 / 245 | 95 | |
| Pure retryheadroom off, retry at unchanged budget | 1.109 | 1.046 | 1.456 | 1.957 | $1.03 | $0.21520.8% of spend | 423 / 321 | 104 | |
| KimiMoonshot AI · direct APIkimi-k32026-08-13 | Bundleheadroom on, retry adds headroom · n=10 | 1.008 | 1.003 | 1.011 | 1.022 | $16.51 | $0.0890.5% of spend | 226 / 224 | 4 |
| Reactive retryheadroom off, retry at raised budget · n=10 | 1.128 | 1.051 | 1.840 | 1.533 | $17.48 | $2.07511.9% of spend | 336 / 231 | 106 | |
| GrokxAI · direct APIgrok-4.62026-08-13 | Bundleproduction configuration · n=10 | 1.000 | 1.000 | 1.000 | 1.000 | $7.16 | $0.0000.0% of spend | 226 / 226 | 0 |
| Headroom lever offablation lever inert on xAI (see note) · n=10 | 1.000 | 1.000 | 1.000 | 1.000 | $7.10 | $0.0000.0% of spend | 229 / 229 | 0 | |
| In the pipe — awaiting first readings | |||||||||
| LlamaMeta · via serving providers | Provider-vs-provider comparison plannedOpen-Weight Index | — | |||||||
| QwenAlibaba | QueuedOpen-Weight Index | — | |||||||
| GPT-5 familyOpenAI | Queued, incl. reasoning-effort ladderFull Observatory | — | |||||||
| ClaudeAnthropic · Haiku, Sonnet | QueuedFull Observatory | — | |||||||
| GeminiGoogle | QueuedFull Observatory | — | |||||||
Queued platforms publish no numbers until their first measurement campaign completes; every future reading carries its own date and sample size. Spend and Discarded are ledger-computed dollars at each provider's list price (DeepSeek $0.435/$0.87 per M in/out; Kimi k3 $3.00/$15.00 per M; Grok 4.6 $2.00/$6.00 per M) with cache-hit input billed at the full input rate (an upper bound), summed across both layers. Token accounting differs by provider and the ledger follows each provider's own reporting: DeepSeek and Moonshot fold reasoning tokens into the billed output count, while xAI reports reasoning as a separate field — Grok's ledger dollars price those reasoning tokens at the output rate, and xAI's own billing console confirms that basis to the cent (console 2026-08-13: reasoning $7.01, completion $6.40; ledger-computed $7.01 and $6.40 from 1,168,846 and 1,067,247 tokens at $6/M). The one divergence is input: the console billed 392.7K cached prompt tokens at a cache discount the ledger books at full rate, so the ledger total overstates the account-billed total by about 4%, in the conservative direction. On 2026-08-13, 46% of Grok's billed output tokens at the agent layer — and 86% at the adversarial layer — were reasoning the end user never sees. DeepSeek's account-billed total, with the provider's cache discount applied, was $3.07 against $3.69 ledger-computed; Kimi's ledger-computed $33.99 reconciles with the account drain to within a few percent (no cache hits were recorded). The two bases are never mixed in one figure. One overnight repetition pair of Kimi's reactive cell hit an account-balance suspension mid-run; that attempt is quarantined as a budget artifact and the pair was re-run cleanly after refunding — the n=10 above contains only funded reps.
- DeepSeek Proactive headroom: Prevention: 1 starved first attempt in 127 agent-layer jobs. IAF = 1.000 by direct enumeration.
- DeepSeek Bundle: The production configuration. Near-clean on both layers this night.
- DeepSeek Reactive retry: 83 starved first attempts in 119 adversarial-layer jobs; retries repaired the scoreboard, not the invoice.
- DeepSeek Pure retry: Worst cell: 48.9% of the adversarial layer’s spend bought attempts whose output was discarded.
- Kimi Bundle: The production configuration on a second provider: 3 starved first attempts in 130 agent-layer jobs, 0.5% of spend discarded. The headroom mechanism is not DeepSeek-specific.
- Kimi Reactive retry: 90 starved first attempts in 106 adversarial-layer jobs with headroom ablated — the same failure shape as DeepSeek. Zero first-attempt-clean repetitions; every clean result was recovered by retry.
- Grok Bundle: First frontier-lab reading: 226 attempts, 226 jobs, zero starvation, zero discarded spend. Every repetition first-attempt clean.
- Grok Headroom lever off: The headroom ablation that starved DeepSeek and Kimi is inert on xAI by API design: Grok draws reasoning from a separate budget, so hidden reasoning cannot consume the completion allowance. In this cell, 76 attempts spent more reasoning tokens than their entire completion cap and still returned content — on providers that fold reasoning into the completion budget, every one of those would have been a starvation candidate. Both Grok cells therefore measure the production condition; read them as one n=20 replication, not an ablation contrast.
Degradation accounting
same runs · DeepSeek (direct API) 2026-08-10/11 · Kimi (Moonshot AI, direct API) 2026-08-13
The companion instrument from study 1: a run-level record of whether the AI actually did its own work. Every repetition is classified by how its result was produced — not just whether it was right. All four configurations reach an equivalent scoreboard; this board shows the different paths they took to get there.
| Platform | Configuration | Repetition outcomes | First-attempt clean | Recovered clean | Degraded |
|---|---|---|---|---|---|
| DeepSeekdirect API2026-08-10/11 | Proactive headroombudget headroom on, retry off | 9 / 10 | 0 / 10 | 1 / 10 | |
| Bundleheadroom on, retry adds headroom | 8 / 10 | 2 / 10 | 0 / 10 | ||
| Reactive retryheadroom off, retry at raised budget | 0 / 10 | 9 / 10 | 1 / 10 | ||
| Pure retryheadroom off, retry at unchanged budget | 0 / 10 | 9 / 10 | 1 / 10 | ||
| KimiMoonshot AI · direct API2026-08-13 | Bundleheadroom on, retry adds headroom · n=10 | 8 / 10 | 1 / 10 | 1 / 10 | |
| Reactive retryheadroom off, retry at raised budget · n=10 | 0 / 10 | 9 / 10 | 1 / 10 | ||
| GrokxAI · direct API2026-08-13 | Bundleproduction configuration · n=10 | 10 / 10 | 0 / 10 | 0 / 10 | |
| Headroom lever offablation lever inert on xAI (see note) · n=10 | 10 / 10 | 0 / 10 | 0 / 10 |
On DeepSeek, the two prevention configurations produced 9 of 10 and 8 of 10 first-attempt-clean repetitions while both retry-only configurations produced zero: every clean result was recovered, not clean on the first pass. Kimi repeats the shape — 8 of 10 first-attempt-clean under the production bundle, 0 of 10 with headroom ablated. A distinction no output-quality scoreboard records, and the reason the economics above diverge while the scoreboards agree. Grok is the counterpoint that sharpens the mechanism: 20 of 20 repetitions first-attempt clean, because xAI's API gives reasoning its own token budget and the starvation pathway the other two providers exhibit — hidden reasoning consuming the completion allowance — cannot occur there by construction. The failure mode this observatory measures is a property of how a provider's API meters reasoning, not a property of reasoning models as such.
Why one reading is never enough
Rescue effectiveness is non-stationary. The same-budget re-ask, on an identical configuration and the same provider:
Three dated readings of the same-budget re-ask on an identical configuration: near-total rescue, then collapse, then collapse again with three times the starvation events. Retry-based cost forecasts inherit this volatility; prevention-based forecasts do not. This is why the observatory takes dated, repeated readings instead of publishing a single score.
Methodology & provenance
Readings are produced by a transport-boundary attempt ledger: one record per physical model invocation, including the attempts that retry logic normally erases from the record — token counts with reasoning and prompt-cache splits, budgets, outcomes, latency. The instrument and its methodology are described in two peer-inspectable preprints, and every number above is independently recomputable from the published raw ledgers and result files.
What this is, and is not: the observatory measures an information asymmetry the market does not price — pricing pages quote cost per token, while pipelines pay per attempt, and the two can diverge widely. It is not an allegation of provider misconduct. In every reading published here, the providers reported token usage honestly and billed exactly what their pricing pages state; the token budgets that produce starved attempts are set by the calling client, not the provider. The gap these boards expose is between sticker economics and effective economics — a gap no quality scoreboard records — and closing it takes attempt-level measurement, which is what this instrument exists to do.
- Study 1 — When the Model Isn't the Problem: Degradation Accounting in Fallback-Enabled Production LLM Pipelines DOI 10.5281/zenodo.21903064
- Study 2 — Economic Integrity in Production LLM Pipelines: Measuring Attempt Amplification, Discarded Reasoning, and Retry Cost DOI 10.5281/zenodo.21903218
- Raw per-attempt ledgers, result files, and compiled report (public repository)
Patent pending. U.S. provisional patent applications have been filed covering the measurement instrumentation.
Fund the observatory
These seed readings come from the published studies — one provider, measured to paper-grade rigor. The instrument is built; what scale requires is measurement budget. We publish the observatory's own spend ledger, so every sponsored dollar is itself accounted for at the attempt level.
DeepSeek + Kimi on the board, dated readings with sample sizes stated, raw ledgers published with every update.
Weekly readings across 5–6 open-weight models (adds Llama, Qwen, and provider-vs-provider comparisons of identical weights on different serving stacks).
Nightly readings, frontier labs on the board, reasoning-effort ladders, and public API access to the full time series.
