# FACTS: the claims ledger

Every number and factual claim in `README.md` and on the `/judge` page, with the command or file that
produces it. Generated by `owin facts write` from `owin/facts.py`; `tests/test_facts.py` recomputes each
value keyless from committed files and fails if any drifts. Run `owin facts check` to see the same.

| id | claim | value | source |
|---|---|---|---|
| `sel.auto` | Answers Owin filled on its own on split v3fresh (every successful reading agrees, at least 2) | **27/33** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.correct` | Correct among the auto-filled answers | **26/27** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.confirm` | Answers handed to a person to review | **6** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.majority` | Needs-review cells where the majority reading would have been right | **2/6** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.sentence` | Correct auto-filled answers that assert a value with the vendor's verbatim sentence | **18** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.notstated` | Correct auto-filled answers that are 'not stated' (the page says nothing on that field) | **8** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.majority_all` | All 33 cells: where a majority vote would have been right (what a majority-vote system ships) | **28/33** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.vendors` | AI vendors in v3fresh positives | **7** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.vendors_new` | Vendor families in v3fresh that appear nowhere in the older corpus (Google Cloud is not new) | **6** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.items` | Features (vendor pages) in v3fresh positives | **11** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.pages` | Distinct vendor pages behind those 11 features | **8** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.swing` | Whole-item points one v3fresh item moves (100 / items) | **9** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.miss` | The one confident miss (self-disclosed label error, kept per pre-registration) | **openrouter-inference-zdr retention_days** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.neg` | Negative-control cells auto-answered correctly (page says nothing; Owin answers 'not stated') | **6/6** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.ref.v3old` | Reference, NOT clean: v3 on the older items, auto / correct | **80/90, 77/80** | `owin eval selective --corpus datause --split v3fresh` |
| `sel.ref.v2old` | Reference, NOT clean: v2 on the older items, auto / correct | **77/90, 72/77** | `owin eval selective --corpus datause --split v3fresh` |
| `v3.items` | Pre-registered v3 on v3fresh: items with all three fields exact (majority of runs) | **7/11** | `owin eval score --corpus datause --brief` |
| `v3.neg` | v3fresh negatives correctly left unanswered | **2/2** | `owin eval score --corpus datause --brief` |
| `v3.base` | Keyword baseline on v3fresh, items exact | **4/11** | `owin eval score --corpus datause --brief` |
| `v3.base.neg` | Keyword baseline on v3fresh, negatives correctly left unanswered | **0/2** | `owin eval score --corpus datause --brief` |
| `v3.passed` | v3 passed its pre-registered 80% bar | **false** | `owin eval score --corpus datause --brief` |
| `v2.old` | Prompt v2 on the older 30 features (after one disclosed relabel) | **23/30** | `eval/datause/v2-score.json` |
| `v1.prereg` | Pre-registered prompt v1, first 24 features (failed) | **16/24** | `eval/datause/preregistered-v1-score.json` |
| `v3.calls` | Token Factory calls in the v3fresh run (11 features + 2 negatives, 3 readings each) | **39** | `owin eval score --corpus datause --brief` |
| `v3.cost.total` | Token Factory spend for the v3fresh run (USD) | **0.115** | `owin eval score --corpus datause --brief` |
| `price.super` | Token Factory list price for Nemotron Super, USD per M input / output tokens | **0.30 / 0.90** | `owin/tokenfactory.py` |
| `cost.call` | Mean Token Factory cost per reading, v3fresh (USD) | **0.0029** | `owin eval score --corpus datause --brief` |
| `cost.feature` | Cost to answer one endpoint (3 readings, USD) | **0.0088** | `owin eval score --corpus datause --brief` |
| `lat.mean` | Mean latency per reading, exactly-right runs (s) | **8.81** | `owin eval score --corpus datause --brief` |
| `lat.p95` | p95 latency per reading (s) | **15.86** | `owin eval score --corpus datause --brief` |
| `model` | Model | **nvidia/nemotron-3-super-120b-a12b** | `owin eval score --corpus datause --brief` |
| `demo.q03` | Demo: answer 3 (ZDR) is not true for /api/crm-sync | **not_true** | `owin pack build --format brief` |
| `demo.quote` | Demo: the vendor sentence behind answer 3 | **The MCP connector is not covered by ZDR arrangements.** | `owin pack build --format brief` |
| `demo.q05` | Demo: answer 5 is Outdated (badge code `stale`) | **stale** | `owin pack build --format brief` |
| `demo.list` | OpenAI sub-processor list dates (before, after) | **June 2, 2026 -> October 14, 2026** | `owin pack build --format brief` |
| `demo.cloudflare` | OpenAI CHANGED Cloudflare's entry (not added) | **changed** | `owin pack build --format brief` |
| `demo.oai` | OpenAI ADDED OAI International | **added** | `owin pack build --format brief` |
| `demo.swap` | Change one input: /api/crm-sync -> Anthropic Messages makes answer 3 Verified (badge code `still_true`) | **still_true** | `owin pack build --format brief --set /api/crm-sync=anthropic-messages` |
| `demo.thin` | Thin evidence: every vendor field for /api/crm-sync is Not stated (verdict code `held`), none invented | **held,held,held,held** | `owin pack build --format brief --set /api/crm-sync=thin-anthropic-rate-limits` |
| `demo.confirm` | Needs review example (verdict code `confirm`): answer 5's /api/embed cell when it calls OpenAI Responses (the v3 readings of OpenAI's page split 2 to 1 on third-party access; answer 5 itself stays Outdated because OpenAI's list changed) | **confirm, 2/3** | `owin pack build --format brief --set /api/embed=openai-responses` |
| `demo.v3` | The report's readings use the headline's prompt version (every registry entry and live check) | **v3** | `owin pack registry` |
| `demo.region` | Region never decides: answer 4 is Not stated (badge code `held_unspecified`) | **held_unspecified** | `owin pack build --format brief` |
| `dpa.items` | Earlier subject (objection windows): DPAs exact, tuned + held-out | **10/10 + 7/7** | `owin eval score --brief` |
| `dpa.runs` | Earlier subject: readings exact on all five fields, tuned + held-out | **51/51** | `owin eval score --brief` |
| `dpa.neg` | Earlier subject: thin-evidence controls correctly left unanswered | **6/6** | `owin eval score --brief` |
| `model.ultra` | Nemotron Ultra on the same 10 labelled DPAs, 3 readings each: DPAs exact by majority / readings exact | **10/10 / 30/30** | `owin eval score --brief --file eval/runs/20261028T233749-Nemotron-3-Ultra-550b-a55b.json` |
| `model.super` | Nemotron Super on those 10 DPAs: DPAs exact by majority / readings exact | **10/10 / 30/30** | `owin eval score --brief` |
| `model.cost` | Cost per reading on those 10 DPAs, Super vs Ultra (USD; Ultra / Super) | **0.002 vs 0.0077 (3.9x)** | `eval/super-vs-ultra.json` |
| `think.off` | Nemotron reasoning OFF, same split, prompt and rule: auto-answered / right | **28/33, 21/28** | `eval/datause/runs/20261029T105717-v3fresh-nemotron-3-super-120b-a12b-nothink.json` |
| `think.cost` | Cost per reading, reasoning off vs on (USD) | **0.0014 vs 0.0029** | `eval/datause/runs/20261029T105717-v3fresh-nemotron-3-super-120b-a12b-nothink.json` |
| `think.lat` | Mean latency per reading, all readings, reasoning off vs on (s) | **3.0 vs 9.7** | `eval/datause/runs/20261029T105717-v3fresh-nemotron-3-super-120b-a12b-nothink.json` |
| `think.tokens` | Output tokens over the 39 readings, reasoning off vs on | **6292 vs 72524** | `eval/datause/runs/20261029T105717-v3fresh-nemotron-3-super-120b-a12b-nothink.json` |
| `think.live` | Reasoning setting every reading uses (the off arm adds 6 confident misses) | **on** | `eval/thinking-off-vs-on.json` |
| `ladder.ultra` | Nemotron Ultra x3 alone, same split, prompt and rule: auto-answered, right, sent to a person | **30/33, 29/30, 3** | `owin eval ladder --corpus datause --brief` |
| `ladder.cost` | Cost per reading, Ultra vs Super (USD) | **0.0136 vs 0.0029** | `owin eval ladder --corpus datause --brief` |
| `ladder.esc` | Pre-registered escalation (Super x3; Ultra x3 only where Super's readings split): auto-answered, right, sent to a person | **31/33, 30/31, 2** | `owin eval ladder --corpus datause --brief` |
| `ladder.esc.k` | Of the 6 cells Super sends to a person: answered by Ultra / right | **4 / 4** | `owin eval ladder --corpus datause --brief` |
| `ladder.esc.cost` | Ultra readings per escalated feature (3 readings, USD) | **0.0352** | `owin eval ladder --corpus datause --brief` |
| `ladder.esc.adopted` | The escalation met its pre-registered bar (fewer than 6 to a person, no new confident miss) | **true** | `owin eval ladder --corpus datause --brief` |
| `ladder.retract` | Pre-registered retraction triggered: Ultra alone gets more cells right than Super alone | **true** | `owin eval ladder --corpus datause --brief` |
| `ladder.nano` | Nemotron Nano 30B-A3B x3 alone: auto-answered, right, sent to a person | **25/33, 24/25, 8** | `owin eval ladder --corpus datause --brief` |
| `ladder.lightning` | Nemotron 3.5 Lightning x3 alone: auto-answered, right, sent to a person | **0/33, 0/0, 33** | `owin eval ladder --corpus datause --brief` |
| `ladder.lightning.err` | Lightning readings that ran out the 8,000-token budget reasoning (errored, abstained), of 39 | **32** | `owin eval ladder --corpus datause --brief` |
| `ladder.nano.lat` | Mean latency per reading, Nano vs Super (s) | **44.8 vs 9.7** | `owin eval ladder --corpus datause --brief` |
| `ladder.cascade` | Pre-registered Nano-first cascade (reported, decides nothing): auto-answered, right, cost per feature vs Super's (USD) | **30/33, 29/30, 0.0091 vs 0.0086** | `owin eval ladder --corpus datause --brief` |
| `sweep.vendors` | Vendors in the sub-processor sweep index | **29** | `data/index/vendors.json` |
| `sweep.live` | One real sweep with the Tavily key (2026-10-29): vendors swept / lists fetched live | **29 / 22** | `eval/sweeps/sweep-3-live-20261029T033329Z.json` |
| `job.readings` | One real `owin job daily` (2026-10-29): Nemotron readings made / errored (abstained) / cost (USD) | **12 / 0 / 0.0651** | `eval/sweeps/job-daily-20261029T033329Z.json` |

## Qualitative claims and where they are enforced

| claim | enforced by |
|---|---|
| Code, not the model, decides every answer; quotes and stated text are never read by a rule | `tests/test_a3_pack.py` ::test_model_and_stated_prose_cannot_change_any_badge |
| An answer is auto-filled only when at least 2 readings succeeded and every successful reading agrees (an errored reading abstains); otherwise Needs review | `tests/test_selective.py` |
| Every asserted value carries a verbatim sentence from the vendor's page | `owin/datause/extract.py::validate`, `tests/test_a1_datause.py` |
| Region text never decides an answer | `tests/test_a3_pack.py::test_region_is_never_computed_from_vendor_free_text` |
| Tavily `extract` fetches the evidence pages; an older copy is set aside, never used | `data/datause/observations.json`, `tests/test_ui.py::test_watch_holds_the_stale_tavily_copy_of_the_openai_list` |
| Lumen Notes and its settings are fictional; vendor sentences, links and fetch times are real | `fixtures/demo/stack.json` `_label` |
| Copy rules: no compliance claims, Nemotron described as open-weight, not legal advice | `tests/test_facts.py::test_copy_rules` |
| Every value here checks on a fresh clone with no key and no page text: the arithmetic recomputes from recorded runs; the keyword baseline is replayed from its recorded output per page sha256 (equal to the recomputed output when the pages are present); the verbatim gate uses its recorded results. CI runs the suite and this ledger on every push | `.github/workflows/proof.yml`, `tests/test_a1_datause.py::test_recorded_baseline_equals_the_recomputed_baseline` |
| Every recorded reading names the same page bytes the labels were written from (the sha256 each reading recorded equals the labelled capture's) | `tests/test_a1_datause.py::test_every_recorded_reading_was_made_on_the_labelled_bytes` |
| The Nemotron ladder's arms, the Ultra escalation rule, its bar and the retraction condition were committed before any arm ran; every arm's run_at is later | `eval/datause/escalation-preregistration.json`, `tests/test_ladder.py::test_every_arm_ran_after_the_preregistration_was_committed` |
| The escalation never re-opens an answer Super gave on its own and only uses a value Super's readings gave | `tests/test_ladder.py` |
| A live Nemotron re-reading on `POST /api/recheck` can only hand fields to a person, in that response only | `tests/test_recheck.py::test_property_live_readings_only_ever_move_fields_to_confirm` |
| The README number badge is computed from the recomputed headline | `owin/facts.py::badge_url`, `tests/test_facts.py::test_number_badge_is_the_recomputed_headline` |