Track B — Generalist-IDM v1
EC's core sales claim is that IDM-inferred actions are the inferior substitute ground-truth capture beats (Guido to GDM; Klindt call-sheet must-get #4: does the premium beat "passive video run through an IDM"?). Track B therefore builds the very baseline our premium SKU must out-price — which is coherent only if argued explicitly: EC owns both tiers and publishes the measured delta. The cross-game adaptation curve is the pricing evidence for the ground-truth premium, not just a product. Never market the IDM as equivalent to captured ground truth; external naming: "action-inference model" — "labeler" is banned register in EC external copy.
The IDM is the highest-leverage artifact the corpus can produce: a model that watches gameplay video and recovers the inputs that caused it. VPT's IDM (trained on 1,962 labeled hours) unlocked 70,000 scraped hours — 35:1 leverage; D2E's "Generalist-IDM" replicated the pattern across 31 games (Technical Evidence §A). Selling a labeler is also a far cleaner IP posture than selling footage (Products & Publishers — Product 2).
Why this is the concentrated form of the corpus
Every labeled hour we hold does double duty: it trains policies/world models directly, and it teaches the IDM to label video we don't hold. The IDM is therefore the mechanism by which a modest consented corpus prices against archives thousands of times larger — the buyer applies our labeler to their scraped-video problem, and our data's value rides on every hour they process. It also compounds internally: every directed campaign adds labeled hours in new regimes → better IDM → cheaper labeling for every future campaign.
The recipe (v1)
- Architecture: non-causal (bidirectional) video-to-action model — the VPT insight: seeing future frames makes action inference ~2 orders of magnitude more data-efficient than causal prediction. v1 follows the VPT/D2E shape rather than innovating; the innovation budget goes to the data, not the architecture.
- Targets: keypress streams (per-key logits), mouse displacement (regression or quantized bins — VPT's camera quantizer is the reference), controller axes where present. Loss-mask input regimes per title.
- Data prep: the labeled corpus across the Phase-1 title set (0.8), spec-conformant clips only; unified event format across titles (D2E's OWA lesson — one schema, per-title adapters); train/val split by contributor, never by clip (annotator-artifact hygiene).
- Curriculum: per-title heads → shared trunk fine-tuning, evaluated both in-title and cross-title.
The milestone that matters: cross-game transfer
In-distribution accuracy is table stakes (VPT's bar: 90.6% keypress accuracy, 0.97 mouse R² on held-out contractor data). The research question — and the moat question — is zero/few-shot labeling of an unseen title:
| Protocol step | Detail |
|---|---|
| 1 · Hold out a full title | From the Phase-1 set — same holdout as the Founding Experiment's OOD slice, for coherence |
| 2 · Zero-shot eval | IDM labels held-out-title video; score against ground-truth inputs we captured but withheld |
| 3 · Few-shot eval | +1 h, +10 h of labeled adaptation data; plot the adaptation curve — this curve is the pricing model for "label a new game" engagements |
| 4 · Downstream proof | Pseudo-label a scraped-video set for the held-out title, train a small policy/world-model probe, compare to training on ground truth (VPT's downstream test, miniaturized) |
Competitive targets. Beat D2E's Generalist-IDM on KBM titles specifically — high-DPI mouse aim is where overlay-parsing (NitroGen) and pixel-IDMs are weakest, and where our ms-aligned ground truth is most differentiated (Competition & Moat — input alignment erodes except KBM precision). Publish the comparison.
Deliverable — the leverage demo
The Product 2 proof artifact, in one sentence a buyer's ML lead can verify:
"N labeled Playroll hours train a labeler that converts M hours of your scraped video into training data of quality X — here is the downstream delta on a public benchmark."
Packaging: model card (titles, input regimes, accuracy per regime, known failure modes), the adaptation-curve chart, and the downstream probe result. Delivered as hosted inference or licensed weights — never as our underlying labeled corpus.
Risks & watch-items
- D2E's zero-shot transfer improving erodes the "new game = new labeled data" moat — track it; our answer is KBM precision + the adaptation curve's few-shot regime + consented provenance of the seed corpus.
- Buyer misuse: a licensed IDM applied to scraped video creates their copyright exposure, not ours — but contract language must say so explicitly (risk allocation, Risk & Compliance #8).
- Overfitting to operator style in early corpora (operators ≠ organic players) — mix fleet data in as soon as Phase-0 gates allow; the naturalness slice doubles as the detector.
Gate
Track B v1 is done when: in-title accuracy meets the VPT bar on ≥3 titles; the cross-game protocol is run with the adaptation curve published internally; and the leverage demo exists in buyer-presentable form.