Learning, Self-Play and Tuning

Status: Design proposal v1 · Subsystem: AI — the learning pipeline · Angle: how the AI gets strong, how we know it did, and what it costs.

This document covers one slice of the AI opponent system: how its numbers are chosen. It does not design the agent — the candidate generator, the three planning layers, the persona system and the difficulty presentation belong to the companion AI proposals. It designs the machinery that turns a plausible agent into a strong one, measures whether it worked, and refuses to ship it when it did not.

The project owner has named advanced AI opponents as a headline feature. The failure mode for a headline feature is over-promising, and this is the document where over-promising is easiest, because the vocabulary of machine learning makes everything sound inevitable. So the organising discipline here is a single line drawn through the middle:

Everything above the line is engineering: known cost, known schedule, known outcome shape. Everything below it is a research bet, where the honest answer is "we do not know, and here is the cheapest experiment that would tell us."

Levels 1 and 2 and the Rehearsal search of §7 are above the line. AlphaZero-style policy learning is below it. Human-log learning is above the line for what it is actually good for and below it for what people usually hope it is good for. I will say which every time.

0. Summary in one page

Three numbers to carry away, each derived below and each resting on estimates I flag as estimates:

• A promotion-grade match (SPRT, 8-Elo bound) is about 11,000 games — roughly 8 hours on one 16-core desktop, or $4 of spot cloud. Compute is not the constraint on this project.
• Resolving a 5-Elo improvement costs about 28,000 games. Noise, not compute, is the constraint, and every design decision in §3 exists to fight it.
• A quantised integer evaluation network costs about 1 nanosecond per multiply-accumulate on reference hardware (measured; §6.5). A 5,000-MAC network is 5 µs, so a 100 ms turn budget buys 20,000 evaluations. Inference is not the constraint either.

1. What this design gives us, priced

The brief names four properties of the design that should be exploited rather than rediscovered. They are real, and three of them are worth more than they look. But each has a price attached that is usually left off, and the pricing is the useful part.

1. The command abstraction collapses the branching factor. This is the big one, and it is worth being precise about how big. Without 13-command.md, an AI turn is an assignment of an order to each of ~150 units, each with a destination on a map of tens of thousands of tiles. With it, an AI turn is an assignment over command objects: ten to thirteen Postings (CM-1150), a handful of Formations and Groups (CM-850, CM-860), and the Requisition tables that turn force structure into build orders (CM-1240). The Posture vocabulary is exactly six (CM-330) and the Sanctions are four dials with named presets (CM-750).

The honest arithmetic: a naive per-unit AI choosing among ~8 order shapes for 150 units faces 8¹⁵⁰ joint assignments. An AI choosing among 6 Postures × 3 Sanction presets for 12 command objects faces 18¹² ≈ 10¹⁵ — still astronomical, but now it is a space you can search greedily and locally, because the objects are semantically separable: what the Northern Posting does is nearly independent of what the Home Posting does, given the map. That is what actually matters. The abstraction does not make the space small; it makes it decomposable, and decomposability is what search needs.

The price: the AI's ceiling is now the vocabulary's ceiling. If a Posture-plus-Sanction tuple cannot express a good plan, no amount of learning will find it, because the plan is not in the space. 13-command.md's own open question 5 flags exactly this risk for humans; it binds the AI identically and more tightly, because a human can drop to hand-ordering at the decisive point and the AI, by CM-050 symmetry, may too — but then it is back in the 8¹⁵⁰ space with none of the human's intuition.

2. An AI turn is a clean bounded function. Orders in, order set out, evaluated by the Cascade against a frozen initiative order (turn-model.md §3.1). No mid-turn prompts, no interleaved opponent input, no partial state. This is what makes §7's Rehearsal search possible at all, and it is the property I would fight hardest to keep.

The price: the function's input is a fog-filtered view, and the output is committed before the opponent's is seen. The AI therefore faces a simultaneous-move game at the turn level, which is the single largest technical obstacle to any principled search (§8.1). The turn model bought wall-clock time from humans by paying in game-theoretic tractability, and the AI pays that bill.

3. The core is deterministic, headless and fast. Measured, in this repo, today: the seeded PRNG draws at 182 million/second and Q16.16 multiply runs at 274 million/second (§6.5). The combat designer measured the exact odds predictor at 0.45 µs per full preview — meaning an AI can evaluate 100,000 candidate attacks in 45 ms using the same predictor the player sees, with no separate AI estimator to drift out of sync. That is a genuine and unusual advantage.

The price: the digest is not fast. FNV-1a over the canonical encoding runs at 50 MiB/s in this repo (measured), and the specified XXH3 replacement will be faster but not free — AR-760 budgets TurnHash at ≤ 25 ms on the reference game and AR-770 shows it is 28% of a full replay. §4.7 therefore turns per-turn hashing off in farm runs and samples it instead. That is a real decision with a real risk and it needs to be made deliberately rather than discovered.

4. Games are event-sourced, so every game is a replayable log. Training data by construction — and, more importantly, minimisable training data: a farm failure ships as a bisected replay container (AR-970), which is the difference between a bug report and a bug.

The price: a replay log is not a training corpus. It is a sequence of orders, not a sequence of labelled positions. Turning it into training data means replaying it and materialising features at every turn, which costs a full re-simulation. §6.7 budgets it; the short version is that generating labels is about as expensive as generating games, so plan for 2× the farm capacity you first estimate, or emit features during the original run.

2. Level 0 — the tuning surface

Nothing in this document is possible until the AI's behaviour is data. This is not a nice-to-have; it is the gate. An AI whose thresholds are constants in TypeScript cannot be tuned, cannot be A/B tested, cannot ship two difficulty tiers from one binary, and cannot have a promotion gate, because there is nothing to promote.

2.1 The manifest

Every number the AI uses to make a choice lives in a parameter manifest: a versioned file of named integers with declared ranges, a declared scale, and a content hash. A plausible shape, with counts that are estimates of the right order rather than a specification:

Two hundred parameters is the right order of magnitude: enough that hand-setting them all well is implausible (which is why we tune), few enough that SPSA can move them jointly in one run (§5).

2.2 Integer discipline, and where it comes from

All manifest values are integers with a declared scale — never decimals, never a float parsed at load. This is stricter than the general plugin rule (which permits float arithmetic inside a WASM guest and forbids it only at the ABI), and the project's existing AI specification already takes the stricter line for shipped AIs. The reason is specific to this document: a promotion decision must mean the same thing on the machine that re-runs it. A tuned weight that is 0.734 in a JSON file is a different number on two decimal-to-binary paths; a tuned weight that is 734 per-mille is the same number everywhere, forever, including in the farm reproduction that adjudicates a desync three years from now.

The scoring arithmetic that consumes the manifest is integer or Q16.16 throughout, and ratio comparisons are cross-multiplied rather than divided, exactly as CM-120 requires of the command layer. The AI is not bound by CM-120 — it is not the command layer — but adopting the same discipline costs nothing and buys the reproduction guarantee.

2.3 The statement I most want on the record

Tuning cannot make the AI consider a plan it never generates.

This is the ceiling, and it is not raised by any level in this document. Weights choose among candidates; they do not invent candidates. If the candidate generator never proposes "abandon the southern salient, redeploy the whole Group to the Academy city, and accept losing two Towns", then no optimiser, no evaluation network, and no amount of self-play will ever produce that move, and every metric in §13 will look healthy while the AI plays a subtly small game.

The practical consequence for this pipeline: the highest-value thing the farm measures is not the weight vector, it is the candidate generator's coverage. §11 defines the instrument. If a tuning run produces less than the expected gain, the first hypothesis is not "the optimiser is bad", it is "the space is too small", and the fix lives in the companion AI architecture document, not here.

3. Measurement, because it is the hard part

Every naive plan for AI learning fails in the same place: it can generate candidates far faster than it can tell which candidate is better. The arithmetic is unforgiving and it is worth doing before anything else, because it determines the shape of everything else.

3.1 The noise arithmetic

A game outcome is a Bernoulli trial. To detect that candidate A is Δ Elo stronger than B, at α = 0.05 one-sided and 80% power, with draws rare (which the Vigil and Tenure design makes true — victory.md §2.3 gives a total order on outcomes, so genuine draws are near-zero):

p = 1 / (1 + 10^(−Δ/400)), δ = p − 0.5, N = (z_α + z_β)² × 0.25 / δ² = 6.183 × 0.25 / δ²

The 50-Elo row is the one the brief asks for: about 300 games unpaired, about 200 paired. That is cheap — minutes. The row that matters for a mature AI is the 5-Elo row, because once the obvious gains are taken, improvements arrive in 2–8 Elo increments, and each costs tens of thousands of games. The whole of §3.3 exists to move rows up this table.

3.2 Elo, and the free-for-all problem

Elo is a model of two-player zero-sum games. This game is not one, and the mismatch is not cosmetic.

In a four-player free-for-all, a candidate's win rate depends on the composition of the whole table, on seat position relative to map geography, and — most corrosively — on coalition dynamics that punish the strongest player. victory.md builds the Alarm and the World Watches (§6.2) precisely so that everyone can see who is close to a Vigil. A stronger AI therefore attracts more attacks and can measure as weaker. This is not a bug in the AI; it is a real property of the game that a naive metric reads backwards.

The resolution, and it is a firm recommendation:

1. Strength is measured head-to-head, two seats, symmetric map. This is the ranking metric, the promotion gate, and the anchor ladder. Elo is sound here.
2. Multi-seat play is measured separately and never used to rank. Its statistic is mean normalised placement — (P − rank) / (P − 1), in [0, 1] — plus the mean Tenure share, and it is a health check: it catches an AI that is strong 1v1 and catastrophic at four seats (a real and likely failure, since diplomacy-free multi-player play rewards behaviours 1v1 never tests).
3. Both legs run. A candidate that gains 15 Elo head-to-head and loses 0.06 mean placement at four seats does not promote; it opens an investigation.

A candidate note on kingmaking: with no diplomacy, an AI cannot form a coalition explicitly, but it can behave as one by targeting the leader. Whether shipped AIs should do that is a game-design question, not a learning question, and it belongs to the companion AI proposal — but the farm must measure it either way, because it changes what the multi-seat numbers mean.

3.3 Variance reduction: four techniques, ranked by value

(a) Seat-paired seeds — mandatory. Every job is issued twice: the same map seed, the same start positions, the arms swapped between seats. Map and start-position variance is enormous in a game where a Metropolis with an Academy trait might be three tiles from one capital and thirty from the other; pairing removes that component of variance entirely rather than averaging over it. Empirically in comparable engine-testing work this cuts required games by 30–40%; the table above assumes 35%. A farm that does not pair is throwing away a third of its compute.

(b) Common random numbers — cheap, partial. Both arms of a pair take the same game seed, so identical situations draw identical combat results. The honest caveat: the games diverge at the first differing order and CRN buys nothing after that. Its real value is early-game variance, which is where the map and the opening live. Keep it; do not overestimate it.

(c) A continuous outcome for the search phase — large, and a trap. Binary win/loss discards almost all the information in a 140-turn game. victory.md hands us something better for free: Tenure is monotone, integrated over the whole game, and already the design's universal tiebreak (§2.3). The final Tenure share, mapped to [0, 1000] per-mille, is a continuous outcome with far lower variance than the binary — plausibly a 2–4× reduction in games needed for the same discrimination.

The trap is Goodhart's, and it is a serious one: optimise Tenure share and you get an AI that plays for score rather than for the win — hoards cities, avoids the decisive risky attack, and loses to a player who takes the Vigil. So the rule is a split:

Continuous Tenure-share outcomes drive the optimiser's search. Binary win/loss drives the promotion gate. Never the other way around, and never both in the same test.

The search can afford a biased-but-cheap signal because a bad direction gets corrected next iteration. The gate cannot, because it is the thing that ships.

(d) Adjudication by proof — free, and better than the usual version. Chess engines resign self-play games on a heuristic eval threshold and accept the resulting bias. We do not have to. victory.md §8.2 defines mathematical elimination: because Tenure is monotone and the per-turn maximum is bounded, the engine can prove a trailing player can no longer reach the leader before the Horizon. Adjudicating there is exactly correct — zero bias, not small bias. Combined with Capitulation (§8.1) and the Ebb's guaranteed termination (§8.4), the farm gets short, provably terminating games with no adjudication heuristic to tune. This is a genuine gift from the victory design to the AI programme and it should be stated as a dependency so it is not optimised away.

3.4 SPRT and the promotion gate

Fixed-N testing wastes games on candidates that are obviously bad. The sequential probability ratio test stops as soon as the evidence is sufficient, and for the many candidates that are clearly worse than the incumbent it stops very early.

Using the normal approximation, with A = ln((1−β)/α) = 2.944 at α = β = 0.05 and score variance σ² = 0.25:

E[N] ≈ 2σ² × A / (μ₁ − μ₀)²

The recommended promotion gate is SPRT with bounds [0, 8], α = β = 0.05, seat-paired, head-to-head, over the map/ruleset matrix of §4.8. About 7,200 paired games: on the cost model of §4.4 that is roughly 5 hours on one 16-core desktop. It is a nightly-feasible gate, which is the property that matters — a gate that takes a week is a gate that gets skipped.

An 8-Elo lower bound is a deliberate choice to accept that improvements smaller than ~8 Elo will sometimes be rejected. That is the right trade for a small team: a rejected 4-Elo gain costs almost nothing, and a wrongly-accepted regression costs a release.

3.5 The anchor ladder

Elo is relative and self-play Elo drifts: an AI that beats last month's AI 60% of the time may be worse against a human, because both have specialised into a shared blind spot. The defence is a frozen anchor ladder:

• Anchor 0 is the hand-set baseline, its manifest frozen forever, its Elo defined as 0. It is never re-tuned, never rebuilt against a newer core except to keep it running, and every promoted version plays a fixed 600-game leg against it. If a version gains 40 Elo against the incumbent and 0 against Anchor 0, we have learned something important and unwelcome.
• Anchors 1..n are the promoted versions, retained. New candidates play a gauntlet against a sample of them, not only the incumbent, which is the standard defence against cyclic non-transitivity (A beats B beats C beats A) — a real hazard in a game with rock-paper-scissors unit interactions, and units.md §5.3's submarine counter chain is exactly such a structure.
• Anchor H, when it exists, is a fixed set of positions from human games (§9.2) scored by best-move agreement, not by Elo. It is the only anchor that is not self-referential and it is worth more than its size suggests.

Ladder hygiene rule: if a candidate beats the incumbent but not the median of the anchor sample, it does not promote. This costs some real progress and it is the price of not walking off a cliff together.

4. The farm

4.1 Shape: deliberately boring

The farm is a coordinator process and N worker processes. It is not a cluster orchestrator, not a message bus, and not a service. The reason is not modesty; it is that the farm has to be reproducible, and every layer of infrastructure is a layer that can perturb a result.

  elc tune / elc match / elc gauntlet          (drivers: what experiment to run)
              │  emits Jobs
              ▼
        Coordinator  ── job queue ──▶  Worker × N   (worker_threads, or remote over HTTP)
              ▲                            │ runs one headless game with @everylastcity/core
              │  Results                   │ emits ResultRecord (+ optional replay container)
              └────────────────────────────┘
                              │
                              ▼
                     results store (JSONL + containers)
                              │
                       analysis / SPRT / metrics

A Job is fully self-describing and content-addressed:

Job = {
  jobId,                     // = hash of everything below; the job IS its inputs
  mapSpec,   gameSeed,       // generator id + seed + parameters
  ruleset,   victoryTerms,   // which Tracks, Vigils, Horizon, Ebb schedule
  seats: [ { agentId, manifestHash, evalModelHash, tierKnobs, budgetFuel } ],
  pairIndex,                 // 0 or 1: which side of the seat-swapped pair
  hashPolicy,                // §4.7
  emitFeatures               // whether to materialise training positions
}

A ResultRecord carries the outcome, the winning Track, final Tenure per seat, game length, and a fixed vector of health metrics (§11). Because jobId is the hash of the inputs, a result is attributable and a disputed result is re-runnable by anyone with the same build.

4.2 What the workers actually run

AR-810 already specifies the CLI verbs the farm needs — new, run-ai, replay --verify, farm. This document adds three drivers rather than new engine capability:

Everything else is the existing headless runner. This matters for scope: the farm is about a week of engineering on top of infrastructure the architecture already requires, and that ratio is why Level 1 is a confident recommendation rather than an aspiration.

4.3 The cost model, and the measurement that replaces it

The farm's workhorse is not the reference game. AR-760's fixture (1,000×1,000, 8 players, 10,000 units) is a performance fixture; using it for tuning would be like tuning a chess engine at correspondence time control. The tuning game is deliberately small:

Standard tuning game: 160×160 map, 4 seats (or 2 for head-to-head), 48 cities per the victory design's reference map, Horizon 200, Capitulation and mathematical elimination on. Mean length ≈ 140 turns. Mean live units per seat over the game ≈ 35, peaking ≈ 60.

Per-turn cost on REF-HW (4 cores @ ~3 GHz, Node 22), simulation-only configuration:

Per-game cost ≈ 140 × (40 ms + 4B).

¹ The measurement machine for §6.5 is an AMD Ryzen 9 9950X3D: 16 physical cores, ~1.8× REF-HW single-thread on the microbenchmarks below. 15 cores usable, 1 reserved for the coordinator.

Every number in that table is an estimate resting on other estimates. AR-760's per-order and upkeep budgets are themselves unmeasured — the architecture document's own open question 21 says so plainly — and CM-2100's 50 ms for 400 activations is a target, not a measurement. So:

The first thing the farm must produce is not a tuned weight vector. It is a measured games/core-hour figure for the Standard tuning game, published as a benchmark alongside elc benchmark, and re-measured on every release. Until that number exists, §4.4's costs are a spreadsheet, and every plan built on them should carry a factor-of-three error bar.

If the measured figure lands within 2× of the table, everything in this document holds. If it lands 10× worse — which would mean ~400 ms/turn of simulation — Level 1 still works (it just runs over a weekend instead of a night) and Level 2's data volumes get uncomfortable. That is the shape of the downside and it is survivable.

4.4 Cost in money

A 16-vCPU (8 physical core) spot instance is roughly $0.15–0.35/hour at current commodity pricing; a 32-vCPU roughly double. Taking the 100 ms row and a 32-vCPU spot box at ~$0.50/hour delivering ~1,500 games/hour:

The finding worth stating loudly: the entire Level 1 and Level 2 programme costs less than a laptop and fits on hardware the team already owns. Compute is not what stands between this project and a strong AI. Engineering time, measurement noise, and the candidate-generator ceiling are.

And the honest counterweight: the last row is not merely expensive, it is expensive and uncertain, which is a much worse combination than expensive alone. §8 treats it accordingly.

4.5 Diversity, or self-play eats itself

The classic self-play failure is convergence to a narrow, mutually-exploitable equilibrium: two copies of the same AI agree on an opening, agree on a doctrine, and get very good at a game no human plays. The countermeasures are cheap and must be built in from the first run, because retrofitting them invalidates every result collected before.

The temperature mechanism deserves a note, because it is the one that looks like it breaks determinism and does not. The AI draws from its own private, seeded, per-position stream — the AI framework already specifies exactly this, with per-position privacy and outcome-neutrality. A temperature-sampled opening is therefore reproducible from the job seed, which is what makes a farm result re-runnable. Randomised behaviour and reproducible behaviour are not in tension here; ambient randomness and reproducibility are, and there is none.

4.6 Termination

A self-play farm's worst failure is not a crash, it is a game that never ends. This design has three independent guarantees and the farm should enable all of them:

1. The Ebb (victory.md T1) lowers the Dominion threshold on a published schedule to 501‰, reaching it at the Horizon. Termination is structural.
2. Mathematical elimination (§8.2) proves a trailing seat cannot catch up and removes it.
3. Capitulation (§8.1) ends mop-up.

Plus one farm-only rule: a game with no state-changing event for 20 consecutive turns is a stalled game, is killed, and files a minimised replay (AR-970 already requires the detector). A stall in the farm is a bug in the AI or the rules, never a legitimate outcome, and it must be loud.

Be honest about the distributional cost: farm games run with aggressive Capitulation and elimination, so the farm never sees the last 20 turns of a mop-up that a human game would play out. That biases the corpus away from endgame positions. The mitigation is a small dedicated endgame corpus: positions checkpointed at 80% of Horizon and played out with adjudication off, ~5% of farm volume.

4.7 Hashing policy, and a measured warning

The canonical digest in this repo runs at 50 MiB/s (measured, §6.5) — and it is FNV-1a, a deliberate placeholder for the specified XXH3. Even assuming XXH3 in pure TypeScript reaches 4–8× that, hashing a multi-megabyte state every turn would dominate the farm's cost, quite possibly exceeding all the simulation and all the AI thinking put together.

Farm hashing policy:

• Ranking and tuning runs: hashPolicy = sampled(25) — hash every 25th turn plus the final state. A divergence is still caught, just up to 25 turns late, and the minimised-replay bisector (AR-970) closes the gap.
• Determinism legs and any run whose result is evidence in a dispute: hashPolicy = every.
• Cross-platform determinism CI: hashPolicy = every, small corpus, run per-commit, and it is a gate rather than a metric — any mismatch blocks.

This is a place where a plausible-looking default (hash everything, it's safer) would silently cost the project half its farm throughput. It is worth a line in the config and a comment explaining why.

4.8 What runs nightly

Weekly, additionally: a large-map leg (does the AI degrade at 1,000×1,000?), a full anchor round robin to re-estimate the whole ladder, an endgame-corpus leg, and a re-measurement of the games/core-hour benchmark.

4.9 What catches a regression

Three distinct kinds of regression need three distinct instruments, and conflating them is a common and expensive mistake:

1. Correctness regressions — a desync, an illegal order, a crash. Caught by the determinism and sanity legs. Always blocking, never negotiable, never "we'll look at it after the release".
2. Strength regressions — the AI got weaker. Caught by SPRT against the incumbent with bounds [−3, 1] when testing a change not intended to gain strength (a refactor, an engine optimisation). That test costs ~29,000 games, which is why it runs weekly and per-release rather than nightly.
3. Behaviour regressions — the AI is no weaker but has become unpleasant to play against: turtles for 60 turns, never uses naval units, spams the cheapest unit. Caught by the metric vector of §11 against a rolling baseline, and this is the one that will actually bite, because it is the one no gate can express as a single number.

5. Level 1 — offline weight tuning

5.1 Why SPSA and not the others

The optimiser must survive three brutal properties of this objective: it is stochastic (one game is one noisy sample), expensive (a reliable evaluation is thousands of games), and high-dimensional (~200 parameters).

Recommendation: SPSA over the full manifest, with CEM for the strategic subvector and Bayesian optimisation for the search-control knobs. SPSA is the workhorse because its cost is independent of K, and K is the thing we cannot reduce.

5.2 The recipe, with numbers

At iteration k, for the integer parameter vector θ with per-parameter range [lo_i, hi_i]:

1. Draw Δ_i ∈ {−1, +1} for every i from the run's seeded stream (Bernoulli, not Gaussian — SPSA requires bounded inverse moments and ±1 is the standard choice).
2. Form two arms: θ⁺ = clamp(θ + c_k·Δ), θ⁻ = clamp(θ − c_k·Δ), rounding to integers.
3. Play G seat-paired games of θ⁺ against θ⁻. Let y ∈ [−1, +1] be the paired outcome margin (Tenure-share based, per §3.3(c)).
4. Update every parameter at once: θ_i ← clamp( θ_i + round( a_k · y / (2 c_k Δ_i) ) ).

Schedules, in the standard Spall form:

c_k = c_end · (k_total / (k + 1))^0.101        // perturbation size, decaying slowly
a_k = a_end · (k_total / (k + 1))^0.602        // step size, decaying faster

Concrete starting values for a first run:

A cheaper first pass — k_total = 4,000, G = 10 → 80,000 games, ~16 h — is what §4.4 costs, and is the right thing to run first, because the first run's job is to find out whether the machinery works, not to produce the final weights.

5.3 Budget transfer: the trap in this plan

Tuning at B = 25 ms and shipping at B = 100 ms is tuning one agent and shipping another. The weights that are best for a shallow search are not always best for a deeper one — a shallow searcher benefits from cautious weights that compensate for its blindness, and those same weights make a deep searcher timid.

This is well-documented in engine tuning and it must be handled explicitly:

After every tuning run, re-rank the top 3 candidates at the shipping budget with a 1,200-game paired match before the full SPRT gate. If the ranking at B = 100 ms differs from the ranking at B = 25 ms, the tuning budget is too far from the shipping budget and must be raised.

That check costs ~1 hour and it is the difference between a tuning pipeline and a tuning pipeline that works.

5.4 What to expect, honestly

The first row is the one to be careful with in a status report. A +300 Elo gain sounds like the AI became three times better; what it actually means is that the hand-set numbers were bad, which was always true and is why we tuned. The gain is measured against our own worst version, and against a human it may be worth much less. The anchor ladder's Anchor H (§3.5) is the only honest instrument here, and it does not exist until humans have played.

5.5 Worked example: night 128

A concrete run of the machinery above, with real arithmetic, as a specification of what the output looks like.

The SPSA run tune-0043 finished at 06:20 having played 79,840 games over 3,992 iterations at B = 25 ms. It emitted three candidates: the final iterate w-129a, the best-of-run by rolling Tenure margin w-129b, and the average of the last 400 iterates w-129c (iterate averaging is standard and usually wins).

Budget-transfer check, 1,200 paired games each at B = 100 ms:

The top two swapped. That is inside the noise for 1,200 games (±1.4% at 1σ) and does not by itself trip §5.3's rule, but it goes in the log; two consecutive nights of disagreement would raise the tuning budget to 50 ms.

w-129c enters the SPRT gate at 07:05. Bounds [0, 8], α = β = 0.05, seat-paired, across the 6-ruleset matrix.

Accepted at 6,880 games — slightly under the 7,230 expectation, which is what an above-elo1 candidate looks like. Point estimate: +15.9 Elo (95% CI +7.1 to +24.8).

Anchor leg, 120 games against each of 5 sampled anchors:

Positive against the median anchor, so §3.5's hygiene rule is satisfied. Note Anchor 0's 94.2%: after 51 promotions the AI beats the hand-set baseline 19 games in 20, and that number will keep creeping toward 100% and stop being informative. When it passes 97% the ladder needs a new floor anchor, and that should be scheduled rather than noticed.

Multi-seat health, 600 games: mean normalised placement 0.541 vs incumbent 0.536 (+0.005, inside noise). No investigation.

Balance metric vector, diffed against the 30-night baseline: two flags.

The naval flag is the interesting one and it is exactly the kind of thing SPRT cannot see: w-129c is 16 Elo stronger and has quietly decided navies are not worth building. Either it is right — in which case units.md has a balance problem the farm just found (§11) — or its candidate generator proposes bad naval plans and tuning learned to avoid the whole category, which is §2.3's ceiling showing itself. w-129c promotes (the gate is strength and it passed), and the naval flag opens a ticket against the candidate generator with a saved 200-game coastal corpus attached.

That last paragraph is the shape of the whole pipeline working correctly: a numeric gate that promotes, and a metric vector that tells you what the gate did not look at.

6. Level 2 — a learned evaluation function

6.1 What it evaluates, and what it emits

Input: one seat's fog-filtered knowledge view of one position, at the start of an Orders phase. Nothing else. The hard constraint from the brief binds here absolutely — the network sees exactly what a human in that seat sees, including the observation ages and the last-known estimates, and never the true board.

Output: a single integer in [−1000, +1000], the predicted normalised final placement of this seat, per-mille. +1000 is a certain win; in a P-seat game the label for a finished game is round(2000 × (P − rank) / (P − 1)) − 1000.

Why placement rather than win probability: in a multi-seat game, "probability of winning" is a poor training signal for a seat that is third of four, where every gradient it receives is "you lost" and none of them distinguishes nearly won from eliminated on turn 30. Normalised placement is dense, ordinal, and correct at both extremes. For head-to-head games it degenerates to exactly ±1000, so the ranking metric of §3.2 is unaffected.

6.2 Features

Features must be integer, computable from the fog view in well under the evaluation budget, and — the non-obvious requirement — scale-free, so a model trained on 48-city maps transfers to 200-city maps. Nearly everything is therefore expressed as per-mille of a map total or as a ratio, not as a count.

Two design notes worth arguing for.

Victory Track features are the highest-value block in the table and they are free. Because victory.md makes every condition one shape — a progress integer, a threshold, a Vigil (§2.1) — the AI's evaluator gets a uniform, already-computed, already-legible feature for every victory condition in the game, including ones added later. An AI in most 4X games has to be taught what each victory condition means. This one is handed it as an integer.

No raw map tensor. A convolutional view of the board is what an AlphaZero-style approach would want, and it is deliberately excluded at this level: it is 100× the inference cost, it does not transfer across map sizes, and the geometry block above captures the spatial facts that matter at the strategic level. The tactical spatial reasoning lives in the candidate generator and in Rehearsal, where it belongs.

6.3 The float boundary — normative

This is the crux the brief asks about, and it deserves a precise answer rather than a reassurance. The answer has five parts and the first one does most of the work.

(1) The model is not part of the rules, and the reason is structural, not a promise. The AI's output is an order. Orders enter the authoritative order log as integers, in the same envelope a human's orders use, and are executed by the core. Replay never re-invokes AI code — it replays the log. Therefore:

A change to the evaluation model cannot invalidate a single saved game or replay. It is not subject to commandLogicVersion (CM-2050), because it is not command-layer logic. The command layer decides what a Posture does; the model only decides which Posture the AI sets, and setting a Posture is an ordinary order any human could issue.

That boundary is worth stating as a rule because it is the thing that makes the whole learned-eval programme safe. Everything a learned model touches is on the authoring side of the order log, and nothing on the executing side.

(2) But reproducibility of AI decisions is still required, in three places. Farm reproduction runs, desync arbitration, and the determinism spot-check all re-run an AI turn and compare order streams. In those contexts a non-deterministic model produces a false divergence report, which is worse than useless because it destroys trust in the divergence detector. So the model must be bit-exact on every platform — not for replay, but for verification.

(3) Training is float and lives outside the core. Training runs in a separate package (@everylastcity/ai-train), in float32, using whatever framework is convenient, on whatever hardware is available, and is never imported by packages/core, never subject to the determinism lint, and never shipped. Its only output is a data file.

(4) Inference is integer-only, and the arithmetic is specified. The shipped artifact is a quantised weight file:

EvalModel {
  modelId,            // content hash of everything below
  version,            // integer, monotonic
  architecture,       // layer sizes, fixed at publish
  inputScale,         // features are already integers; this states their expected range
  layers: [ { weights: Int8Array | Int16Array, bias: Int32Array, outShift: uint8 } ],
  outputScale         // shift/multiply to land the result in [-1000, 1000]
}

with these rules:

• Weights are int8 (dense layers) or int16 (sparse feature columns). No float appears in the file, at rest or in memory.
• Accumulators are int32. Activations are clipped ReLU into [0, 127].
• Rescaling between layers is an arithmetic right shift by a per-layer constant outShift. Never a divide, never a multiply by a reciprocal. Shifts are exact and identical on every platform.
• Overflow is proven, not hoped. For a layer of width W, the accumulator bound is |bias| + W × 127 × 127. At W = 512 that is 8,258,048, comfortably inside int32, and at W = 4096 it is 66,064,384 — still inside. The build asserts |bias| + W × 16129 < 2³¹ for every layer and refuses to publish a model that cannot. This is a two-line check that eliminates an entire class of platform-dependent bug.
• No Math.* transcendental appears on the inference path. There is no sigmoid, no softmax, no exponential — the output is a raw integer score and it is ranked, never converted to a probability. If a probability is genuinely wanted for display, it comes from a 256-entry integer lookup table shipped with the model.
• A NaN cannot exist, because no float exists.

Worked example, one neuron, exactly: layer 0 has W = 4 (shortened for the example), weights w = [+96, −40, +12, +127], bias b = 3200, outShift = 6. Input activations x = [80, 127, 5, 64].

acc = 3200 + (96×80) + (−40×127) + (12×5) + (127×64)
    = 3200 + 7680 − 5080 + 60 + 8128
    = 13988
s   = acc >> 6 = 218          (arithmetic shift, floors toward −∞)
out = clamp(218, 0, 127) = 127

Every step is integer; >> on a value in int32 range is identical in every JavaScript engine and every WASM engine. This is reproducible by construction, not by testing.

(5) One subtlety that is easy to miss: fuel stability. If the AI runs as a metered WASM plugin and shapes its search on remaining fuel, then the instruction count of an evaluation must not depend on the data, or two hosts with identical fuel budgets will complete different amounts of search. A dense network with fixed loop bounds is fuel-stable. A sparse-input network with a variable number of active features is not. Therefore, if the sparse architecture of §6.6 Tier C is adopted, the active-feature list must be a fixed-length array of 256 slots, padded with a null bucket whose column is all zeros. The padding costs a few microseconds and buys fuel stability, which is not optional.

6.4 Quantisation, and its gate

Train in float32 with quantisation-aware training — simulate the int8 rounding and the clipped ReLU in the forward pass during training so the model learns weights that survive quantisation. Post-training quantisation without QAT typically costs 10–30 Elo in comparable systems; QAT typically costs under 5.

The gate is explicit and it runs before any model ships:

The quantised model must score ≥ 47% against its own float32 parent over 2,000 paired games. A larger gap means the quantisation is wrong, not that int8 is inadequate, and the model does not ship until it is fixed.

Running the float parent requires a float inference path, which exists only in the training package and only in the farm. That is fine: the float model never ships and never enters a player's game.

6.5 Measured inference cost

Measured on this repository, today, on Node 22.23.2 / AMD Ryzen 9 9950X3D (16C, 4.3 GHz), Windows 11. REF-HW is 4 cores at ~3 GHz; the derate below assumes 1.8× single-thread for that machine and 3.6× for a 2019-class phone, both estimates.

The three dense rows are strikingly consistent — 0.573, 0.579 and 0.595 ns per multiply-accumulate — which yields a rule worth carrying around:

Scalar integer inference in TypeScript costs ≈ 0.58 ns per MAC on a fast desktop core, ≈ 1.0 ns on REF-HW, ≈ 2.0 ns on a 2019 phone. So: cost in µs ≈ MACs ÷ 1000 on REF-HW.

Two consequences:

• Inference is not the bottleneck. A 5,000-MAC network is 5 µs on REF-HW. A 100 ms turn budget buys 20,000 evaluations; the local-interactive budget in the AI framework is far more generous still. There is no need to compromise the architecture for speed at this scale.
• The accumulator refresh is the expensive operation, not the network. 188 µs on REF-HW for a 400-feature refresh dwarfs the 18 µs head. Sparse architectures only pay off when positions are evaluated incrementally, and in Rehearsal (§7) consecutive candidates differ substantially. This is a strong argument for the dense architecture at Tier B and against rushing to Tier C.

On WASM SIMD: the AI runs as a WASM plugin, and the AI framework permits deterministic SIMD while banning relaxed SIMD. Integer SIMD (i16x8.add, i32x4.dot_i16x8_s) is bit-exact by specification, so it is available without any determinism cost. A hand-written SIMD kernel should give 4–8× on the dense layers. This is the one place in the whole project where WASM SIMD is both permitted and worth having, and it is worth knowing it is in reserve — but it is an optimisation for later, not a dependency, because the scalar numbers already fit.

6.6 Three architectures, and which to build

Tier C's size is not a coincidence and it is worth pointing out, because two independent constraints agree on it. The mobile memory envelope and the client bundle both want the model under ~1 MB. And the training data available to a small team (§6.7) honestly supports on the order of 10⁵–10⁶ parameters and no more. 2048 × 128 × 2 bytes = 512 KiB sits inside both. When the data budget and the deployment budget agree on a number, that is the number.

Tier A deserves more respect than it usually gets. A tuned linear evaluator over 180 well-chosen features, trained on real self-play outcomes, is a serious opponent; the gap between "no model" and "linear model" is much larger than the gap between "linear model" and "small MLP". It is also the only tier whose decisions can be explained in a sentence — "scored +180: your Dominion progress is 612‰ against their 540‰, and you hold two Landmarks" — which the AI's decision-explanation surface needs. Ship Tier A first even if Tier B is planned, because Tier A is the baseline that tells you whether Tier B was worth it.

6.7 Training data: labels, volume, generations

Label generation. A game of 140 turns × 4 seats yields 560 labelled positions. Two labels per position:

• The outcome label: the seat's final normalised placement. Correct but distant.
• The bootstrap label: the model's own evaluation of the position N turns later (TD-style). Nearer and lower-variance, but self-referential — it can only be used after generation 1.

Recommended: train generation 1 on outcome labels alone, then blend 0.7 × outcome + 0.3 × bootstrap(N=10) for later generations. This is a well-trodden path and the blend ratio is a knob worth tuning once, cheaply.

Volume, honestly. The raw position count overstates the information: consecutive turns in the same game are ~95% correlated, so 560 positions per game is perhaps 20–40 effective independent samples. For each tier:

Generations, and the bootstrap problem. The first model learns to predict the outcomes of games played by a weak AI. It is therefore an accurate evaluator of weak play, which is not what we want. The fix is iteration — train, deploy, generate better games, retrain — and each generation costs a full corpus. Budget 3 generations to reach a useful model: ~150,000 games, ~4 days on one desktop, ~$50 on cloud. Expect diminishing returns after generation 3 unless the search improves too, because the corpus quality is bounded by the agent that generated it.

The cost of materialising features. Turning a replay into labelled positions requires re-simulation. It is cheaper to emit the feature vector during the original farm run (emitFeatures: true in the Job) at a cost of ~200 integers per seat-turn — 560 × 200 × 4 bytes ≈ 450 KB per game, or ~8 GB for an 18,000-game corpus. That is a laptop SSD, not a data platform. Emit during the run; do not plan to re-derive.

6.8 Expected payoff, honestly

The honest position: the learned evaluator is the part of this document most likely to under-deliver, and it should be sequenced after Level 1 and after Rehearsal for exactly that reason. Both of those are cheaper and both have higher expected value. If the schedule is tight, ship Tier A, tune it with SPSA alongside everything else, and treat Tier B as the following release's work.

7. Level 2.5 — Rehearsal, the search this turn model makes cheap

This is not learning, and it belongs in this document anyway, because it is the multiplier that makes learning worth doing: search converts evaluation quality into play quality, and without it a better evaluator produces only a slightly better greedy choice.

The Orders-and-Cascade model hands us something most strategy games cannot offer. A turn is a pure function — order set in, resolved world out — with no player input during resolution, a frozen activation order, and a fast deterministic core. So the AI can simulate its own turn before committing it:

Rehearsal(K):
  candidates ← generator produces K distinct order-sets (varying Posture assignment,
               Sanction presets, Posting weights, Requisition shape)
  for each candidate c:
      world' ← Cascade(world, c, opponentModel)      // one full turn, forked state
      score  ← Eval(world' from our seat's view)
  commit argmax score, ties by lowest candidate index

Cost. One Cascade on the Standard tuning game is ~40 ms of simulation (§4.3) — too expensive for K = 32. Two properties rescue it:

1. Most candidates differ locally. Two candidates that differ only in the Northern Posting's Requisition produce identical activations everywhere else. A copy-on-write fork plus activation-level memoisation should bring the marginal Cascade to a small fraction of the full one; call it 4–8 ms, and flag it as the estimate this whole section rests on.
2. Rehearsal need not simulate the whole turn. Evaluating after only the own-seat activations plus adjacent enemy reactions captures most of what matters and costs proportionally less.

At 6 ms per marginal Cascade plus 22 µs of Tier B evaluation, K = 16 costs ~96 ms — one release tier turn budget — and K = 4 costs ~24 ms, which fits the tuning tier. That is a real search over a real branching structure, and it is affordable because the turn model made a turn into a function.

The honest problem: opponentModel. Rehearsal simulates our orders against something. The options, in increasing cost and increasing honesty:

Recommendation: persistence at the tuning tier, cheap-policy at the release tier, and the knowledge-view constraint is absolute — the opponent model may only use what our seat can see. This is the one place in the AI where cheating would be invisible and catastrophic to trust, so it wants an explicit test: run the AI against a modified core that reports different hidden state and assert the order stream is unchanged. That test is cheap, it is decisive, and it should be in the nightly sanity leg.

Rehearsal's other virtue is that it is the natural home for the policy prior of §8.2: a learned policy that orders the candidate list lets a smaller K find the same answer.

8. Level 3 — learned policy and AlphaZero-style search

8.1 Four structural obstacles, priced

The pattern that produced superhuman play in Go, chess and shogi is: a policy-and-value network, MCTS guided by the policy, self-play at scale, iterate. Every element of it meets a specific obstacle here, and the obstacles are not engineering inconveniences — they are the reasons the technique does not transfer for free.

(1) The action is a vector, not a move. AlphaZero's policy head emits a distribution over ~4,700 chess moves or 362 Go points. Here the action is a joint assignment over every command object: a Posture, four Sanctions, and a geometry (a Screen line, a Survey region, a March route) for each of ~12 objects. There is no natural fixed-size output layer. Autoregressive decoding — emit one command object's order at a time, conditioned on the previous ones — is the known technique and it works, at the cost of one network evaluation per object per node, which multiplies the search cost by ~12.

(2) There is no cheap make/unmake. Chess make/unmake is nanoseconds. Here a single "move" is a whole Cascade: ~40 ms full, maybe 6 ms marginal (§7). AlphaZero used ~800 simulations per move. At 6 ms that is 4.8 seconds per turn of pure simulation before the network is consulted — 50× the local budget and 200× the tuning budget. This is the hardest of the four and it is arithmetic, not opinion.

(3) Imperfect information. MCTS assumes a known state at each node. The AI has a fog-filtered view. The available techniques — determinization with particle sampling, information-set MCTS, counterfactual-regret-style methods — are all substantially more complex than MCTS, all have known pathologies (determinization in particular is strategy fusion: the search assumes it will know things it will not), and none is a drop-in.

(4) Simultaneous moves. Both seats commit before the Cascade. A node in the tree is therefore a matrix game, not a max node, and its correct value is a mixed-strategy equilibrium. Decoupled UCT is the common heuristic and is known to converge to the wrong thing in adversarial cases. Solving each node properly means a linear program per node, which is not affordable at any node count that matters.

Any one of these is a research problem. All four together, for a team that also has to ship a game, is not a plan.

8.2 What is affordable, and worth doing

A policy prior as a candidate orderer, not a candidate replacer. Train a small network to predict, for each command object, the order the current best AI (or a human, §9) would give it. Use its output to rank the candidate list Rehearsal searches, so K = 8 finds what K = 32 would have.

• Cost: one forward pass per command object per turn — ~12 × 22 µs = 0.26 ms. Negligible.
• Training data: the same farm corpus, relabelled with the chosen order rather than the outcome.
• Risk: low. If the prior is bad, the search still works, just less efficiently. This is the correct shape for a policy in this game: an accelerator with a graceful failure mode, not an oracle.

Depth 2 Rehearsal for the decisive turn. Once per game — when a Vigil is within reach, or a Seat is threatened, or elimination arithmetic is close — spend 10× the budget on a two-turn lookahead over the top 4 candidates. victory.md gives the AI exactly the signal it needs to know when that turn has arrived, since Track progress and Vigil counters are explicit integers. Situational depth beats uniform depth when the budget is small, and this design conveniently publishes when the situation is decisive.

8.3 The Skirmish lab — the affordable probe

If the question "would AlphaZero-style learning transfer here?" is worth answering — and I think it is, because the answer shapes a multi-year roadmap — then answer it on a version of the game small enough that the full loop is affordable, rather than arguing about it.

Skirmish: 40×40 map, 2 seats, 8 cities, Industry tier 1 only (units.md §4.1 is explicitly "the complete simple game"), Dominion Track alone, Horizon 60 turns, no Cadre, no naval. It is a real instance of the same rules — the same Cascade, the same Clash, the same Postures — not a toy.

Estimated cost: ~15 units per seat, ~60 turns → a Cascade of perhaps 2 ms per turn, so a game costs 60 × (2 ms + 2B):

Read that table honestly. Skirmish makes a 10⁵–10⁶-game AlphaZero loop affordable — days, not months — and that is the scale at which the technique has been shown to work on small domains. A full 10⁷-game run at a genuine MCTS budget is still months, on Skirmish, on the team's own hardware. The lab does not make the ambitious version cheap; it makes the question cheap, which is the point.

What it would tell us, and this is why it is worth the two weeks:

• Whether the joint-action policy factorises usefully over command objects (obstacle 1).
• Whether determinization is tolerable at this fog density, or whether strategy fusion wrecks it (obstacle 3).
• Whether decoupled UCT is good enough at this simultaneity depth (obstacle 4).
• Whether a learned agent beats the Level-1+2+Rehearsal agent on Skirmish. If it does not win there, it will not win on the full game, and the answer cost two weeks instead of two years.

The Skirmish lab has a second, immediate payoff that justifies it independently: it is a fast regression corpus for the whole pipeline, where a full tuning run takes an hour instead of a day. Build it for that reason even if the research question is never asked.

8.4 Verdict

Do not plan an AlphaZero programme. Plan the policy prior (§8.2), plan the Skirmish lab (§8.3), and let the lab decide whether anything more ambitious is warranted. If a specialist joins the team later, the lab is the environment they would need on day one anyway.

9. Level 4 — learning from human games

9.1 Volume, honestly

The hopeful version of this is "thousands of players generate millions of games and the AI learns from all of them". The arithmetic does not support it. A niche turn-based strategy game might reach 20,000–100,000 owners. Telemetry opt-in rates for game clients run 5–20%. Games per opted-in player per year, perhaps 15–30.

Plausible best case: ~3,000 consenting players × 20 games = 60,000 games per year.

The farm produces that in 12 hours. So the conclusion is unavoidable and should be stated in the first paragraph of any plan that mentions human data:

Human game logs are not a volume play. Self-play beats them on volume by three orders of magnitude, forever. Their value is entirely in being different from self-play, and every use we make of them should exploit difference rather than quantity.

9.2 What they are actually good for

9.3 Consent and privacy — normative

• Opt-in, explicit, per-game, revocable. Never a default, never bundled into a general telemetry consent, and revocation must delete the already-uploaded logs, not merely stop future ones.
• All seats must consent, or the game is not collected. A multiplayer replay is one artifact containing every seat's orders; one seat cannot consent on another's behalf. Be honest about the consequence: multiplayer corpora will be small and skewed toward friend groups, and the practical corpus is solo games against AI, where there is exactly one human to ask.
• Identity is stripped at source. The client uploads the order log and setup, with a per-game salted pseudonym, and no account identifier. Nothing in the pipeline needs to know which games came from the same person except difficulty calibration, which needs a per-game self-reported or MMR-derived skill band, not an identity.
• Free text never leaves the machine. Chat, unit names, map annotations, save names, custom Doctrine and Posting names (CM-770, CM-1130 make these player-authored strings) are the only personal data in a replay, and they are stripped client-side before upload rather than server-side after. The order log itself is integers.
• Retention is bounded and published, and the corpus version used to train each shipped model is recorded in that model's provenance, so a deletion request can be answered honestly about what was and was not derived from that player's data.
• A player's own data must be exportable and inspectable. A player who consented should be able to see what was uploaded.

9.4 The fog-safety hazard

A replay container holds the full game. Distributing one to a training pipeline mid-game would leak every seat's hidden information — and the architecture already contemplates a fog-safe transfer form for exactly this reason. Only completed games are collected, and the collection path must assert the game is terminal before upload. This is a one-line check that prevents a category of information-leak bug that would otherwise be discovered by a player noticing an AI knew something it could not.

9.5 Behaviour cloning, and its ceiling

Cloning human orders produces an agent that plays like the average of its corpus. The average player of a strategy game is not strong. An AI trained to imitate them is capped at their level and, worse, inherits their systematic errors — overvaluing early conquest, underinvesting in Integration, ignoring the Vigil clock.

Use human logs to seed a prior and to benchmark. Never use them as the strength target.

The exception, and it is a real one: cloning is the right tool for a low difficulty tier. An opponent that plays like an average human is exactly what "Regular" should mean, and it fails plausibly by construction — which is the hardest property to engineer deliberately and the one that makes easy AI feel like an opponent rather than a broken one.

10. Difficulty tiers from one model, without cheating

The hard constraint is absolute: same fog-filtered view, no resource bonuses, difficulty from better play. That constrains the levers to four, and all four are data in the manifest (§2.1):

Eval blur is the important one and it must be applied to the score, never to the legality or the arithmetic. A tier that mis-scores a plan looks like a player with different judgement. A tier that mis-computes odds looks like a bug. The blur draw comes from the seeded per-position AI stream, so a "random" mistake is reproducible in a replay and can be explained afterwards, which is a small delight: "Regular rated that assault 620 when it was 700; it went in and it lost."

The plausibility floor. The easiest tier must still never: leave a city without a production setting; leave a unit Idle for more than one turn (CM-540 makes that visible and it must not be the AI's normal state); ignore an adjacent undefended city (which CM-710's Seize: Undefended default makes automatic anyway); strand a transport permanently; or leave its Seat ungarrisoned when an enemy is within reach. These are not strength; they are the difference between "easy" and "broken", and they belong in the nightly sanity leg as assertions rather than as metrics.

Monotonicity is a gate, not an aspiration: each tier must beat the tier below with ≥ 60% over ≥ 1,000 farm games, or the tier ladder is a lie. Cheap to check, and it runs weekly.

11. The farm's second product: balance

A self-play farm that only measures AI strength is being used at half capacity. Every design document in this set ships specific numbers chosen by argument — the 700‰ Engagement default and the hysteresis constant of 2 (CM-590, CM-460), the four-turn Forming Up penalty, the 25–35 turn city payback target, the Vigil lengths and the Ebb schedule. The farm is the only instrument that will ever test them at volume, and it is nearly free to point at them.

The metric vector emitted by every farm game:

Two honest warnings about using AI games for balance.

AI-measured balance is AI-shaped balance. If the farm reports that submarines are never built, the two hypotheses are "submarines are underpowered" and "the AI does not know how to use submarines", and they are indistinguishable from the metric alone. The disambiguator is a scripted probe: a hand-written agent that uses the questioned unit competently, played against the tuned AI. If the probe wins, the unit is fine and the AI is blind. This costs an afternoon per question and it is the only way to make the balance data trustworthy.

Balance measured against a tuning-tier AI is measured against a weak player. Balance flags should be re-checked at the release tier before anyone changes a number in a design document.

12. Honest tradeoffs

The farm is a permanent tax. It is roughly a week to build and then a continuing cost forever: someone reads the nightly report, someone maintains the anchor ladder, someone investigates the behaviour flags. A farm nobody reads is worse than no farm, because it manufactures false confidence. If the team cannot commit to reading it, build the determinism and sanity legs only and skip the rest — those two pay for themselves without a human in the loop.

Tuned weights break replay compatibility in a narrow but real sense. A replay is exact regardless (the orders are in the log), but reproducing an AI turn requires the same manifest and the same model. So every game records the manifest hash and the model hash, and old versions must be kept retrievable. That is a content-versioning obligation the platform must carry, and it grows without bound unless pruned.

Self-play makes the AI good at playing itself. §4.5's diversity mechanisms mitigate and do not eliminate this. The only real cure is human games, and §9.1 says how few of those there will be. The honest expectation is that the AI will have a characteristic style, that strong human players will find its weaknesses within a few dozen games, and that patching those weaknesses is an ongoing content activity rather than a one-time training run. Plan for AI updates as a live-service activity, or accept that the AI is at its strongest on release day and declines relatively thereafter.

The tuning-tier / shipping-tier gap is a permanent source of subtle error. §5.3's check contains it; it does not remove it.

Level 2 might not pay. It is the largest engineering investment in this document and the one whose payoff I am least sure of. Sequencing it third — after tuning and after Rehearsal — is a deliberate hedge, and the hedge should be honoured even under schedule pressure, because the pressure version of this plan is "skip the tuning, go straight to the neural network", which is exactly backwards.

Everything numeric here rests on unmeasured simulation budgets. §4.3 says so and it is worth repeating in the tradeoffs, because if the measured game cost is 10× the estimate, the nightly gate becomes a weekly gate and this document's rhythm changes shape.

13. What could go wrong, and how we would see it

The single fastest diagnostic, and the one I would build the report around: Anchor 0 win rate and Anchor H agreement, plotted together over every promotion. The first should rise and saturate. The second should rise. If the first rises and the second is flat, the AI is getting better at beating its own ancestors and no better at playing the game, and everything else in the report is noise.

14. Contracts on other documents

Obligations this design places on subsystems it does not own, recorded so a change on either side is detectable.

1. The AI architecture proposal must expose every decision number as a manifest parameter with a declared range (§2.1), must define the candidate generator whose coverage is this pipeline's ceiling (§2.3), and must guarantee that its opponent model in Rehearsal reads only the acting seat's knowledge view (§7).
2. 10-turn-model.md (TM) must keep the Cascade a pure function of committed orders and frozen initiative, and must make a forked Cascade over a candidate order-set possible without mutating the live world. Rehearsal is entirely dependent on this and on nothing else.
3. 13-command.md (CM) already provides what is needed and this document takes a dependency on CM-330 (six Postures), CM-570 (four Sanctions), CM-1240 (Requisitions) and CM-2100's activation cost bound. CM-2050's commandLogicVersion explicitly does not cover the AI's evaluation model or manifest (§6.3), and that exclusion should be stated where commandLogicVersion is defined so it is not inferred the other way.
4. 11-combat.md (CB) must keep the exact odds predictor at the measured cost, and must keep it evaluable over a fog-limited knowledge view. The AI using the player's own predictor removes an entire class of divergence bug and is worth protecting.
5. 14-victory.md (VC) must keep Track progress, Vigil counters and Tenure available as integers from a fog-filtered view — they are the highest-value features in §6.2 — and must keep mathematical elimination and Capitulation available as farm settings, because they are the farm's unbiased adjudicator (§3.3d, §4.6).
6. 12-economy.md (EC) must expose Integration, Levy, Reach and Cadre state as integers in the view, and should treat the farm's capture-to-Integration-50 metric as a calibration instrument for its 25–35 turn payback target.
7. 03-architecture.md (AR) must add a games/core-hour figure for the Standard tuning game to the benchmark suite (§4.3), and must allow the farm to set a sampled hashing policy (§4.7). The existing nightly farm requirement is a subset of §4.8 and should be reconciled rather than duplicated.
8. The services and privacy documents own the consent flow of §9.3; this document states the requirements the training pipeline places on it, in particular all-seats consent, client-side stripping of player-authored strings, and completed-games-only collection.

15. Open questions I could not settle

1. Is the marginal Cascade cheap enough for Rehearsal? §7 assumes a copy-on-write fork with activation memoisation brings a candidate Cascade to ~15% of a full one. That is the single load-bearing unmeasured assumption in this document, and if it is wrong — if every candidate costs a full Cascade — then K drops from 16 to 2 and Rehearsal stops being the best value here. It is measurable as soon as the Cascade exists, and it should be measured before anything else.

2. Should the optimiser see continuous Tenure margin or binary outcomes? §3.3(c) splits them by phase and argues for it, but the Goodhart risk is real and the split is a guess. The experiment that settles it is cheap: run the same SPSA configuration twice, once on each signal, and compare the resulting candidates through the same binary gate.

3. Is head-to-head the right ranking metric for a game that will mostly be played multi-seat? §3.2 argues yes on statistical grounds and I believe it, but it optimises for a mode that may not be the modal one. A hybrid gate — head-to-head for promotion, multi-seat for a veto — is what I have specified, and the veto threshold of 0.04 placement is invented.

4. How many parameters can SPSA actually move at once here? 200 is at the upper end of what the method is usually asked to do. It may be necessary to tune in blocks — economy, then military, then strategic — which introduces its own coupling errors. The first run will say.

5. Does the learned evaluator transfer across map sizes and player counts? §6.2's scale-free feature design is an attempt to make it so and is untested. If it does not transfer, we need one model per map tier, which triples the training cost and the shipping size.

6. What is the right corpus for Anchor H, and who plays it? The only non-self-referential benchmark in this document depends on having strong human games, which we will not have until after release — precisely when the AI most needs to already be good. A pre-release internal corpus from the team is better than nothing and is systematically biased toward the designers' own understanding of their game.

7. Should low tiers use behaviour cloning from human logs (§9.5) or eval blur (§10)? Cloning produces more plausible failure; blur is available immediately and needs no data. Probably both, with cloning arriving in a later release, but I do not know whether they compose or fight.

8. Is the Skirmish lab a real instance or a different game? §8.3 argues that Industry tier 1 on a small map is genuinely the same game, and units.md §4.1 supports that reading. But if the strategic content of the full game lives in the interactions the lab strips out — Cadre logistics, Reach, multi-Track victory racing — then a positive result on Skirmish would not transfer and the probe would have told us something comforting and false. That is a worse failure than a negative result and it is worth thinking about before the two weeks are spent.