Build the sim, reuse the intelligence

The balance agents improved every sprint. Almost none of that showed up in the co-pilot.

That's a common shape inside studios. One group builds bots for balance, regression, and training. Another builds opponents, co-pilots, and new ways for players to give orders. Both work against the same rules. Both get filed under AI. They almost never act through the game the same way.

The org chart makes that split feel normal. Eval sits with QA and ML. Player modes sit with design and client. The expensive part isn't only the duplicated code. It's that learning doesn't transfer. Headless runs don't improve what players touch. Player sessions don't sharpen the agents you use to stress the economy.

Why the stacks diverge

The separation is not laziness. It followed from how each stack was justified when compute, latency, and tooling looked different.

Evaluation wanted scale and trust in the numbers. Teams reached for vision proxies, scripted bots, or simplified sims so they could run thousands of games without driving the full client. Player AI wanted something responsive that felt usable. Teams wired models to UI flows, hotkeys, or special-case parsers that only existed in the shipped build.

Each choice made sense under its own constraint.

Together they produced agents that do not speak the same language as the game.

An eval bot that reacts to pixels or a stripped state dump is not proposing the same moves a human can make. A player-facing agent that clicks menus is not exercising the same constraints the rules engine uses. Eval cares about win rates and whether a fallback contaminated the run. Player modes care about latency and whether language maps to something the engine will accept.

Once the interfaces differ, learning cannot transfer. You cannot promote an eval policy into a co-pilot if the co-pilot must emit valid engine actions and the eval policy never had to. You cannot use live player transcripts to improve balance agents if those agents never saw the same action space.

The org ends up paying twice for intelligence that should compound.

What if they were the same loop?

Two constraints loosened at once.

Models got good enough at planning over structured state when the action space is discrete and constrained. Inference got cheap and fast enough that an agent can inspect state, propose a plan, check every step against legal moves, and repair failures inside a turn budget players will tolerate.

That combination makes a shared interface practical.

If the agent always acts through the real engine's legal actions, headless simulation and player-facing modes stop being different products. They become different schedules and different surfaces over the same loop: read a command or goal, inspect state, plan, emit atomic actions, validate, repair, commit.

The engine stays the authority.

You do not substitute a vision proxy for "what happened." You ask the same ruleset that ships to players. Statistics still tell you how the run went. Errors still tell you whether you can believe them. That discipline matters in QA. It also matters when a player says "push the left flank" and expects a legal sequence rather than a confident guess.

Map, build, ship, keep learning

The useful investment order is structural. Feature-shaped spending comes later.

Before you train anything, map the game. Expose state the agent can reason over and the legal-action surface the engine already enforces: cards, positioning, movement, terrain, status, abilities, sequencing, whatever your ruleset actually checks. Mapping is unglamorous. It is also the work that later lets evaluation and experience share the same agents. If the agent cannot see or propose what the engine considers legal, you invent a second game in software, and that second game drifts.

Then build and evaluate against that map. Run scripted, RL, and language-model agents on identical legal moves. Compare win rates with intervals. Re-run prior games against a new engine for regression. Ask results in plain language when production, QA, design, or ML needs an answer, and keep a path from every claim back to the game that produced it. Contaminated metrics from provider errors, fallbacks, and engine exceptions are first-class failures rather than footnotes.

Then choose the player experience. The same planning loop can become conversational control, an onboarding coach, a sub-unit commander that eats grind, or an adaptive opponent. Language becomes input: text or voice mapped into constraint-aware actions. Core mode stays. The new surface serves players who want the depth without the interface the original design assumed. Mobile companions become feasible when language abstracts the parts that used to require dense UI.

Then keep learning from play. Headless volume and live sessions should improve the same models and the same validators. Balance findings from large sims should change what the co-pilot suggests. Failures and repairs from player language should harden the agent you use for evaluation.

When those feedback loops share an interface, investment compounds instead of forking.

Evidence from a hard ruleset

We explored this with Arena Tactics.

Once you combine cards, positioning, movement, terrain, status effects, abilities, and sequencing, the number of possible game states sits in the septillions. We did not want AI to make that game simpler. We wanted a single agent loop that could stress the rules at scale and also help a player express intent through the same legal actions.

The loop is explicit: read the command, inspect state, plan, break the plan into atomic actions, validate each against legal constraints, repair when validation fails. That is the same loop you want in forensic QA and in a live language mode. It supports conversational play, adaptive opponents, and intelligent co-play.

Because language could abstract some of the most interface-heavy parts of the game, we were able to build a mobile companion experience in roughly a month. The game didn't become smaller to fit the device. The interface became more expressive.

Latency mattered too, though that is a longer story elsewhere. Early versions could take roughly fourteen seconds to respond. Technically the system worked. As a game, it didn't. After we rebuilt around a multi-agent architecture and faster inference, AI-powered modes went from less than 5% of gameplay to more than 60%. Speed did not invent the architecture. It made the shared loop usable for players as well as for batch evaluation.

Two returns from one architecture

When evaluation and experience share a legal-action interface, you stop choosing which AI program gets the better model. Improvements to planning, validation, and repair show up in overnight balance runs and in the mode a new player uses on their phone.

We built Playthrough and Playable around that idea. Playthrough is the high-scale simulation and interrogation loop. Playable is the player-facing conversational and co-play surface. They are two returns of one architecture rather than two products that happen to share a brand.

For a studio CTO or AI lead, the useful question is whether agents will act through the same legal surface the shipped game uses.

If they will, the sim you build to trust the ruleset can become the intelligence players reuse.

If they will not, you keep paying for two stacks that look related in the org chart and refuse to transfer learning in production.

If you are already running separate eval bots and player AI, map where their interfaces differ. The cheapest place to start is usually the legal-action boundary, before another model swap.