How to train a model on gameplay data

How to train a model on gameplay data

Game engines can produce an unusually rich form of training data: synchronized video, player inputs, structured state, physics, rules, rewards, and outcomes. A growing world-model data category—including companies such as Worldmodeldata and Pleiada—is building around this opportunity.

But telemetry is not automatically a training set.

The useful unit is an accepted, attributable transition from an authoritative engine. For each turn, you need to know the state the engine was actually in, the move submitted from that state, whether the engine accepted it, and what happened next. Everything else—video, prompts, transcripts, rewards, and final results—must be aligned to that receipt.

That distinction determines whether a model learns the game’s dynamics or artifacts from a logging pipeline.

Start at the legal-move door

The first interface should not be “pixels in, clicks out.” Start where the game engine decides whether a move is legal.

A minimal transition records four facts for each turn: the authoritative pre-move engine state, the action submitted from that state, whether the engine accepted or rejected that action, and the resulting state at a clearly defined transition boundary.

Video and other observations can then be attached to the same receipt, before the move and after it. The observation might contain rendered frames, camera state, audio, UI state, or text visible to the player. The engine state remains authoritative. A displayed board, model prompt, or reconstructed state may omit hidden information, lag behind the simulation, or contain formatting errors.

This is why a winner file, result record, or completed transcript is not enough. Those artifacts can tell you that a run ended and perhaps who won. They do not prove that every recorded action was submitted from the stated position or accepted by the engine.

Instrument the engine boundary

Collection should happen around the point where the engine receives and resolves actions. For every attempt, record enough information to reconstruct the event without guessing:

• Run, episode, and turn identifiers
• Engine build and relevant game-content version
• The actor responsible for the attempt
• The authoritative pre-move state
• The observation shown to that actor
• The exact action submitted
• The engine’s acceptance or rejection
• The resulting state for accepted actions
• Synchronized frame or engine-tick references
• Reset, termination, timeout, and engine-failure events
• Any model or policy handoff

Use the engine’s own ordering. Wall-clock timestamps alone are often insufficient because rendering, inference, networking, and simulation may run on different clocks. Engine ticks, sequence numbers, or another monotonic ordering mechanism make it possible to align state, action, and video reliably.

The transition endpoint also needs a game-specific definition. In a turn-based title, it may be the state after the action resolves and control passes. In a real-time title, it might be the next simulation tick or a fixed action horizon. In a physics-heavy environment, an action may be accepted immediately while its consequences unfold over many steps. Define this boundary before collection rather than inferring it later from the footage.

Keep attempts distinct from accepted transitions

Models retry. Clients time out. Actions are duplicated. An agent may submit a malformed command, receive a rejection, and then submit a corrected one.

Do not collapse those attempts into a cleaned transcript.

Each attempt should have its own identity and engine response. Rejected moves should be dropped from the training set. If the model retries, the rejected attempt and the later accepted attempt remain separate events; the accepted retry can become a training transition with its own receipt.

This protects the causal structure of the data. Otherwise, a cleaning step can accidentally pair the pre-state from one attempt with the action or post-state from another.

Rejected attempts can remain in operational logs for diagnosing the collection system, but they should not be presented as valid gameplay transitions.

Make provenance visible at the turn level

A run-level label such as “generated by model A” is not enough if the actor can change during the run.

A common failure mode is a silent handoff: a smaller model times out, a backup policy takes over, or a larger model completes the run. If the final transcript attributes every action to the original model, the resulting data is misleading even when all moves were legal.

Drop runs with silent handoffs unless provenance is explicit. If handoffs are supported, record the responsible actor on each attempt and transition. The same principle applies to human interventions, scripted openings, fallback policies, and replayed actions.

Useful provenance generally includes:

• The policy, model, human, or script that produced the action
• The relevant model or policy version
• The game and engine build
• The collection configuration
• The run and turn lineage
• Whether the action was original, retried, replayed, or supplied by a fallback

The purpose is not merely auditability. Provenance determines what the model is actually learning. A corpus generated by several policies is a mixture of their state distributions and action preferences. Without attribution, that mixture cannot be inspected, reweighted, or evaluated correctly.

Build the dataset from receipts, not transcripts

A transcript is a presentation of a run. A receipt is evidence of a transition.

Construct training records by joining engine events around a stable run and turn identifier:

1. Read the authoritative pre-move state.
2. Identify the observation associated with that state.
3. Read the submitted action.
4. Confirm the engine’s acceptance.
5. Read the resulting state at the defined boundary.
6. Attach synchronized video and other observations.
7. Attach actor and build provenance.
8. Validate that no reset, failure, or unrecorded handoff crossed the transition.

Only then should the event enter the accepted-transition corpus.

Do not start with a cleaned transcript and try to reconstruct these relationships. Cleanup often removes retries, rejection events, timeouts, and intermediate states—the exact evidence needed to determine whether a transition is valid.

A final result can be joined later for outcome-conditioned or value training, but only after the underlying transition sequence has passed validation.

Exclude runs that only appear successful

A written winner is not proof of a valid episode.

Drop runs that completed through an engine failure, even if a winner or result file was produced. Examples include a simulation crash followed by a default result, a timeout resolved by an external wrapper, or a corrupted state that still reaches a terminal-writing path.

Validation should distinguish at least:

• Normal terminal states
• Explicit forfeits or rule-defined timeouts
• User or system cancellations
• Engine failures
• Collector failures
• Missing or contradictory events

Only rule-valid terminations should support terminal rewards or outcome labels. An engine failure should not become a strange but apparently successful strategy in the training data.

Choose the learning target after establishing the data contract

The same accepted-transition corpus can support several model types, but each requires a different view of the records.

Action or gameplay policies

For behavioral cloning, train the model to predict an accepted action from the corresponding pre-move state, observation, and optional goal.

Represent actions in the engine’s canonical action space where possible. Mouse coordinates and controller events may be useful observations, but they are often an indirect representation of game intent. A structured action such as selecting an entity, target, ability, or destination is easier to validate against the engine.

If the deployed model must operate through a UI, the structured action can still serve as supervision for a lower-level control layer. The important point is that the policy target is tied to an engine-accepted move.

World models

A world model learns the next state and the next observation from the current state, the current observation, and the accepted action.

Depending on the project, the target may be the next structured state, future video, a latent representation, or a sequence over several engine steps. Accepted actions are essential because rejected actions do not represent ordinary state transitions under the game’s dynamics.

Sequence windows must not cross resets, corrupted states, unexplained actor handoffs, or engine failures. For long-horizon training, retain the turn-level lineage so every window can be traced back to its component receipts.

Vision-language-action and physical-action models

For a VLA-style objective, observations may include frames, language instructions, goals, and state-derived annotations. Actions should still be aligned to the engine’s accepted control boundary.

Game engines are attractive for physical AI because they can expose geometry, contacts, object identities, velocities, and other latent variables that are difficult to obtain from real-world video. Those signals are useful only if their coordinate systems, sampling times, and relationship to the rendered observation are explicit.

Value and outcome models

Value models can use terminal outcomes, score changes, or state-derived rewards, but those labels should be propagated only through validated episodes.

Do not use a final winner as a substitute for turn validation. First establish that each transition is accepted and attributable. Then attach the outcome to the resulting sequence.

Preserve both state and observation

Engine state and player observation answer different questions.

The observation describes what the acting system could perceive. The engine state describes what the world actually contained. Training only on state may produce a model that relies on information unavailable at deployment. Training only on pixels may discard structured supervision that makes world-model learning and evaluation more reliable.

Retain both when possible:

• Use observation-only inputs when matching the deployed information boundary.
• Use engine state as a target, auxiliary signal, validator, or privileged training feature.
• Record visibility and hidden-information rules explicitly.
• Avoid replacing the pre-move engine state with a board reconstructed from screenshots or text.

This also enables sharper evaluation. A predicted frame can look plausible while representing an impossible game state. Comparing predicted structured state with the authoritative successor exposes that error.

Split by episodes and situations, not adjacent frames

Randomly splitting individual frames or turns usually causes leakage. Adjacent observations from the same run are highly correlated, and repeated scenarios may differ only cosmetically.

Create train, validation, and test splits at the run or scenario level. Depending on the game, also consider grouping by:

• Map or level
• Scenario template
• Initial state or seed family
• Opponent or generating policy
• Content or engine version

The right split depends on the intended generalization. If the model must handle unseen maps, hold out maps. If it must improve within a fixed title and build, keep the build fixed while separating episodes and initial conditions.

Report performance by provenance slice as well as in aggregate. A model can appear strong because one generating policy dominates the corpus or because near-duplicate situations appear in multiple splits.

Evaluate in the authoritative engine

Offline loss is necessary but insufficient. A model can predict actions that resemble the dataset while failing at the legal-move boundary.

Evaluation should therefore include:

• Whether submitted actions are accepted by the engine
• Whether predicted next states match authoritative successors
• Whether rollouts remain consistent over multiple steps
• Whether behavior changes across maps, builds, policies, or other provenance slices
• Whether episodes terminate through valid game rules
• Whether the evaluation actor remains the attributed actor throughout the run

Keep offline and closed-loop evaluation separate. Offline evaluation measures prediction on recorded states. Closed-loop evaluation measures the state distribution created by the model’s own actions. Errors compound in the latter, so it is the more meaningful test for an agent or world model intended to roll forward.

As with training data, a completed run is not automatically a valid evaluation. Engine failures and silent model takeovers must not be counted as successful completion.

A practical pipeline

A reliable gameplay-training pipeline can be organized into five layers:

1. Play: An agent, human, or script acts in the real game.
2. Receipt capture: The engine records pre-state, action, acceptance, successor state, and provenance.
3. Validation: The pipeline rejects invalid transitions, failed runs, missing state, and unexplained handoffs.
4. Dataset assembly: Accepted receipts are converted into policy, world-model, VLA, or value-training examples.
5. Engine evaluation: Trained models return to the same legal-move boundary for closed-loop testing.

At ReBlink, Playthrough uses studio-specific agents that play the studio’s real game and produce matches. The important design choice is the starting door: the engine must accept the move. From there, every usable example can be traced to the state the engine was really in, the action submitted from that state, and the engine’s acceptance of that action.

That is the foundation for training on gameplay data. Engines provide rich supervision, but the value comes from preserving the connection between state, action, consequence, and actor. Without that connection, telemetry is just a collection of correlated files. With it, gameplay becomes a defensible transition dataset.