Capture the lead-up before the alert
Automated detection often happens after the important transition. A navigation alert may fire only when the agent has been motionless for a while. A crash handler runs after the process has already failed. If collection starts at that point, the interaction that triggered the problem may be missing. Use bounded rolling buffers for relevant inputs, events and state summaries.
Choose the buffer length based on the suspected failure mechanism and available resources. A short visual glitch and a long quest chain need different evidence. Retain enough history to understand the transition, but avoid recording everything indefinitely. Capture overhead can alter timing, consume storage and expose information unrelated to the investigation.
Identify the exact thing that ran
A branch name is not a reliable build identifier because its contents change. Record an immutable revision, artifact identifier or content hash, together with the build configuration. Include engine version, target platform, graphics settings, installed content and relevant feature flags. If a test harness or learned agent influenced the run, record its version and configuration too.
Git and Perforce Helix Core can help identify source revisions, but a source revision alone may not identify downloaded assets, configuration services or generated content. Connect those dependencies to the report. A developer should not have to guess whether the evidence came from a development build with cheats enabled or a release-like configuration.
Assemble a compact evidence bundle
Environment
Build, platform, configuration, initial save and relevant hardware or network conditions.
Sequence
Actions, checkpoints and timestamps leading to the observed failure.
Observation
Expected behavior, actual behavior, logs and a short visual capture where useful.
Replay record
Attempts, outcomes and changed conditions, including attempts that did not reproduce.
Do not confuse replay with determinism
Recording the same buttons does not guarantee the same execution. Input timing may depend on frame rate. Physics can diverge across platforms. Thread scheduling, streaming and network delivery introduce additional uncertainty. A fixed random seed helps only with the random systems that actually use that seed and preserve the same call order.
State snapshots can reduce setup cost, but they may omit external services, operating-system state or transient resources. Explain what a checkpoint restores and what it does not. If a snapshot is incompatible with a newer build, keep the original build available under the studio's retention rules rather than assuming the replay remains portable.
Illustrative evidence statement
“The character remained inside the moving platform after the dismount action. Replay succeeded only when dismount occurred during the direction change. Other timings did not reproduce the overlap.” This describes the observed boundary without inventing a cause or a success rate.
Reduce the sequence carefully
A long agent run is difficult to inspect. Try removing actions or replacing early navigation with a known checkpoint while keeping the original trace intact. This is related to delta debugging: reduce the input while testing whether the failure remains. The result is a smaller investigation surface, not proof that every omitted action was irrelevant in all environments.
Change one meaningful condition at a time when diagnosing sensitivity. Compare timing, hardware or content variants explicitly. A clean replay on another machine does not invalidate the original report; it narrows the circumstances. For intermittent failures, keep a ledger of attempts and outcomes so later reviewers can distinguish a stable reproduction from an occasional observation.
Protect the evidence and the people in it
Crash dumps, logs and saves can contain account identifiers, chat messages, tokens or unreleased content. Minimize collection, restrict access and define retention before sending captures to external systems. Redaction must not silently remove details that are essential to interpreting the evidence. Keep an internal record of what was changed and why.
A report is ready for triage when another person can understand the expectation, retrieve the right environment and attempt the documented path. It does not need a confident root-cause story. An honest boundary around uncertainty is more useful than a polished explanation that the evidence cannot support.
Methodology note: distinguish recovery from reproduction
A workaround that restores play is not a reproduction step. Store recovery actions separately so they do not overwrite the failing state. When possible, duplicate the save or snapshot before applying a workaround.
Primary references to inspect
- Git revision identification — resolving source revision names.
- Perforce Helix Core — version management used in development pipelines.
- Crashpad — crash-report collection infrastructure.
- Mozilla rr — record-and-replay debugging, with platform and workload limits.
- Andreas Zeller's Delta Debugging resources — systematic reduction of failure-inducing inputs.
- Information Commissioner's Office: data minimisation — limiting personal data to what is necessary.