Add a seeded automated playtester and a read-only playtest state export #20

Merged
ValleyGeek merged 2 commits from feat/playtest-bot into main 2026-11-07 17:34:21 +00:00
Member

Adds an automated playtester under tools/playtest/ and one flag-gated, read-only state export for it.

What it does

bash tools/playtest/run.sh --seeds 1,2,3 plays a new game per seed: title → opening → creation → overworld grind for the gate fee → Calwen (guild quest, inn, merchant) → Green Stair floors 1–10 → the Woldbeast. It acts only through InputEventKey and mouse events, at --fixed-fps 60 with Engine.time_scale 4 (time is accelerated 4x; noted in Technical Outline §11.1). Each run writes playtest_out/<stamp>/seed-<n>/report.json, report.md, godot.log, per-finding snapshots under events/, and Xvfb screenshots under shots/. playtest_out/ is gitignored.

  • bot.gd: SceneTree main loop, input, a Logger for engine and script errors, and the reports
  • brain.gd: the policy. It reads live state, finds paths with BFS over exported walkability, and moves focus with the arrow keys. It has no fixed coordinates or routes. In battle it heals with herbs or Heal when HP is low, targets the weakest foe, and attacks.
  • monitor.gd: detects script errors, softlocks, scene/mode mismatches, panels outside the viewport, out-of-range values, battle anomalies, missing damage feedback, frame-time hitches, and spec checks (gate fee, early encounters, inn, shop, chests, quest turn-in, wipe placement, boss phase, entrance circles)
  • run.sh refuses to run if a bot source names a game singleton (GameManager, ContentDb, …), so all reads go through PlaytestState

Game-code change (the hook)

  • scripts/debug/playtest_state.gd (PlaytestState) is active only under the user arg --playtest or ANIMRPG_PLAYTEST=1. Without the flag, every call returns {} or [], and the new test tests/test_playtest_state.gd checks this. With the flag it returns copies of the state (position, menus with screen rects, battle bodies, doors, stairs, tile walkability). It writes nothing, moves nothing, and grants nothing.
  • scripts/autoload/game_manager.gd: two lines. Under the same flag, --playtest-seed=N seeds the GameManager RNG after randomize(), so a seed replays the same session. Normal play is unchanged. Please review this one deliberately: it is the only line in a gameplay file.

Versioning

No version step. VERSION stays 0.4.4, the formal version on main. The squash publishes no tag.

Results (against 0.4.2, before the 0.4.3 and 0.4.4 squashes)

On seeds 1–4 the bot defeated the Woldbeast each time, with one wipe per seed at the floor-10 boss, 15.3k–16.5k frames, and about 4.5 minutes each under Xvfb with llvmpipe. Those runs predate the pack art and the window scale. Findings were filed as playtest issues and were not fixed here:

  • #21 World view calls set_input_as_handled on a null viewport after camp or stairs start a battle
  • #22 Battles stop showing damage numbers, sounds, and shake after the log reaches 8 lines
  • #23 After a wipe the hero wakes at the Calwen gate, not at the healer
  • #24 Inn rest can leave HP below max when the rest raises the HP skill
  • #25 Bosses transform at half HP, but Technical Outline says 0 HP (spec mismatch, intent unconfirmed)
  • #26 Green Stair entrance has no teleport circle (spec mismatch)
  • #27 Magic page shows spell cost as 3.0 MP

Frame-time hitches show up under Xvfb with llvmpipe: 35 frames over 50 ms and over 3x the median on seed 4, worst 75 ms. Software rendering on a shared box likely causes these, so I didn't file them.

Checks on the rebase onto ba8f789: tools/lint_gdscript.sh clean, tools/run_tests.sh 33/33.

Adds an automated playtester under `tools/playtest/` and one flag-gated, read-only state export for it. ## What it does `bash tools/playtest/run.sh --seeds 1,2,3` plays a new game per seed: title → opening → creation → overworld grind for the gate fee → Calwen (guild quest, inn, merchant) → Green Stair floors 1–10 → the Woldbeast. It acts only through `InputEventKey` and mouse events, at `--fixed-fps 60` with `Engine.time_scale` 4 (time is accelerated 4x; noted in Technical Outline §11.1). Each run writes `playtest_out/<stamp>/seed-<n>/report.json`, `report.md`, `godot.log`, per-finding snapshots under `events/`, and Xvfb screenshots under `shots/`. `playtest_out/` is gitignored. - `bot.gd`: SceneTree main loop, input, a `Logger` for engine and script errors, and the reports - `brain.gd`: the policy. It reads live state, finds paths with BFS over exported walkability, and moves focus with the arrow keys. It has no fixed coordinates or routes. In battle it heals with herbs or Heal when HP is low, targets the weakest foe, and attacks. - `monitor.gd`: detects script errors, softlocks, scene/mode mismatches, panels outside the viewport, out-of-range values, battle anomalies, missing damage feedback, frame-time hitches, and spec checks (gate fee, early encounters, inn, shop, chests, quest turn-in, wipe placement, boss phase, entrance circles) - `run.sh` refuses to run if a bot source names a game singleton (`GameManager`, `ContentDb`, …), so all reads go through `PlaytestState` ## Game-code change (the hook) - `scripts/debug/playtest_state.gd` (`PlaytestState`) is active only under the user arg `--playtest` or `ANIMRPG_PLAYTEST=1`. Without the flag, every call returns `{}` or `[]`, and the new test `tests/test_playtest_state.gd` checks this. With the flag it returns **copies** of the state (position, menus with screen rects, battle bodies, doors, stairs, tile walkability). It writes nothing, moves nothing, and grants nothing. - `scripts/autoload/game_manager.gd`: two lines. Under the same flag, `--playtest-seed=N` seeds the GameManager RNG after `randomize()`, so a seed replays the same session. Normal play is unchanged. Please review this one deliberately: it is the only line in a gameplay file. ## Versioning No version step. `VERSION` stays `0.4.4`, the formal version on `main`. The squash publishes no tag. ## Results (against 0.4.2, before the 0.4.3 and 0.4.4 squashes) On seeds 1–4 the bot defeated the Woldbeast each time, with one wipe per seed at the floor-10 boss, 15.3k–16.5k frames, and about 4.5 minutes each under Xvfb with llvmpipe. Those runs predate the pack art and the window scale. Findings were filed as `playtest` issues and were not fixed here: - #21 World view calls `set_input_as_handled` on a null viewport after camp or stairs start a battle - #22 Battles stop showing damage numbers, sounds, and shake after the log reaches 8 lines - #23 After a wipe the hero wakes at the Calwen gate, not at the healer - #24 Inn rest can leave HP below max when the rest raises the HP skill - #25 Bosses transform at half HP, but Technical Outline says 0 HP (spec mismatch, intent unconfirmed) - #26 Green Stair entrance has no teleport circle (spec mismatch) - #27 Magic page shows spell cost as `3.0 MP` Frame-time hitches show up under Xvfb with llvmpipe: 35 frames over 50 ms and over 3x the median on seed 4, worst 75 ms. Software rendering on a shared box likely causes these, so I didn't file them. Checks on the rebase onto `ba8f789`: `tools/lint_gdscript.sh` clean, `tools/run_tests.sh` 33/33.
Add a seeded automated playtester and a flag-gated read-only state export.
All checks were successful
CI / test (push) Successful in 14s
CI / lint (push) Successful in 25s
CI / test (pull_request) Successful in 16s
CI / lint (pull_request) Successful in 26s
CI / release (pull_request) Has been skipped
CI / release (push) Successful in 1m19s
e03b0e81f6
tools/playtest/ plays a new game from the title through Calwen and Green Stair
to the Woldbeast with keyboard and mouse input events only, and writes
report.json, report.md, event snapshots, screenshots, and log excerpts under
playtest_out/<stamp>/. Before this, play probes covered only single screens.

scripts/debug/playtest_state.gd exports copies of state for the bot, so it
reads positions, menus, and battles without parsing pixels. It answers only
under --playtest or ANIMRPG_PLAYTEST=1 and returns empty results otherwise.
GameManager seeds its RNG from --playtest-seed under the same flag, which
replaces randomize() for that run only, so a seed replays the same session.
Keep Godot from importing playtest output.
All checks were successful
CI / test (push) Successful in 13s
CI / test (pull_request) Successful in 17s
CI / lint (push) Successful in 27s
CI / release (push) Successful in 5s
CI / lint (pull_request) Successful in 24s
CI / release (pull_request) Has been skipped
95692e48c8
run.sh now writes playtest_out/.gdignore. Before, the editor import pass
treated each run's screenshots as project textures.
Grok force-pushed feat/playtest-bot from 95692e48c8
All checks were successful
CI / test (push) Successful in 13s
CI / test (pull_request) Successful in 17s
CI / lint (push) Successful in 27s
CI / release (push) Successful in 5s
CI / lint (pull_request) Successful in 24s
CI / release (pull_request) Has been skipped
to ae53503c41
All checks were successful
CI / test (push) Successful in 15s
CI / test (pull_request) Successful in 17s
CI / lint (push) Successful in 29s
CI / lint (pull_request) Successful in 26s
CI / release (pull_request) Has been skipped
CI / release (push) Successful in 1m23s
2026-11-07 17:33:13 +00:00
Compare
ValleyGeek deleted branch feat/playtest-bot 2026-11-07 17:34:21 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
ai-collab/animrpg202610!20
No description provided.