Repository navigation
bug(plan-phase): the stall watch never observes real completion: it matches markers in the agent's own prompt and definition, and a revision can only end as stalled #5182
Description
Pre-submission checklist
- I have searched existing issues and this bug has not already been reported
GSD Version
1.15.0 (next @ 62549bf)
Runtime
Multiple (specify in description)
Operating System
WSL
Node.js Version
v24.13.0
Shell
/bin/bash
Installation Method
Updated from a previous version
What happened?
User impact. With the default planner.stall_detection_enabled: true, /gsd:plan-phase's wait for its background planner and plan-checker does not observe when they actually finish:
- Claude Code (reproduced): the wait reports the planner and the checker as finished at its first check, whether or not they are. A planner still working at the 5-minute mark can be treated as done, and a checker that hangs is never reported as stalled. Separately, every plan revision round, on every runtime, can only end in the "Accept as revised / Retry / Stop" stall prompt, about 10 minutes after dispatch, instead of continuing when the revision is done.
- Other runtimes (code-level, not reproduced live): the wait only sees completion through the
{outputFile}of a Claude-Code-styleAgent()result. On a host whose background dispatch returns no such file, the marker check can never succeed, so the planner, checker and revision spawns can end only asstalled(the helper's own docs already note the checker has no other working completion signal).
Where: gsd_stall_watch in gsd-core/workflows/plan-phase/steps/stall-detection-helpers.md, used for the planner, plan-checker and revision spawns (plan-phase.md steps 8, 10, 12) on all runtimes.
What did you expect?
The wait returns when the spawned agent has actually returned (with its real marker), returns stalled only when the agent stopped making progress, and behaves the same on every runtime that dispatches background agents.
The helper's design text sets this expectation: for the checker the marker path "is the ONLY working completion signal", and declaring a succeeded agent stalled is "worse than the original unbounded wait" (stall-detection-helpers.md, "Plan-checker's artifact glob needs the marker").
Steps to reproduce
Root cause (next @ 62549bf):
stall-detection-helpers.md:156-159decides completion withgrep -qF "$m" "$output_file"over the entire output file, and only when[ -f "$output_file" ].- On Claude Code that file is the subagent's JSONL transcript (
~/.claude/projects/{project}/{sessionId}/subagents/agent-{agentId}.jsonl, per the Claude Code sub-agents docs). The transcript holds the agent's inputs as well as its output:- record 1 is the prompt. The checker prompt lists its own markers (
plan-phase.md:1074-1075,- ## VERIFICATION PASSED — all checks pass); - a
prompt_snapshotattachment record (records 9-12 in every transcript below) carries the agent definition, and both definitions quote their markers:agents/gsd-planner.md:929-934("## Return Markers ... Your orchestrator dispatches on exact marker strings in your final output. Emit exactly one of: ... ## PLANNING COMPLETE") andagents/gsd-plan-checker.md:957-960.
- record 1 is the prompt. The checker prompt lists its own markers (
plan-phase.md:1189: the revision watch passes no markers and repeats whilewaiting/active;gsd_stall_should_recoverreturnsmarker_receivedonly when a marker matched, sostalledis its only exit. The helper header (lines 12-13) states the watch runs "regardless of whether the backgrounded agent's own completion notification ever arrives".- The
{outputFile}the helper depends on is not part of Claude Code's documented interface (the sub-agents docs describe only that "a background subagent's results reach Claude as a completion notification in a later turn"), and other hosts dispatch background agents through their own primitives (for Codex, GSD's conversion rules mapAgent()tospawn_agent, with completion observed throughcollaboration.wait_agent,src/runtime-artifact-conversion.cts).
Reproduction 1, deterministic (runs the shipped functions with sleep stubbed; run from a gsd-core checkout):
#!/usr/bin/env bash
set -euo pipefail
REPO="${1:-.}"; REF="${2:-origin/next}"
T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT
git -C "$REPO" show "$REF:gsd-core/workflows/plan-phase/steps/stall-detection-helpers.md" \
| awk '/^gsd_stall_should_recover\(\) \{/{f=1} f{print} f&&/^}$/{n++; if(n==2) exit}' > "$T/helpers.sh"
PLANNER_STALL_INTERVAL_MINUTES=0; PLANNER_STALL_THRESHOLD_MINUTES=10
source "$T/helpers.sh"
# 1: checker transcript holding only the prompt record (nothing returned yet)
mkdir -p "$T/phase"
printf '%s\n' '{"type":"user","message":{"role":"user","content":"... Return:\n- ## VERIFICATION PASSED — all checks pass\n- ## ISSUES FOUND — structured issue list\n"}}' > "$T/transcript.jsonl"
echo "1 -> $(gsd_stall_watch "$(date +%s)" "$T/transcript.jsonl" "$T/phase/*-PLAN.md" "## VERIFICATION PASSED" "## ISSUES FOUND")"
# 2: revision spawn (no markers); the revision finished and rewrote a PLAN.md
PLANNER_STALL_INTERVAL_MINUTES=5
echo '# plan' > "$T/phase/01-PLAN.md"
echo "2a -> $(sleep() { :; }; gsd_stall_watch "$(( $(date +%s) - 120 ))" "$T/transcript.jsonl" "$T/phase/*-PLAN.md")"
touch -d '6 minutes ago' "$T/phase/01-PLAN.md"
echo "2b -> $(sleep() { :; }; gsd_stall_watch "$(( $(date +%s) - 360 ))" "$T/transcript.jsonl" "$T/phase/*-PLAN.md")"
echo "2c -> $(sleep() { :; }; gsd_stall_watch "$(( $(date +%s) - 660 ))" "$T/transcript.jsonl" "$T/phase/*-PLAN.md")"
# 3: a host that returned no output file; the checker finished at 2 min (nothing on disk changes for a PASS)
echo "3 -> $(sleep() { :; }; gsd_stall_watch "$(( $(date +%s) - 660 ))" "$T/no-such-file" "$T/phase/none-*-PLAN.md" "## VERIFICATION PASSED" "## ISSUES FOUND")"Observed:
1 -> marker_received
2a -> active
2b -> waiting
2c -> stalled
3 -> stalled
Case 1 should be waiting. In case 2 the revision finished at 2a and the loop can only leave through 2c. Case 3 is a finished checker on a host without an output file.
Reproduction 2, live (Claude Code 2.1.288): a run_in_background=true Agent() whose prompt listed the two checker markers and whose task took about 3 minutes. Its outputFile resolved to .../subagents/agent-<id>.jsonl; line 1 was the prompt and the only line containing ## VERIFICATION PASSED. The shipped gsd_stall_watch (interval 0) returned marker_received 8 seconds after dispatch; the agent later returned ## ISSUES FOUND.
Reproduction 3, real transcripts (one machine, Claude Code 2.1.267-2.1.287, 2026-09-09 to 2026-10-02): in every stored background gsd-planner run (71) and gsd-plan-checker run (67), the agent's own marker (## PLANNING COMPLETE / ## VERIFICATION PASSED) was already in the transcript before the agent's final reply, at the latest in the prompt_snapshot record (records 9-12). 67 of the 71 planner runs lasted longer than 5 minutes (median 22.6 min) and 9 of the 67 checker runs did (median 1.1 min), so with the default 5-minute interval the first check would match while those agents were still working. (Durations and marker positions, not observed misroutes: that machine ran a locally patched watch.)
Error output / logs
None; the wrong result is a normal return value of gsd_stall_watch.
How often does this happen?
- Every time (100% reproducible)
Impact
Moderate — Feature is broken but I have a workaround
Workaround (if any)
planner.stall_detection_enabled: false (#4570): plan-phase waits through the runtime's own Agent() completion and never calls gsd_stall_watch, at the cost of #2650's automatic stall recovery.
Additional context
Context: this is the first of a few issues I am filing from my GSD Core speed pack, a set of local speed and friction changes I run on top of next (https://github.com/davesienkowski/gsd-core-speed-pack). It carries a local workaround for this bug; the fix proposed here is a different, runtime-neutral design. I will file the others one at a time, after this one is triaged.
Recommended fix: a GSD-owned return receipt, the same on every runtime.
- The orchestrator gives each watched spawn (planner, checker, revision) a receipt path under the phase dir, e.g.
${PHASE_DIR}/.gsd-returns/<spawn>-<TS>.md, in its prompt. - The agent's last action is to write its return marker line to that path.
gsd-plannerhas Write;gsd-plan-checkerhas Bash but no Write, so the checker's contract gains one small write (a maintainer call on its read-only role). gsd_stall_watchtakes the receipt path instead of{outputFile}and matches the markers only in that file. Prompts, agent definitions and transcripts can never match it, and no host output format is read.- The revision spawn gets a receipt too (
## REVISION COMPLETE/## REVISION_CONFLICT), giving it a success exit. - The runtime's own completion (Claude Code's completion notification, Codex
wait_agentterminal status) stays the primary way the orchestrator reads the agent's full return; the receipt is what the bounded watch observes. A missing receipt degrades to today's boundedstalledpath, never a hang.
Why this over the alternatives:
- It is runtime-neutral: no
runtime === …branch and no reliance on an undocumented host file, matching ADR-1239 (per-host behavior "explicit and testable rather than scatteredruntime === '…'checks"; per-host values "documentation-sourced ... never inferred"). - There is in-repo precedent for moving a return contract from a marker string to an artifact:
fix(#3440): retire GAP CLOSURE PLANS CREATED marker, document artifact return contract(fix(#3440): retire GAP CLOSURE PLANS CREATED marker, document artifact return contract #3443);agents/gsd-planner.md:912"gap-closure returns are artifact-based (fix(#1891 F9): retire the GAP CLOSURE PLANS CREATED marker — artifact route is canonical #3440)". - Considered and not recommended: (a) parsing the Claude Code transcript for the final assistant record (
stop_reason: end_turn). It fixes Claude Code only and depends on an undocumented record schema. (b) Removing the literal markers from prompts. The agent definitions are snapshotted into the transcript, so this cannot work.
Acceptance criteria:
- Markers present in a prompt, agent definition or transcript, but not in the receipt, yield
waiting. - A checker whose receipt says
## ISSUES FOUNDroutes to step 12, never step 13. - A finished revision yields a success result within one poll interval, not
stalled. - A spawn that never writes its receipt still yields
stalledafter the threshold. - The same results on Claude Code and on a host with no
{outputFile}. - Regression tests that fail on
nextfirst, intests/plan-phase-stall-detection.test.cjs.
Behavior change to disclose: the checker and planner waits no longer return at their first check, and the "Accept as revised" menu no longer follows every successful revision.
Related: origin #2650 and PR #3015 (introduced gsd_stall_watch); #4570 / PR #4585 (the disable switch); #4079 / PR #4299 (no wake-up calls between cycles); #3923 (approved: make the threshold configurable; a longer threshold only delays the revision menu); #2699 (general subagent watchdog, wontfix; this fix stays inside the existing helper and contracts).
Activity
trek-e commentedon Oct 2, 2026
Diagnosis
Reproduced: Yes. Running the shipped
gsd_stall_watchagainst a transcript containing only the prompt text## VERIFICATION PASSEDreturnsmarker_received(should bewaiting); with no output file and 11 minutes elapsed it returnsstalled. The marker-less revision watch has no success exit.Root cause:
gsd-core/workflows/plan-phase/steps/stall-detection-helpers.md:156-159(gsd_stall_watch) greps the entire output file for markers, which includes the agent's prompt and definition snapshot;:124-126(gsd_stall_should_recover) returns success only on a marker match, so the revision watch atplan-phase.md:1189can only endstalled. Call sites:plan-phase.md:958,1092,1189.Introduced by: 067a4d1, fix(#2650) (#3015).
Blast radius: plan-phase steps 8, 10, 12 (planner, checker, revision) on all runtimes with the default
planner.stall_detection_enabled: true; checker PASS/ISSUES routing and every revision round.Coupled (must not miss):
tests/plan-phase-stall-detection.test.cjs; return-marker contracts inagents/gsd-planner.mdandagents/gsd-plan-checker.md(checker has no Write tool); the spawn prompts inplan-phase.md; docs forplanner.stall_detection_enabled; related open #3923 (threshold).Regression test: markers appearing only in prompt/definition content of the watched output yield
waiting; a checker return of## ISSUES FOUNDroutes to step 12; a finished revision yields a success result within one poll interval; a spawn that never signals completion yieldsstalledafter the threshold; identical results when the host provides no output file.Agent Brief
Category: bug
Summary: The plan-phase stall watch decides completion from text that includes the agent's own prompt and definition, so it reports completion too early, and revisions can never report success.
Current behavior: The watch returns "marker received" at its first check whenever the watched output contains the marker strings from the prompt or agent definition. A revision watch with no markers can only end as stalled. On hosts with no output file, every watched spawn ends as stalled.
Desired behavior: The watch returns success only when the spawned agent has actually returned its marker, returns stalled only when the agent stops making progress, and behaves the same on every runtime that dispatches background agents.
Key interfaces: the stall watch helper and its decision function; the planner, checker and revision spawn contracts and their return markers; the orchestrator routing after the checker result.
Acceptance criteria:
nextfirst.Out of scope: a general subagent watchdog (#2699); changing the default threshold (#3923).
docs(open-gsd#5182): add changeset for the plan-phase stall-watch ret…
davesienkowski commentedon Oct 3, 2026
Fix PR: #5192. It follows the Agent Brief above, and each of the six acceptance criteria maps to a regression row in
tests/plan-phase-stall-detection.test.cjs. The new rows fail onnextfirst; the PR body has the counts.Two things worth your eye:
steps/chunked-planning-mode.mdhad the same first-check match (their prompts sayReturn: ## ... COMPLETE). The existing test also pins the helper contract at exactly five sites.read-onlysandbox for it. The PR keeps that: the checker writes its receipt with one Bashprintf, and a refused write is non-fatal. On a host that runs the checker read-only, its watch endsstalledat the threshold, as it does today, unless the runtime's completion result arrives first. The PR says plainly that AC5 is therefore met for the checker only where it can write one file. The alternative that closes the gap is Group B for the checker (Write allowed, which flips its Codex sandbox toworkspace-write). That is your call; I can add it to the PR or as a follow-up.test(open-gsd#5182): pass the unsafe phase-dir values through a file,…
fix(open-gsd#5182): watch a GSD-owned return receipt instead of the h…
docs(open-gsd#5182): add changeset for the plan-phase stall-watch ret…
test(open-gsd#5182): pass the unsafe phase-dir values through a file,…
fix(open-gsd#5182): fit plan-phase.md under its cap after the rebase …
test(open-gsd#5182): add fast-check properties for the receipt helper…
154 remaining items
fix(open-gsd#5182): pin the receipt walk's own link test with in-proj…
fix(open-gsd#5182): scope the changeset's symlink claim to what the r…
docs(open-gsd#5182): say the plan-checker receipt grant widens to ful…
fix(open-gsd#5182): drop the plan-checker's receipt Write on hosts th…
fix(open-gsd#5182): anchor the receipt containment on the project roo…
fix(open-gsd#5182): a receipt-only REVISION_CONFLICT fails closed ins…
fix(open-gsd#5182): read the receipt's byte cap boundary exactly, and…
test(open-gsd#5182): drive the freshness window with a find shim, and…
test(open-gsd#5182): regenerate the compact-content benchmark baselin…
chore(open-gsd#5182): regenerate the Pi package roles for the planner…
test(open-gsd#5182): pin the receipt-Write scoping table against what…
test(open-gsd#5182): pin the checker's scoped edit posture on global …
fix(open-gsd#5182): fail closed on a receipt-only VERIFICATION PASSED…
fix(open-gsd#5182): point a body-less VERIFICATION PASSED at 11a in s…
fix(open-gsd#5182): give 11a its own case for a receipt-routed checke…