Skip to content

feat(bin): safely retire task records without teardown - #2466

Open
coreldh wants to merge 9 commits into
kunchenguid:mainfrom
coreldh:fm/c0816-fm-retire-followups
Open

feat(bin): safely retire task records without teardown#2466
coreldh wants to merge 9 commits into
kunchenguid:mainfrom
coreldh:fm/c0816-fm-retire-followups

Conversation

@coreldh

@coreldh coreldh commented Aug 16, 2026

Copy link
Copy Markdown

Intent

Close F1 and F2 from the independent record-retirement enforcement gate on top of PR #2458, and add focused behavioral coverage for F3, F4, and F5. Prove all five previously uncovered marker_active call sites discriminate by reverting each guard and observing RED, then restore GREEN. Stop re-deriving seen and heartbeat surfaced paths by making the producer-owned path functions the single spelling used by retirement. Individually close the four initially surviving planning mutants: F2-08 producer-path omission, F4-04 target-window normalization bypass, F4-05 competing-record normalization bypass, and F4-06 artifact-window normalization bypass; all four must be named in the PR body with their one-substitution RED verdicts and zero harness-class exits. Preserve the hard boundary: never execute fm-record-retire.sh, never run any test that invokes retirement, and touch no real state record or pooled worktree copy; safe focused library, watcher, spawn, and session-start fixtures are allowed only in scratch homes. Never weaken a property. Publish an open HTTPS pull request to kunchenguid/firstmate without merging. Upstream has zero configured checks, so treat the completed local pipeline gate as delivery evidence and do not wait for maintainer-controlled checks.

What Changed

  • Add state-only task-record retirement with explicit scout-report or remote-ref safety proofs, runtime-slot collision checks, durable markers, and no endpoint or working-copy teardown.
  • Make marker-aware decisions, status presentation, wakes, watchers, session start, and fresh spawns preserve live supervision; replacement metadata is published atomically before valid inherited markers are cleared, including fail-closed partial-bin compatibility.
  • Centralize producer-owned seen, heartbeat, and watcher paths and add focused F1–F5 regression coverage. F2-08 producer-path omission, F4-04 target-window normalization bypass, F4-05 competing-record normalization bypass, and F4-06 artifact-window normalization bypass each reached RED with one substitution and 0 harness-class exits.

Risk Assessment

✅ Low: Captain, both prior defects are closed at their shared boundaries: metadata publication is atomic and marker-safe, while partial-bin spawns preserve marker-free compatibility and fail closed on existing markers.

Testing

Repository and safety preflight, focused library fixtures, watcher/spawn/session-start behavioral suites, a direct live-versus-retired CLI demonstration, and six counterfactual RED substitutions all succeeded; evidence is transcript-based because this is shell/CLI behavior with no rendered UI, and the prohibited retirement executable and real fleet state were never touched.

Evidence: Live, retired, and counterfactual RED behavior demo
LIVE MARKER PLUS META
demo.record	demo	needs-decision	captain action remains visible
snapshot: demo.record	58	16777231:136754275
captain scan: /var/folders/cq/xf4qcb9j0qzc2dbh173mflbm0000gn/T/no-mistakes-evidence/01M04NGZ6P9JEWX6C3N5SF6SB1/demo-state/state/demo.record.status	demo.record	needs-decision: [key=demo] captain action remains visible
queued wake: 1786865088	1	check	/var/folders/cq/xf4qcb9j0qzc2dbh173mflbm0000gn/T/no-mistakes-evidence/01M04NGZ6P9JEWX6C3N5SF6SB1/demo-state/state/demo.record.check.sh	check: demo

RETIRED MARKER AFTER META REMOVAL
open decisions: <>
snapshot: <>
captain scan: <>
queued wake bytes: 0

COUNTERFACTUAL REVERT TO marker_valid WHILE META EXISTS
full with reverted guard: <> (RED: live surface hidden)
incremental with reverted guard: <> (RED: live surface hidden)
snapshot with reverted guard: <> (RED: live surface hidden)
captain with reverted guard: <> (RED: live surface hidden)
wake append with reverted guard bytes: 0 (RED: live wake hidden)
watcher signal scan with reverted guard: <> (RED: live signal hidden)
Evidence: Focused marker-library transcript
CASE test_retired_marker_hides_full_open_decision_scan
ok - full open-decision scans omit validly retired tasks
CASE test_retired_marker_hides_incremental_open_decision_scan
ok - incremental open-decision scans omit validly retired tasks
CASE test_retired_marker_hides_status_snapshot
ok - status presentation snapshots omit validly retired tasks
CASE test_live_metadata_keeps_marker_consumers_audible
ok - live metadata keeps full decisions, snapshots, and check wakes audible after marker publication
CASE test_surface_artifact_paths_are_producer_owned
ok - retirement obtains seen and heartbeat surface paths from their producers
CASE test_watcher_key_collapses_every_occurrence
ok - watcher keys fully collapse and classify raw and collapsed runtime-slot collisions
CASE test_muted_wake_releases_queue_lock
ok - muted wakes release the queue lock before returning
Evidence: Watcher and heartbeat transcript
ok - scan_signals keeps a marker-bearing task audible while canonical metadata exists
ok - heartbeat surfacing writes through the shared punctuation-collapsing path producer
ok - watcher heartbeat reader and writer share the punctuation-collapsing path producer
ok - handle_push_transition: a blocked crew enqueues a stale wake naming its window and wakes the supervisor
ok - handle_push_transition: enqueue failure cannot commit the Herdr dedupe marker
ok - handle_push_transition: a declared-pause crew is absorbed (no fast wake), left to the poll loop's long cadence
ok - event_wait_or_sleep: herdr windows go on the event pane list, but kind=secondmate endpoints are excluded
ok - event_wait_or_sleep: one cached capability probe owns validation across bounded waits
ok - event_wait_or_sleep: a home with no push-capable window is inert (sleeps POLL, never touches the event path)
ok - event_wait_or_sleep: consecutive event-path failures disable the fast-path and revert to pure polling (fail-closed)
# fm-supervision-events.test.sh: all assertions passed
Evidence: Spawn marker lifecycle transcript
ok - no --model/--effort records defaults and types the claude launch instructions
ok - non-cursor launches clear inherited Cursor identity markers
ok - relative home overrides ignore CDPATH and become absolute before spawn launch construction
ok - FM_HOME defaults resolve relative paths and preserve absolute spellings
ok - absolute override spellings are preserved in spawn launch paths
ok - unresolvable relative spawn overrides fail with named diagnostics
ok - active crew-dispatch profile requires an explicit harness for ship spawns
ok - active crew-dispatch profile requires an explicit harness for scout spawns
ok - active crew-dispatch profile allows an explicit resolved harness
ok - active crew-dispatch profile allows the legacy positional harness form
ok - active crew-dispatch profile allows the raw launch-command escape hatch
ok - claude receives --model and --effort profile flags
ok - codex receives --model and model_reasoning_effort profile flags
ok - codex omits unsupported max effort instead of passing a bad config value
ok - grok receives --model and --reasoning-effort profile flags
ok - grok omits unsupported max reasoning effort
ok - grok omits unsupported xhigh reasoning effort
ok - cursor receives its model-qualified reasoning class and exact task workspace
ok - cursor refuses model ids absent from its resolved binary's live catalog
ok - cursor preserves the requested model when its live catalog is unreachable
ok - opencode receives --model and omits the unsupported effort axis
ok - pi receives --model and --thinking max profile flags
ok - Pi launch probing omits --tui-mode on older Pi and preserves it on supporting Pi
ok - pi-signed shares Pi launch semantics while preserving its configured and recorded identity
ok - pi-signed refuses safely and actionably when the selected executable is unavailable
ok - pi-signed is a distinct persistent secondmate runtime with shared Pi supervision semantics
ok - batch dispatch forwards shared --harness, --model, and --effort to every pair
ok - claude forwards firstmate's CLAUDE_CONFIG_DIR so the crewmate uses the same credential store
ok - claude omits the config-dir prefix when firstmate runs with the single-store default
ok - non-claude harnesses do not receive the claude CLAUDE_CONFIG_DIR prefix
ok - active crew-dispatch profile does not block secondmate launches
ok - partial-bin spawns permit no marker and preserve regular or symlink markers
ok - fresh spawn clears a valid retirement marker and refuses an invalid one
# all fm-spawn-dispatch-profile tests passed
Evidence: Session-start orphan visibility transcript
ok - context digest distinguishes ABSENT, empty-but-present, and populated files
ok - a lock refusal prints a loud read-only banner, skips every mutating step, and still completes the digest
ok - session start stays read-only when lock ownership cannot be published
ok - locked session start freezes trace context and lock refusal leaves it unchanged
ok - concurrent session-lock acquisition admits exactly one live harness
ok - digest sections are ordered safety-preamble first, live fleet state before curated memory
ok - the read-once contract is stated once, ahead of the sources it governs
ok - session start: configured and auto-detected Herdr homes never require tmux
ok - session start: an absent recorded tmux window relaunches its Pi secondmate exactly once, off the blocking path
ok - session start: a deferred relaunch is always reported, so the digest's stale endpoint record cannot stand
ok - session start: an unreachable host delays a reported check, not the digest
ok - session start: a deferred result the digest outran still reaches the agent as a wake
ok - session start: a read-only session declares its skipped network checks rather than dropping them
ok - session start: the tasks-axi compatibility verdict is computed once and reused
ok - session start: an existing ambiguous Pi process prevents duplicate recovery
ok - session start: transient tmux unreadability never licenses a relaunch
ok - session start: the proven bare-shell recovery path remains intact
ok - session start: a confirmed Herdr husk is closed and relaunched
ok - status tail is bounded to the configured line count, with the full log path always printed
ok - status tail lines are capped with a truncation marker while the full log stays reachable
ok - orphan status logs are printed once with bounded tails
ok - session start keeps marker-bearing orphan status visible while a metadata symlink remains
ok - tmux endpoint liveness is reported per task: alive for a live window, dead for a gone one
ok - herdr endpoint liveness is reported per task: alive for a live pane, dead for a gone one
ok - fm-session-start.sh composes the real fm-lock.sh, fm-bootstrap.sh, and fm-wake-drain.sh output verbatim
ok - compatible tasks-axi backlog rendering drops done rows and keeps every in-flight, held, and blocked row
ok - the startup backlog bound cuts only dispatchable queued rows and discloses the remainder exactly
ok - manual backlog rendering drops done rows, keeps every held or blocked title line, and bounds the rest
ok - unavailable or incompatible tasks-axi falls back to compact manual backlog rendering
ok - an empty fleet reports (none) for in-flight tasks and an absent AFK flag
ok - session start emits X-mode cadence guidance in the harness supervision block
ok - next step delegates watcher ownership to the AFK daemon
ok - session start emits exactly one detected harness block and reports Pi extension load state
ok - session start preserves pi-signed primary identity while applying Pi extension guarantees
ok - session start rejects stale Pi loaded markers
ok - session start accepts current Pi markers written before lock acquisition
ok - session start rejects Pi sessions missing the turn-end guard marker
ok - session start rejects Pi loaded markers from previous sessions
ok - the pure-Bash watchdog bounds session start, kills its hung grandchild, and emits the truncation contract
ok - the portable timeout path force-kills a command that ignores TERM
ok - a session start inside its budget prints no truncation banner
ok - the runtime bound leaves enough ancestry headroom for a deeply nested session to take the lock
ok - --reemit reprints the digest without repeating startup's mutating sweeps and still drains queued wakes
ok - true-start AGENTS baselines stay immutable while every drifted Pi compact re-emits the current contract
ok - read-only Pi compact refreshes against the rebuilding session identity without mutation
ok - Codex reset sources do not claim an unavailable instruction-refresh channel
ok - instruction baselines require SHA-256 and successful startup completion
ok - --reemit re-verifies lock ownership and keeps repair ownership with whoever holds it
# fm-session-start.test.sh: all assertions passed

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed (2) ✅
  • 🚨 bin/fm-spawn.sh:2628 - The fresh-spawn path removes the retirement marker before replacement metadata is safely published. If the process is interrupted or the metadata write at line 2693 fails, no metadata and no marker remain, so late writes from the deliberately untouched retired agent become actionable again. Keep the validated marker through an atomic metadata publication, then clear it under the existing lifecycle locks only after the complete canonical record exists.

🔧 Fix: Captain, preserve retirement markers through atomic metadata publication
1 error still open:

  • 🚨 bin/fm-spawn.sh:2627 - The atomic-publication fix now calls fm_record_retire_marker_path and fm_record_retire_marker_validate_for_spawn directly, but fm-wake-lib.sh's supported partial-bin fallback defines neither. A recovery copy without fm-record-retire-lib.sh therefore aborts every spawn here with command not found, even when no marker exists. Extend the shared fallback with path resolution and fail-closed prevalidation, preserving its rule that absent support permits marker-free spawns but refuses any existing marker.

🔧 Fix: Captain, complete partial-bin retirement marker compatibility
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • git diff --stat ef35d799a846d676c2fd30b1d1e3ed47b0fb2c22..252ceef80ac9310ba4421613de31c913496fe845 and targeted call-site/diff inspection
  • FM_RECORD_RETIRE_TEST_ONLY=&lt;marker-safe selector&gt; bash tests/fm-record-retire.test.sh for full and incremental decision scans, status snapshots, live-metadata audibility, producer-owned paths, normalization collisions, and queue-lock release
  • bash tests/fm-supervision-events.test.sh
  • bash tests/fm-spawn-dispatch-profile.test.sh
  • bash tests/fm-session-start.test.sh
  • Manual scratch-home CLI probe comparing marker-plus-live-metadata behavior against post-metadata-removal retirement behavior
  • Counterfactual marker_active -&gt; marker_valid substitutions for full, incremental, snapshot, captain-scan, wake-append, and watcher-signal surfaces; each produced the expected RED hidden-live-state verdict
  • git status --short after testing
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant