Skip to content

fix: never build a registry write on a read that failed - #59

Merged
ndr-ds merged 1 commit into
mainfrom
a-failed-registry-read-is-not-an-empty-registry
Aug 18, 2026
Merged

ndr-ds merged 1 commit into
mainfrom
a-failed-registry-read-is-not-an-empty-registry

Conversation

@ndr-ds

@ndr-ds ndr-ds commented Aug 17, 2026

Copy link
Copy Markdown

The half of the 18:22 wipe that #58 did not fix

#58 stopped the snapshot (sandboxes.json) from publishing an observation it could not place. It did nothing for the registry (sandbox-sessions.json), which is the file that actually lost 41 sessions — the whole linera-agent-09bfb4b7c635 key vanished at 18:22:26Z, one minute before the snapshot went empty.

sandbox-sessions.json   20702 bytes, 42 sessions across 2 sandboxes
                    ->    575 bytes,  1 session  across 1 sandbox

Root cause

pub fn load() -> Registry {
    match fs::read(sandbox_registry_path()) {
        Ok(bytes) => serde_json::from_slice(&bytes).unwrap_or_default(),  // parse error -> EMPTY
        Err(_) => Registry::default(),                                    // read error  -> EMPTY
    }
}

Both failure modes are indistinguishable from "nothing is recorded", and replace_sandbox_slice builds a whole-file write on top of that read (sandbox_registry.rs:686). So one unreadable read, while any hook saves its own slice, serialises that slice as the entire registry and deletes every other sandbox from disk.

The read failing is not itself surprising and cannot be prevented here: the file is rewritten in place by the host collector while in-sandbox hooks read it over virtiofs, where fresh metadata over stale pages is a known hazard on this setup, and half of a 20 KB JSON does not parse. Publishing a write built on that read is the bug.

Change

try_load() keeps the difference between the two cases:

  • file absent ⇒ Ok(empty) — a first run, and empty is the honest answer
  • read error or parse error ⇒ Err — we do not know what is recorded

load() stays as-is for read-only callers (renderers, tests) and is now try_load().unwrap_or_default(). All three read-modify-write paths — replace_sandbox_slice, forget_session, mark_session_departed — use try_load()? and abandon the write rather than publish one built on a base they never read.

Tests

regression_an_unreadable_registry_never_deletes_another_sandbox replays the incident: two sandboxes recorded, the file truncated to half its bytes (a torn read), then a write for sandbox A. The write must fail, the file must be byte-identical afterwards, and once the file is readable again sandbox B must still be there.

Verified it fails without the fix, not merely that it passes with it — with try_load()? reverted to load() in replace_sandbox_slice the assertion trips at the surviving-key check. A test that passes against the unfixed code is how this class of bug keeps shipping, so it was checked both ways.

a_registry_that_has_never_been_written_is_legitimately_empty pins the other half: absence is a real answer, and treating a first run as a failure would mean no sandbox could ever record its first session.

cargo test 725/726. The one failure, perf_do_refresh_io_does_not_explode_with_repeated_calls, fails identically on unmodified origin/main in this same worktree (a full-suite control run: main 722 passed / 2 failed, branch 725 passed / 1 failed) — it reads the real ~/.claude, which on this box holds 36 live sessions with multi-MB transcripts on virtiofs, against a 5 s bound. usage_ledger has zero references to sandbox_registry. cargo clippy --all-targets -- -D warnings and cargo fmt --check clean.

The pattern, for the third time today

Every one is not observed recorded as not there. live_claude_procs() already states the rule for ps — "callers must degrade to their previous liveness source, never to 'no sessions'" — and it wants applying to every reader in this crate, not one.

@ndr-ds
ndr-ds marked this pull request as ready for review August 18, 2026 01:30
@ndr-ds
ndr-ds merged commit 4102064 into main Aug 18, 2026
5 checks passed
@ndr-ds
ndr-ds deleted the a-failed-registry-read-is-not-an-empty-registry branch August 18, 2026 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant