Skip to content

Bound the watcher event channel and full-reload on overflow - #42

Merged
fohara merged 1 commit into
mainfrom
feat/bounded-watcher-channel
Aug 9, 2026
Merged

fohara merged 1 commit into
mainfrom
feat/bounded-watcher-channel

Conversation

@fohara

@fohara fohara commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Phase D of docs/live-tui-updates.md, and the last open item in
tasks/tasks.live-updates.md.

What was wrong

The watcher's outbound channel was unbounded. A burst large enough to matter (a
git checkout across a branch that touches thousands of task files) queued
thousands of paths for a consumer that drains on a ~100ms tick and reindexes
each one individually. Memory and latency both grew with the size of the burst,
and the per-path work was pointless anyway once the answer became "most of the
project changed".

The fix

The channel is a sync_channel of 256, comfortably above what an editing
session produces and far below what a branch switch does. Past it, the
debouncer stops sending and raises an overflow flag.

WatcherEvents::drain hands the consumer the paths and the flag together, so a
partial path list cannot be mistaken for a complete one. Reading the flag
clears it, so one overflow is reported to one drain. Paths are taken before the
flag: a path dropped in between is still covered by the flag, whereas clearing
first could discard an overflow whose paths were never delivered.

The TUI answers an overflow with Store::handle_watcher_overflow, which emits
StateDelta::FullReload and clears the self-write hash table. That table exists
to recognize the watcher echoing one of our own writes; an echo lost in the
overflow will never arrive, and a stale entry would let a later genuine edit
that reproduces those bytes be mistaken for the echo and dropped.
handle_full_reload reindexes the whole project (still incremental, so
unchanged files are skipped), refreshes the file tree, and restores cursor and
expansion state the same way the single-file path does.

Sends never block. The debouncer thread is also what observes the shutdown
flag, so parking it on a full channel would make drop wait for a consumer that
may itself be waiting on us.

API change

watcher::start returns the receiver rather than taking a Sender, since the
overflow flag has to travel with it. The only caller is the TUI.

Tests

  • A burst within capacity does not overflow.
  • A burst past capacity fills the channel, delivers what fits, and reports the
    drop.
  • An overflow is reported to exactly one drain.
  • 200 due paths through a channel of 1 returns rather than deadlocking, which
    is what a blocking send would do.
  • Store: overflow asks for a full reload, and forgets pending self-write hashes
    so the same content reads as an external edit afterwards.

Task file cleanup

tasks/tasks.live-updates.md had several items still unchecked that landed in
earlier PRs (Mutation::CreateTask, format_file_in_place using
write_atomic, the stale-modal flag). Those are ticked, and two genuinely
deferred items are marked waived with the reason: the modal R-to-reload key
(Esc-and-retry covers it) and directory-scoped watching (no perf complaint, and
the bounded channel now caps the damage from a large burst).

The watcher's outbound channel was unbounded. A burst large enough to
matter (a `git checkout` across a branch that touches thousands of task
files) queued thousands of paths for a consumer that drains on a ~100ms
tick and reindexes each one individually. Memory and latency both grew
with the size of the burst, and the per-path work was pointless anyway
once the answer became "most of the project changed".

The channel is now a `sync_channel` of 256, comfortably above what an
editing session produces and far below what a branch switch does. Past
it, the debouncer stops sending and raises an overflow flag.

`WatcherEvents::drain` hands the consumer the paths and the flag
together, so a partial path list cannot be mistaken for a complete one.
Reading the flag clears it, so one overflow is reported to one drain.
Paths are taken before the flag: a path dropped in between is still
covered by the flag, whereas clearing first could discard an overflow
whose paths were never delivered.

The TUI answers an overflow with `Store::handle_watcher_overflow`, which
emits `StateDelta::FullReload` and clears the self-write hash table. That
table exists to recognize the watcher echoing one of our own writes; an
echo lost in the overflow will never arrive, and a stale entry would let
a later genuine edit that reproduces those bytes be mistaken for the echo
and dropped. `handle_full_reload` reindexes the whole project (still
incremental, so unchanged files are skipped), refreshes the file tree,
and restores cursor and expansion state the same way the single-file path
does.

Sends never block. The debouncer thread is also what observes the
shutdown flag, so parking it on a full channel would make `drop` wait for
a consumer that may itself be waiting on us.

`watcher::start` now returns the receiver rather than taking a `Sender`,
since the flag has to travel with it.

Tests: a burst within capacity does not overflow; one past capacity fills
the channel, delivers what fits and reports the drop; an overflow is
reported to exactly one drain; 200 due paths through a channel of 1
returns rather than deadlocking. Plus store tests that overflow asks for
a full reload and forgets pending self-write hashes.

Closes the last open item in tasks/tasks.live-updates.md.
@fohara
fohara merged commit 53ab6ce into main Aug 9, 2026
21 checks passed
@fohara
fohara deleted the feat/bounded-watcher-channel branch August 9, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant