Bound the watcher event channel and full-reload on overflow - #42
Merged
Merged
Conversation
The watcher's outbound channel was unbounded. A burst large enough to matter (a `git checkout` across a branch that touches thousands of task files) queued thousands of paths for a consumer that drains on a ~100ms tick and reindexes each one individually. Memory and latency both grew with the size of the burst, and the per-path work was pointless anyway once the answer became "most of the project changed". The channel is now a `sync_channel` of 256, comfortably above what an editing session produces and far below what a branch switch does. Past it, the debouncer stops sending and raises an overflow flag. `WatcherEvents::drain` hands the consumer the paths and the flag together, so a partial path list cannot be mistaken for a complete one. Reading the flag clears it, so one overflow is reported to one drain. Paths are taken before the flag: a path dropped in between is still covered by the flag, whereas clearing first could discard an overflow whose paths were never delivered. The TUI answers an overflow with `Store::handle_watcher_overflow`, which emits `StateDelta::FullReload` and clears the self-write hash table. That table exists to recognize the watcher echoing one of our own writes; an echo lost in the overflow will never arrive, and a stale entry would let a later genuine edit that reproduces those bytes be mistaken for the echo and dropped. `handle_full_reload` reindexes the whole project (still incremental, so unchanged files are skipped), refreshes the file tree, and restores cursor and expansion state the same way the single-file path does. Sends never block. The debouncer thread is also what observes the shutdown flag, so parking it on a full channel would make `drop` wait for a consumer that may itself be waiting on us. `watcher::start` now returns the receiver rather than taking a `Sender`, since the flag has to travel with it. Tests: a burst within capacity does not overflow; one past capacity fills the channel, delivers what fits and reports the drop; an overflow is reported to exactly one drain; 200 due paths through a channel of 1 returns rather than deadlocking. Plus store tests that overflow asks for a full reload and forgets pending self-write hashes. Closes the last open item in tasks/tasks.live-updates.md.
This was referenced Aug 9, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase D of
docs/live-tui-updates.md, and the last open item intasks/tasks.live-updates.md.What was wrong
The watcher's outbound channel was unbounded. A burst large enough to matter (a
git checkoutacross a branch that touches thousands of task files) queuedthousands of paths for a consumer that drains on a ~100ms tick and reindexes
each one individually. Memory and latency both grew with the size of the burst,
and the per-path work was pointless anyway once the answer became "most of the
project changed".
The fix
The channel is a
sync_channelof 256, comfortably above what an editingsession produces and far below what a branch switch does. Past it, the
debouncer stops sending and raises an overflow flag.
WatcherEvents::drainhands the consumer the paths and the flag together, so apartial path list cannot be mistaken for a complete one. Reading the flag
clears it, so one overflow is reported to one drain. Paths are taken before the
flag: a path dropped in between is still covered by the flag, whereas clearing
first could discard an overflow whose paths were never delivered.
The TUI answers an overflow with
Store::handle_watcher_overflow, which emitsStateDelta::FullReloadand clears the self-write hash table. That table existsto recognize the watcher echoing one of our own writes; an echo lost in the
overflow will never arrive, and a stale entry would let a later genuine edit
that reproduces those bytes be mistaken for the echo and dropped.
handle_full_reloadreindexes the whole project (still incremental, sounchanged files are skipped), refreshes the file tree, and restores cursor and
expansion state the same way the single-file path does.
Sends never block. The debouncer thread is also what observes the shutdown
flag, so parking it on a full channel would make
dropwait for a consumer thatmay itself be waiting on us.
API change
watcher::startreturns the receiver rather than taking aSender, since theoverflow flag has to travel with it. The only caller is the TUI.
Tests
drop.
is what a blocking send would do.
so the same content reads as an external edit afterwards.
Task file cleanup
tasks/tasks.live-updates.mdhad several items still unchecked that landed inearlier PRs (
Mutation::CreateTask,format_file_in_placeusingwrite_atomic, the stale-modal flag). Those are ticked, and two genuinelydeferred items are marked waived with the reason: the modal
R-to-reload key(Esc-and-retry covers it) and directory-scoped watching (no perf complaint, and
the bounded channel now caps the damage from a large burst).