perf(webapp): history stops re-downloading every journal on every request (BEA-85)

[ssowonny]

TL;DR

• History took 8–10s because every request downloaded every journal in the project and re-parsed every op ever written — and each "load more" paid it again, which is why pagination made it worse rather than better.
• No index was ever going to help: journals live in object storage, never in the DB.
• Journals only ever grow, so the (size, mtime) that List already returns proves a cached parse is still good. A warm request now does one List and zero downloads.
• Cold loads got faster too — the journal fetches run concurrently instead of one serial round trip per device.
• Known gap: the reporter's real project is unmeasured. Local numbers below; if the cold path is still slow there, the next step is tail-reads, not a looser target.

What was actually slow

flowchart LR
    R["every request"] --> L["List journal/"]
    L --> G["Get EVERY journal<br/>one at a time"]
    G --> P["parse EVERY op<br/>ever written"]
    P --> S["sort twice,<br/>slice one page"]
    classDef hot fill:#ef444422,stroke:#ef4444,stroke-width:2px
    class G,P hot

Loading

loadSourcedOps is the funnel every reader goes through — history, folder listings, restore, and the hub's own appendOp — and it cached nothing. A cursor is a position in an ordering, not a snapshot, so paging re-loaded everything and skipped forward in memory.

What it does now

flowchart LR
    R["every request"] --> L["List journal/<br/>one round trip, always"]
    L --> C{"size + mtime<br/>unchanged?"}
    C -- hit --> U["reuse the parsed ops"]
    C -- miss --> F["Get + parse<br/>concurrently, limit 8"]
    U --> A["fresh outer slice<br/>callers sort it"]
    F --> A
    classDef good fill:#22c55e22,stroke:#22c55e,stroke-width:2px
    class U,F good

Loading

The enabling fact is already load-bearing elsewhere in the repo: journals only grow. A device appends only to its own (the one-writer invariant) and the hub's appendOp is a read-modify-write that rewrites the key with strictly more bytes. Nothing shrinks or rewrites one in place, so a matching (Size, Modified) proves the parse we hold is current — no staleness window, no time-based expiry, and no new Backend method. All three real backends report both fields.

Files() needed no change: it goes through the same funnel, so folder listings get the cache for free. Please don't add a second cache next to volume.snapshot.

Numbers

Measured locally with a throwaway test (not committed): 16 000 ops across 8 journals, behind a backend with a simulated 40 ms round trip.

	cold	warm
before	465 ms	467 ms — every request identical, which is the reported symptom
after	124 ms	54 ms

The warm request is now the List plus assembly. The 1s target in the issue is inferred, not given, and the reporter's real project is unmeasured — if the cold path there is still slow, that's the signal to do the tail-read step rather than loosen the number.

What can't break

• Output is byte-identical. TestHistoryOutputUnchangedByCache captures /history across five query shapes warm, drops the cache, and compares byte for byte — ordering, cursors, add/edit/delete classification and the device-registry join all unchanged. Ops are assembled in the same List order the old loop used, which matters because both downstream sorts are SliceStable.
• Callers still sort what they're handed. Every call assembles a fresh outer slice; the cached []journal.Op is never handed out directly, or the first sort would scramble the cache. journal.Op is all value fields, so appending one copies it.
• A corrupt journal is still ignored — and now cached as zero ops, or it would be re-downloaded on every request for as long as it stays corrupt.
• appendOp reads through the same cache while holding upmu and computes maxLamport/mySeq from what it reads. Safe because invalidation is by (size, mtime) and it re-Lists inside the lock, so it sees its own previous write. Deliberately no "skip the cache for writes" shortcut — that would reintroduce the serial fetch on the upload path.

What you're accepting

• Memory. What was transient per request is now resident. Peak per request is unchanged; the new cost is that it persists, across every project the hub has served since start (s.vols never evicts). Bounded at 64 MiB of raw journal bytes per project with all-or-nothing eviction — over the cap it drops everything and that pass costs exactly what every pass cost before. The real ceiling is that times the number of projects touched, named in a ponytail: comment with the upgrade path.
• No singleflight. Two requests missing the same journal at the same moment both fetch it — which is what every request did before this existed.
• Modified granularity on file:// is coarse on some filesystems. Size is the primary token and always changes on append; the mtime pairing is belt-and-braces.

Deviation from the plan

One, small: the plan pruned vanished journals in the store pass. That pass only runs when something was fetched, so a journal that disappeared would never be seen and its entry stayed resident forever. Pruning moved into the partition pass, which runs on every request. TestJournalCacheDropsVanishedJournals pins it.

Architecture changes

architecture/webapp-server.md: RemoteSource gained the cache fields (jcache, jbytes) and cacheJournals; new value type cachedJournal composed into it. Nothing removed.

✅ added · ❌ removed (strikethrough) · unmarked = unchanged

flowchart TB
    RemoteSource["<div style='text-align:left'><b>RemoteSource</b><br/>+Backend remote.Backend<br/>+Device Identity<br/>+PresignTTL time.Duration<br/>+Remove(ctx, path, who, note)<br/>+OpenBlob(ctx, sha)<br/>-verify(ctx, sha) re-hash until sealed<br/>-blobStat(ctx, blob) remote.Object<br/>-sealed sync.Map sha→proved immutable<br/><span style='background:#22c55e55;padding:0 4px;border-radius:3px'>✅ -jcache map key→cachedJournal</span><br/><span style='background:#22c55e55;padding:0 4px;border-radius:3px'>✅ -jbytes int64 raw bytes cached</span><br/>-loadSourcedOps(ctx) []sourcedOp<br/><span style='background:#22c55e55;padding:0 4px;border-radius:3px'>✅ -cacheJournals(keep, misses, parsed, sizes)</span><br/>-appendOp(ctx, op)</div>"]
    sourcedOp["<div style='text-align:left'><b>sourcedOp</b><br/>+Op journal.Op<br/>+From journal key's device</div>"]
    cachedJournal["<div style='text-align:left'><b>cachedJournal</b><br/>+size int64<br/>+mod time.Time<br/>+bytes int64<br/>+ops []journal.Op</div>"]
    Why["Journals only GROW, so List's<br/>size + mtime is a free version token.<br/>No expiry, no new Backend method.<br/>Bounded by a raw-byte cap per project."]
    RemoteSource -. "attribution comes from the journal key" .-> sourcedOp
    RemoteSource -- "<span style='background:#22c55e55;padding:0 5px;border-radius:3px'>✅ parsed ops, keyed on size+mtime</span>" --> cachedJournal
    cachedJournal -.- Why
    classDef added fill:#22c55e22,stroke:#22c55e,stroke-width:2px
    classDef noteBox fill:#88888822,stroke:#888888,stroke-dasharray:2 2
    class cachedJournal added
    class Why noteBox
    linkStyle 1 stroke:#22c55e,stroke-width:2px

Loading

What was run

• go build ./..., go vet ./... — clean
• go test ./... — all packages pass
• go test -race -timeout 40m ./internal/webapp/ — clean (23 min; it needs the longer timeout because unrelated slow tests blow past Go's 10m default)
• npm run e2e — 153 passed, 1 skipped (the skip is pre-existing)
• New tests: internal/webapp/journalcache_test.go — warm request fetches zero journals, one push invalidates only that journal, paging fetches nothing further, output byte-identical with and without the cache, vanished journals are dropped and the byte total stays honest

No frontend changes, so static/ is untouched.

Closes BEA-85.

Build session

cd $(git worktree list | grep bea-85 | awk '{print $1}') && claude --resume e2d2c4c2-4810-4cba-ac69-57ee1f08a3f7

(only works on this machine)