feat(indexes): stake address log for O(page) account address queries - #1225
Closed
slowbackspace wants to merge 2 commits into
Closed
feat(indexes): stake address log for O(page) account address queries#1225slowbackspace wants to merge 2 commits into
slowbackspace wants to merge 2 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The /accounts/{stake_address}/addresses endpoint diverged from
Blockfrost in two cases:
- order=desc sorted addresses by their latest on-chain appearance.
Blockfrost returns the exact reverse of the asc list, which orders
addresses by first appearance. Reused addresses came out in the
wrong position.
- Accounts that only appear inside pool registrations (reward account
or pool owner) returned 404. Blockfrost knows these credentials and
returns an empty list.
Fixes #1140
The /accounts/{stake_address}/addresses endpoint scans every archive
block that touches the account. With correct first-appearance ordering,
a desc request must scan the account's full history: 305 seconds
measured on mainnet for an exchange account with 400k+ addresses.
The stake address log stores each (stake credential, address) pair
once, at its first on-chain appearance, ordered by slot, transaction
order, and output order. Both orders become one page read.
- fjall: new stake-log keyspace with membership entries (the write
probe and undo key) and ordered entries (the page read).
- memory: same semantics, so ToyDomain tests exercise the log.
- redb3 and noop answer None; redb3 is deprecated for index stores.
- The apply path emits appearances from index_block. The undo path
derives them from the same function, so a rollback removes exactly
what apply inserted, and only when the undone block was the pair's
first appearance.
- A ready marker gates reads. Genesis bootstrap sets it, so stores
synced from scratch serve the log. Existing stores answer None and
the endpoint falls back to the archive scan until a resync.
vladimirvolek
force-pushed
the
feat/minibf-stake-address-log
branch
from
August 19, 2026 21:48
e9afc4c to
dcf459f
Compare
vladimirvolek
force-pushed
the
fix/minibf-account-addresses
branch
from
August 19, 2026 22:26
d30cf75 to
ea597ea
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1221 (the base branch). Follow-up to the performance concern flagged there.
Problem
With the correct first-appearance ordering from #1221,
GET /accounts/{stake_address}/addresses?order=descmust scan the account's entire archive history before it can reverse the list. Measured on a fully synced mainnet snapshot: 305 seconds (cold and warm — the cost is CPU-bound block decoding) and ~274MB transient RSS for an exchange account with 400k+ addresses. Anydescpage on such an account pays the full scan, which is also a DoS shape on public nodes.Design
A stake address log: each
(stake credential, address)pair stored once, at its first on-chain appearance, ordered by(slot, tx order, output order). Bothascanddescbecome a single page read (descis a reverse range read — Blockfrost defines it as the exact reverse ofasc).stake-logkeyspace with two entry shapes: a membership entry per pair (the write-path probe and the undo key) and an ordered entry (the page read). The write batch cannot read its own pending inserts, so the writer keeps a batch-local seen-set — one writer can span many blocks in WAL catch-up.ToyDomainendpoint tests exercise the log. redb3 and noop answerNone(redb3 is deprecated for index stores).index_block, socompute_undoderives the appearances from the samestake_appearances_from_blockfunction the apply path uses. A pair is removed only when the undone block is its stored first appearance; a block that merely repeated an address leaves the log untouched.Noneand the endpoint transparently falls back to the archive scan from fix(minibf): fix ordering and 404 edge cases in account addresses #1221 until they resync. No doctor/backfill job — resync is the rollout path.Testing
tests/index_roundtrip.rsruns against fjall and memory: marker gating, first-appearance dedup (including inside one multi-block writer), ordered paging from both ends, offset windows, stake isolation, and undo-of-first-appearance-only.Not yet done: an end-to-end run against a freshly synced network store (my local preview/mainnet stores predate the log, so they exercise only the fallback). Happy to re-sync preview and attach numbers if you want them before undrafting.
🤖 Generated with Claude Code