diff --git a/text/0024-pihd-header-sync.md b/text/0024-pihd-header-sync.md new file mode 100644 index 0000000..7999c15 --- /dev/null +++ b/text/0024-pihd-header-sync.md @@ -0,0 +1,240 @@ +- Title: pihd-header-sync +- Authors: wiesche89 +- Start date: June 29, 2026 +--- + +## Summary +[summary]: #summary + +Parallel Initial Header Download (PIHD) lets a node request deterministic block +header segments from one or more peers during header sync, alongside legacy +header sync. + +This RFC adds two p2p messages: a request for a header segment and a response +with the headers. Peers signal support through a new capability bit. Nodes that +do not support PIHD continue to use legacy header sync. + +## Motivation +[motivation]: #motivation + +Header sync is the first step of node synchronization. The current +locator-based approach is simple, but it is sequential and usually talks to one +peer at a time. This can make initial sync slower when a node is far behind the +network tip. + +PIBD already requests independent segments for later sync stages. PIHD applies +the same idea to block headers. Headers form a linear chain and are already +transferred in bounded batches, making deterministic segmentation +straightforward. + +PIHD does not replace legacy header sync. It adds a parallel path when +PIHD peers are available. Legacy header sync stays available as a fallback. + +## Community-level explanation +[community-level-explanation]: #community-level-explanation + +A syncing node needs all historical block headers before it can move to the +next sync stage. Traditionally it asks a peer for headers after a locator and +continues from there. With PIHD the node splits the missing header history into +fixed-size segments. It can ask different peers for different segments. + +Each segment is identified by a segment height and an index. For PIHD the +segment height is fixed, so each full segment contains 512 headers, matching +the maximum header batch size. + +When peers with PIHD support are available, the node requests header segments in +parallel. Segments can arrive out of order. The node temporarily buffers them +until the next expected segment is ready. If PIHD stalls or peers do +not provide valid data, the node falls back to legacy header sync. + +## Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +A header segment is identified by the tuple `(h, i)`, where `h` is the segment +height and `i` is the zero-based segment index. Here, `h` is the PMMR segment +height from `SegmentIdentifier`, reused from PIBD. It is the binary logarithm +of the segment capacity, not a block height. + +For PIHD, `h` is fixed to `9`. The segment capacity is therefore: + +```text +2^9 = 512 headers +``` + +This value matches `MAX_BLOCK_HEADERS`. Larger header segments do not fit the +current header message limits, and smaller segments add round trips and sync +state without improving throughput. Peers should reject PIHD header segment +requests with any other segment height. + +The first block height for segment `(h, i)` is: + +```text +i * 2^h + 1 +``` + +The last block height is the first height plus the segment capacity minus one, +capped at the serving peer's header head. Segment index `0` therefore starts at +height `1`. The genesis header is not requested as part of PIHD. The final +segment may contain fewer than `MAX_BLOCK_HEADERS` headers when it reaches the +serving peer's header head. A request starting above the serving peer's header +head returns an empty segment. + +Peers signal support for serving deterministic header segments with a new +capability bit: + +```text +PIHD_HIST +``` + +In the reference implementation this capability is part of the default +capability set, so nodes advertise PIHD support by default. + +Nodes use this capability to avoid sending the new message types to peers that +do not advertise support. Capability gating avoids a protocol version bump for +peers that do not use these messages. + +### Header segment messages + +This RFC introduces two p2p messages. + +The request contains a `SegmentIdentifier`, using the same encoding as PIBD: + + * segment height `h` (1 byte) + * zero-based segment index `i` (8 bytes) + +The response contains: + + * segment height `h` (1 byte) + * zero-based segment index `i` (8 bytes) + * number of headers `N` (2 bytes) + * list of `N` block headers + +The response header count must not exceed `MAX_BLOCK_HEADERS`. + +The new p2p messages are: + +ID | Name | Message type +--- | --- | --- +29 | GetHeaderSegment | Request +30 | HeaderSegment | Response + +### Serving header segments + +A node should only serve header segments to peers that advertised the PIHD +capability. On a request, the serving node checks that the requested segment +height matches the fixed PIHD height. It then returns the deterministic header +range for the requested index. Here, deterministic is relative to the serving +peer's current header chain. A peer on another fork may return different +headers for the same segment identifier, because the identifier commits only to +a height range and not to a chain anchor. + +Malformed or unsupported requests should be ignored. Serving nodes also limit +header segment requests per peer to avoid request spam. + +### Requesting header segments + +This section describes the reference requester. The protocol does not mandate a +particular scheduling strategy. + +A syncing node selects peers that advertise PIHD support, have more total +difficulty than its current header head, and are near the network tip. It then +sends several header segment requests in parallel. The node tracks which peer +each segment was requested from. + +Each response is checked against the requested segment identifier and the +expected header window. The headers must start at the expected height and must +be contiguous. Valid segments are applied to the header chain in index order. +Segments that arrive ahead of the next expected index may be buffered briefly and +applied later. + +Unsolicited or duplicate responses are dropped. Peers that return segments +violating the PIHD segment rules may be banned. + +If header segment requests repeatedly time out and PIHD makes no header progress +for a fixed period, the node disables PIHD temporarily and falls back to legacy +header sync. PIHD is retried automatically once the disable period elapses. + +### Security and validation model + +PIHD header segments cannot be verified independently, unlike PIBD segments. +The parallelism is in fetching header ranges. Validation still happens in chain +order. A node applies ready segments only when they connect to the current +header head. The headers go through the same proof of work, difficulty, and +chain validation rules used by legacy header sync. + +Peers that send segments with the wrong segment height, wrong start height, +headers that are not contiguous, or headers that do not validate may be +rejected or banned. The security assumption is the same as sequential header +sync: headers are accepted only by validating them against the local chain +state. + +### Sync status + +The reference implementation records whether header sync is using PIHD or +legacy header sync and exposes this through the node status interfaces. + +## Drawbacks +[drawbacks]: #drawbacks + +PIHD adds another header sync path and some additional sync state. A node must +track pending segment requests, handle out-of-order responses, and apply +segments only when they connect to the current header head. + +The fixed segment height keeps the protocol small, but it also means the segment +layout is part of the PIHD protocol. Changing it later would require a protocol +or capability update so that old and new peers do not interpret the same segment +index differently. + +## Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +The main alternative is to keep only legacy header sync. This keeps the +implementation smaller, but it does not allow the node to make progress from +several peers in parallel during the header sync stage. + +Another alternative is to allow variable header segment heights. This was not +chosen. The fixed height maps exactly to the `MAX_BLOCK_HEADERS` limit. +Smaller values provide no clear benefit and larger values require a larger +wire-format change. + +Legacy header sync remains available and is used whenever PIHD is not +available or does not make progress. + +## Prior art +[prior-art]: #prior-art + +PIHD follows the segmentation model introduced by PIBD in RFC 0020, but applies +it to block headers. + +## Unresolved questions +[unresolved-questions]: #unresolved-questions + +The sync parameters, such as in-flight limits, timeouts, and fallback windows, +are currently fixed at compile time. It remains open whether any of these should +be exposed as configuration parameters. + +PIHD is advertised by default in the reference implementation. This may be +revisited if deployment shows that an opt-in rollout is preferable. + +Peer selection and fallback behavior may need more tuning on small networks, +or on networks where only a few peers support PIHD. + +## Future possibilities +[future-possibilities]: #future-possibilities + +Future versions may adjust the header segment size or negotiate different +segment layouts, but such changes should be treated as protocol changes. + +Header segments that can be verified on their own, based on a committed header +MMR or checkpoint, would be a stronger design. This would require a separate +consensus-level change. + +Request scheduling, peer selection, cache limits, and fallback behavior can be +adjusted after more network use. + +## References +[references]: #references + + * [1] RFC 0020: PIBD messages + * [2] Reference implementation: [PIHD header segment protocol](https://github.com/mimblewimble/grin/pull/3879) + * [3] Reference implementation: [PIHD header sync](https://github.com/mimblewimble/grin/pull/3882)