feat: fold storage pool ID into pooled blob object consistency digest#3494
Merged
halfprice merged 2 commits intoJun 30, 2026
Merged
Conversation
The pooled per-object blob consistency check previously folded only the certified blob object ID (the ObjectID key) into the cross-node digest. Also fold each entry's storage pool ID into the hash, so a node that disagrees about which pool a blob object belongs to is detected as a cross-node inconsistency. - Make `compose_blob_object_list_digest` take an `extra_hash_input` closure that contributes extra value-derived bytes per entry. The regular per-object caller passes an empty slice (digest unchanged); the pooled caller folds in `storage_pool_id()`. - Add a `PerObjectPooledBlobInfo::storage_pool_id()` accessor. - Add unit tests verifying the pooled digest differs when only the storage pool ID differs, and matches for identical entries.
Address review feedback on the pooled blob object consistency digest: - Change `compose_blob_object_list_digest`'s `extra_hash_input` from a closure to `Option<F>`, so the key-only case is expressed as `None` rather than an empty-slice closure. The regular per-object caller passes `None` (digest unchanged); the pooled caller passes `Some(|info| info.storage_pool_id())`. - Combine the two pooled-digest unit tests into one that checks both that a differing storage pool ID changes the digest and that an identical pool ID reproduces it.
shuowang12
approved these changes
Jun 30, 2026
shuowang12
left a comment
shuowang12
left a comment
Collaborator
There was a problem hiding this comment.
LGTM. I agree pool ID an important property to be included in the consistency digest.
halfprice
deleted the
zhewu/pool_blob_object_consistency_check_add_pool_id
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The pooled per-object blob consistency check (added in #3490) folded only the certified blob object ID (the
ObjectIDkey) into the cross-node digest. This PR also folds each entry's storage pool ID into the hash, so a node that disagrees about which pool a blob object belongs to is detected as a cross-node inconsistency.Changes
compose_blob_object_list_digesttake anextra_hash_inputclosure (Fn(&T) -> impl AsRef<[u8]>) that contributes extra value-derived bytes per entry.[0u8; 0]), so its digest is unchanged.storage_pool_id().PerObjectPooledBlobInfo::storage_pool_id()accessor (plus a#[cfg(test)]constructor).Testing
cargo nextest run -p walrus-service— all pass (including the two new testspooled_digest_detects_storage_pool_id_inconsistencyandpooled_digest_matches_for_identical_entries).The existing simtest infrastructure (
storage_node_certified_pooled_blob_object_digestfail point wired intoBlobInfoConsistencyCheck) already verifies pooled digests match across all nodes; with this change those checks now also cover pool-ID divergence end-to-end.