Skip to content

Honour field masks in ReadParams and WatchMempool - #1394

Draft
carbolymer wants to merge 11 commits into
mgalazyn/feature/grpc-waitfortxfrom
mgalazyn/feature/rpc-support-fieldmasks
Draft

carbolymer wants to merge 11 commits into
mgalazyn/feature/grpc-waitfortxfrom
mgalazyn/feature/rpc-support-fieldmasks

Conversation

@carbolymer

@carbolymer carbolymer commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Context

Every cardano-rpc method that carries a field_mask accepted it and ignored it.
This PR makes ReadParams and WatchMempool honour the mask, through one generic implementation driven by proto-lens descriptors.
There are no hand-written per-message tables, so each remaining method needs only a few lines to support masks.

Mask semantics:

  • Paths start at the response message, for example values or tx.native_bytes.
  • A path may continue through repeated fields and then applies to every element. This extends field_mask.proto, which allows a repeated field only as the last segment. The utxorpc Go SDK and Dolos both use such paths.
  • Paths that do not name a field are ignored without an error. AIP-161 allows this for reads.
  • An absent or empty mask, or a * path, returns everything.
  • A mask whose paths are all ignored returns an empty response.

Laziness: proto-lens records have strict fields, so trimming a fully built response saves only bandwidth.
The handlers therefore skip the node queries and conversions that the mask does not need.
ReadParams skips the protocol parameters query and the era lookup when values is not selected, and the chain tip queries when ledger_tip is not selected.
WatchMempool serialises and converts a transaction only when the mask or the request predicate needs it.
Behaviour note: a ReadParams request that selects only ledger_tip no longer determines the era.

Not in this PR: ReadGenesis, ReadEraSummary, ReadUtxos, SearchUtxos, FetchBlock and FollowTip will follow in separate PRs.
ReadState, ReadData, ReadTx and DumpHistory are not implemented yet.

This PR is stacked on #1351 (the grpc-waitfortx branch), so the diff shows only the field mask commits.

References:

How to trust this PR

Review commit by commit. Each commit is one phase, and its tests were written first.

  1. Add field mask path normalisation: a port of the normalisation in protobuf Java FieldMaskTree. Paths with empty segments are dropped.
  2. Resolve field masks against proto descriptors: turns the normalised paths into a tree of known fields.
  3. Trim responses to a field mask: oneof members are reset only when active, because proto-lens setters replace the whole oneof.
  4. Add lazy field mask helpers: handlers describe each response field as a fill, and the mask decides which fills run.
  5. Apply field masks in ReadParams.
  6. Apply field masks in WatchMempool.
  7. Changelog fragment.

Tests are unit tests and Hedgehog properties per phase.
The laziness tests pass error "forced" for the unselected inputs and fully evaluate the result.
The whole cardano-rpc-test suite passes.

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. See Running tests for more details
  • Self-reviewed the diff
  • Changelog fragment added in .changes/

@carbolymer
carbolymer changed the base branch from master to mgalazyn/feature/grpc-waitfortx October 23, 2026 06:27
@carbolymer carbolymer self-assigned this Oct 23, 2026
@carbolymer
carbolymer force-pushed the mgalazyn/feature/rpc-support-fieldmasks branch from ba1c957 to f6ce428 Compare October 23, 2026 09:14
@carbolymer carbolymer changed the title Mgalazyn/feature/rpc support fieldmasks Honour field masks in ReadParams and WatchMempool Oct 23, 2026
@carbolymer
carbolymer force-pushed the mgalazyn/feature/rpc-support-fieldmasks branch 2 times, most recently from 7687d83 to 3a71f22 Compare October 23, 2026 11:15
Port the path normalisation of protobuf's Java FieldMaskTree: a field absorbs its sub-paths, duplicates collapse and paths read back sorted. Paths with empty segments are dropped.
Paths are rooted at the response message and may continue through repeated message fields. Paths that do not name a field are dropped whole; an absent mask or a "*" path selects everything.
Fields outside the mask are reset to their defaults, recursing into singular and repeated message fields. A oneof member is reset only when it is the active member, because proto-lens setters replace the whole oneof.
Handlers describe each response field as a fill: its lens label and the action that obtains its value. The mask decides which fills run, so a node query or conversion for an unselected field never runs and its value is never evaluated.
ReadParams now returns only the fields its mask selects, and skips the protocol parameters query or the chain tip queries when the mask does not need them.
Paths start at the response message, for example tx.native_bytes. The transaction is serialised and converted only when the mask or the request's predicate needs it.

Proto/Api/UtxoRpc/Submit.hs now exposes the fieldMask lens of the request.
subMaskEach gives the part of a field mask below a repeated message field, and fillEach builds each element of that field lazily against it.
ReadParams, ReadUtxos and SearchUtxos build ledger_tip from one shared query, so each can skip it when its field mask does not select ledger_tip.
ReadUtxos returns only the fields its mask selects. It skips the UTxO query when items is not selected and the chain tip queries when ledger_tip is not, and serialises and converts each UTxO only for the fields the mask selects.
SearchUtxos returns only the fields its mask selects. It skips the UTxO query when neither items nor next_token is selected and the chain tip queries when ledger_tip is not, and converts each UTxO of the page only for the fields the mask selects.

The new fillShared helper runs a shared action once for all the fields that read its result.
@carbolymer
carbolymer force-pushed the mgalazyn/feature/rpc-support-fieldmasks branch from 3a71f22 to ef9cfd6 Compare October 26, 2026 10:42

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant