Skip to content

[2.x] Decode Codama nodes into decoded node trees - #1214

Merged
lorisleiva merged 1 commit into
mainfrom
10-07-decode_codama_nodes_into_decoded_node_trees
Oct 9, 2026
Merged

lorisleiva merged 1 commit into
mainfrom
10-07-decode_codama_nodes_into_decoded_node_trees

Conversation

@lorisleiva

@lorisleiva lorisleiva commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

This PR adds getNodeCodec, which decodes bytes into a tree of decoded nodes rather than plain values. Each decoded node keeps the path of the node that decoded it, so consumers such as explorers and the upcoming display code can read values from their types rather than guess them from JavaScript shapes.

const decoded = getNodeCodec([root, program, counterAccount]).decode(bytes);
decoded.value; // { authority: '9BbW…ftkT', count: 42n }
assertIsDecodedNode(decoded.data, 'structTypeNode');
decoded.data.fields[1].type; // { path: [root, program, counterAccount, struct, countField, u64], value: 42n, preOffset: 32, postOffset: 40 }
  • Decoded nodes: each one has a path, a value in the format of getNodeValueCodec, and the absolute cursor positions before and after reading it (preOffset and postOffset, transforms included, as in Kit's offset codecs). Children are named after their node's attributes: data, type, fields, variant, item, items and entries. Map keys keep their decoded values, for example 7n.
  • Links are transparent: a linked type decodes as its definition, and that definition's path is the one reported. This works across programs and for recursive types. A link's own transforms are included in the cursor positions of the type it decodes as.
  • Types: there is one explicit Decoded…Node type per node kind, plus DecodedNode<TNode>, GetDecodedNodeFromKind, DecodedNodeCodec, and the isDecodedNode / assertIsDecodedNode guards.
  • One source of truth: getNodeValueCodec and its visitor are now derived from getNodeCodecVisitor, and their behaviour is unchanged. On mpl-token-metadata's create data, value decoding goes from 12 to 22 µs and value encoding from 16 to 20 µs. That cost was accepted to avoid maintaining two codec paths.

@lorisleiva

Copy link
Copy Markdown
Member Author

@trevor-cortex

@trevor-cortex trevor-cortex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Adds getNodeCodec, a codec that decodes bytes into a DecodedNode tree (path + value + absolute byte range + named children) and re-derives getNodeValueCodec / getNodeValueCodecVisitor from it via mapVisitor(..., getValueCodec). The visitor now builds every node as an "unranged" decoded node through getLeafCodec / getCompositeCodec, and the intercept layer stamps offset/size on top with withRange. Byte-slicing transforms (fixedSize, sizePrefix, sentinel) go through withSlicedBytes, which maintains a baseOffsets stack so inner ranges stay absolute even when Kit hands the inner codec a copy of the bytes. Links are transparent (decode as the linked definition's type, reported at the definition's path), recursive types work via the existing lazy codec, and decoded.ts adds one Decoded…Node type per kind plus isDecodedNode / assertIsDecodedNode.

The design is sound and the implementation is careful. I verified the withSlicedBytes slice-start logic against Kit's fixDecoderSize / addDecoderSizePrefix / addDecoderSentinel (including the fixBytes pad/truncate path and the "no slice when offset is 0 and length matches" shortcut, both handled by the bytes !== outer.bytes check), and the value-codec behaviour matches the previous implementation line by line (tuple missing items, option undefined → None, struct undefined → {}, map keys as strings, enum string shorthand). Nested size prefixes, recursive types, cross-program links and link transforms are all covered by tests.

Things to watch

  • Changeset. @codama/dynamic-codecs is public and this adds new API, but I don't see a .changeset/*.md in the changed files. CONTRIBUTING asks for one per user-facing change (npx changeset add --empty). If it lives elsewhere in the stack, ignore this.
  • Byte ranges under non-padded offset transforms (inline on withRange). offset/size are the cursor span, which diverges from the "first byte read / bytes read" wording in DecodedNodeBase for absolute/relative/preOffset strategies and can produce size: 0 or a negative size. Only the padded strategy is tested.
  • inProgress.codec is the unranged composite (inline in visitDefinedType). Harmless today because the recursive link path only ever reads .type, but it's a latent trap.
  • NodeCodec isn't exported although it's the return type of the public getNodeCodecVisitor (inline).
  • Link transforms are attributed to the linked type's node. it includes the transforms of links in the byte range of their type makes this deliberate, and it's consistent with "transforms included", but note the consumer-facing asymmetry: the reported path ends at the definition's type, which doesn't carry the link's transforms, so an explorer can't explain the extra bytes from the path alone. Might be worth a sentence in the README's "Links are transparent" paragraph.

Notes for subsequent reviewers

  • baseOffsets is shared by every codec produced by one visitor instance. That's fine because reads are synchronous and nested (push/pop in try/finally), but a codec from visitor A nested inside a codec from visitor B wouldn't see B's base offset. Not a realistic path through the public API.
  • Kit's sentinel-count array strategy compares at item boundaries without slicing, so sentinel-counted collections don't need withSlicedBytes. The existing test only exercises one at offset 0; a quick case inside a struct would make that explicit.
  • The TupleTypeNode test change isn't a behaviour change: a 2-item tuple encoded from [42] already failed in the string codec with actualType: 'undefined'; the test just pins it now that getCompositeCodec passes input?.value through.

Comment thread packages/dynamic-codecs/src/codecs.ts Outdated
Comment thread packages/dynamic-codecs/src/codecs.ts
Comment thread packages/dynamic-codecs/src/codecs.ts Outdated
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from 343c9c9 to 2e1a5ca Compare October 7, 2026 10:57
@changeset-bot

changeset-bot Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 123c3f8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@trevor-cortex trevor-cortex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review

Re-fetched everything. All the points from my first pass are addressed:

  • Cursor positions. offset/size became preOffset/postOffset, documented as cursor positions in Kit's vocabulary rather than "bytes read", which removes the ambiguity under non-padded offset transforms. The three new tests (relative pre-offset moving back, absolute post-offset rewinding, absolute pre-offset on a composite whose children land elsewhere) pin the semantics, and postOffset < preOffset is now a documented possibility rather than a surprise.
  • inProgress.codec is now withRange(codec) with a comment explaining why, so the lazy recursive codec hands back ranged DecodedDefinedTypeNodes. The outer intercept withRange still wraps the returned unranged codec, so there's no double application on the direct path and the recursive path gets exactly one. I checked the linked-list test's expected offsets against this and they're consistent.
  • DecodedNodeCodec is exported from decoded.ts and used as the visitor's return type.
  • README now states that a link's transforms are included in the cursor positions of the type it decodes as, even though that type's node doesn't carry them.
  • Sentinel-counted array inside a struct is covered by it keeps cursor positions absolute within arrays ending with a sentinel.

Nothing new to flag in the updated codecs.ts; the Unranged<T> distributive Omit and the satisfies checks per visitor are a nice way to keep each decoded node shape honest without widening the public types.

The only item still outstanding is the changeset: the file list is unchanged (9 files, no .changeset/*.md). Since @codama/dynamic-codecs is public and this adds getNodeCodec, getNodeCodecVisitor and the Decoded… types, it'll need one before release unless it's being handled elsewhere in the stack. Not blocking the approval.

@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from 2e1a5ca to ce304e7 Compare October 9, 2026 09:08
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch 2 times, most recently from 03b5d2b to f3f5bbb Compare October 9, 2026 09:11
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch 2 times, most recently from 361b8ef to 9b0fbca Compare October 9, 2026 09:12
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch from f3f5bbb to 73c32ec Compare October 9, 2026 09:12
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from 9b0fbca to f143ca4 Compare October 9, 2026 09:13
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch from 3a89937 to 86485ca Compare October 9, 2026 09:14
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from f143ca4 to fa8eac0 Compare October 9, 2026 09:14
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch from 86485ca to a9b7be2 Compare October 9, 2026 09:15
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from fa8eac0 to 12782e9 Compare October 9, 2026 09:15
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch from a9b7be2 to 7e05b3c Compare October 9, 2026 09:16
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch 2 times, most recently from 1248d65 to fd9f4ec Compare October 9, 2026 09:17
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch 2 times, most recently from 572de5a to 51c3f8b Compare October 9, 2026 09:17
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch 2 times, most recently from 0ad1db3 to fcc31ac Compare October 9, 2026 09:18
@lorisleiva
lorisleiva force-pushed the 10-06-report_defined_types_without_a_finite_value_in_validators branch from 51c3f8b to 2acf133 Compare October 9, 2026 09:18
Base automatically changed from 10-06-report_defined_types_without_a_finite_value_in_validators to main October 9, 2026 09:18
@lorisleiva
lorisleiva force-pushed the 10-07-decode_codama_nodes_into_decoded_node_trees branch from fcc31ac to 123c3f8 Compare October 9, 2026 09:18
@lorisleiva
lorisleiva marked this pull request as ready for review October 9, 2026 09:19
@lorisleiva
lorisleiva merged commit 9abee8d into main Oct 9, 2026
1 of 3 checks passed
@lorisleiva
lorisleiva deleted the 10-07-decode_codama_nodes_into_decoded_node_trees branch October 9, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants