Repository navigation
Decode all zstd frames when decompressing bytecodes on wasm32 - #6820
Merged
Merged
Conversation
ma2bd
added a commit
that referenced
this pull request
Sep 13, 2026
…essing bytecodes on wasm32 (#6821) Backport of #6820. ## Motivation The web wallet decompresses bytecodes with `ruzstd`, whose `StreamingDecoder` decodes a single zstd frame and then keeps returning `Ok(0)`. Our `wasm32` code did not account for that, as noted in #2710: * `decompress` looped `while !decoder.get_ref().is_empty()`, so a bytecode made of more than one frame made it **loop forever**, freezing the tab, instead of merely decoding the first frame. * `decompressed_size_at_most` measured the first frame only, so it underestimated the size of a multi-frame bytecode. Validators use the `zstd` crate, which accepts several concatenated frames, both for the size check in `linera-execution/src/policy.rs` and for decompression in `linera-storage/src/lib.rs`. Nothing that we compress produces multiple frames, but nothing rejects them either: a module published as two frames is valid to a validator and hangs every web client that loads it. ## Proposal Decode frame by frame on `wasm32`, creating a `StreamingDecoder` per frame and stepping over skippable frames using the length in their header, which is what `zstd` does. The two targets now accept the same bytecodes, so this is not a change in validation. The frame loop is shared by `decompress` and `decompressed_size_at_most`, and replaces the `TODO(#2710)` comments in both. Also drops the two `#[cfg(with_metrics)]` blocks from the `wasm32` impl: `with_metrics` is defined as `all(not(target_arch = "wasm32"), feature = "metrics")`, so they were dead code, and the one in `decompress` referred to an unqualified `BYTECODE_DECOMPRESSION_LATENCY` that would not have compiled had it ever been enabled. ## Test Plan Cherry-picked from #6820 with no conflicts, then rechecked against this branch's Rust 1.86 toolchain: `cargo test -p linera-base`, `cargo clippy -p linera-base --all-targets --all-features -- -D warnings`, `cargo clippy -p linera-base --target wasm32-unknown-unknown -- -D warnings`, and `cargo fmt --check` on `nightly-2025-04-03`. The three unit tests that come with the fix cover a bytecode of two frames followed by a skippable frame: all frames are decompressed, all frames count towards the size limit, and trailing garbage is an error rather than a hang. ## Release Plan - These changes should be backported to the latest `testnet` branch, then - be released in a new SDK. ## Links - #6820 - #2710 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The web wallet decompresses bytecodes with
ruzstd, whoseStreamingDecoderdecodes a singlezstd frame and then keeps returning
Ok(0). Ourwasm32code did not account for that, asnoted in #2710:
decompressloopedwhile !decoder.get_ref().is_empty(), so a bytecode made of more than oneframe made it loop forever, freezing the tab, instead of merely decoding the first frame.
decompressed_size_at_mostmeasured the first frame only, so it underestimated the size of amulti-frame bytecode.
Validators use the
zstdcrate, which accepts several concatenated frames, both for the sizecheck in
linera-execution/src/policy.rsand for decompression inlinera-storage/src/lib.rs.Nothing that we compress produces multiple frames, but nothing rejects them either: a module
published as two frames is valid to a validator and hangs every web client that loads it.
Proposal
Decode frame by frame on
wasm32, creating aStreamingDecoderper frame and stepping overskippable frames using the length in their header, which is what
zstddoes. The two targetsnow accept the same bytecodes, so this is not a change in validation.
The frame loop is shared by
decompressanddecompressed_size_at_most, and replaces theTODO(#2710)comments in both.Also drops the two
#[cfg(with_metrics)]blocks from thewasm32impl:with_metricsisdefined as
all(not(target_arch = "wasm32"), feature = "metrics"), so they were dead code, andthe one in
decompressreferred to an unqualifiedBYTECODE_DECOMPRESSION_LATENCYthat wouldnot have compiled had it ever been enabled.
Test Plan
Three new unit tests in
linera-basecover a bytecode of two frames followed by a skippableframe, and check the new code against
zstdon the same bytes: all frames are decompressed, allframes count towards the size limit, and trailing garbage is an error rather than a hang. They
run on the usual targets rather than on
wasm32, whereBytecode::compressis unavailable, soCI covers them;
ruzstdis a dev-dependency offwasm32for that purpose.Checked with
cargo test -p linera-base,cargo clippy -p linera-base --all-targets --all-featuresandcargo clippy -p linera-base --target wasm32-unknown-unknown.Release Plan
testnetbranch, thenLinks