Skip to content

Decode all zstd frames when decompressing bytecodes on wasm32 - #6820

Merged
ma2bd merged 1 commit into
mainfrom
ma2bd/2710-multi-frame-zstd
Sep 13, 2026
Merged

ma2bd merged 1 commit into
mainfrom
ma2bd/2710-multi-frame-zstd

Conversation

@ma2bd

@ma2bd ma2bd commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Motivation

The web wallet decompresses bytecodes with ruzstd, whose StreamingDecoder decodes a single
zstd frame and then keeps returning Ok(0). Our wasm32 code did not account for that, as
noted in #2710:

  • decompress looped while !decoder.get_ref().is_empty(), so a bytecode made of more than one
    frame made it loop forever, freezing the tab, instead of merely decoding the first frame.
  • decompressed_size_at_most measured the first frame only, so it underestimated the size of a
    multi-frame bytecode.

Validators use the zstd crate, which accepts several concatenated frames, both for the size
check in linera-execution/src/policy.rs and for decompression in linera-storage/src/lib.rs.
Nothing that we compress produces multiple frames, but nothing rejects them either: a module
published as two frames is valid to a validator and hangs every web client that loads it.

Proposal

Decode frame by frame on wasm32, creating a StreamingDecoder per frame and stepping over
skippable frames using the length in their header, which is what zstd does. The two targets
now accept the same bytecodes, so this is not a change in validation.

The frame loop is shared by decompress and decompressed_size_at_most, and replaces the
TODO(#2710) comments in both.

Also drops the two #[cfg(with_metrics)] blocks from the wasm32 impl: with_metrics is
defined as all(not(target_arch = "wasm32"), feature = "metrics"), so they were dead code, and
the one in decompress referred to an unqualified BYTECODE_DECOMPRESSION_LATENCY that would
not have compiled had it ever been enabled.

Test Plan

Three new unit tests in linera-base cover a bytecode of two frames followed by a skippable
frame, and check the new code against zstd on the same bytes: all frames are decompressed, all
frames count towards the size limit, and trailing garbage is an error rather than a hang. They
run on the usual targets rather than on wasm32, where Bytecode::compress is unavailable, so
CI covers them; ruzstd is a dev-dependency off wasm32 for that purpose.

Checked with cargo test -p linera-base, cargo clippy -p linera-base --all-targets --all-features and cargo clippy -p linera-base --target wasm32-unknown-unknown.

Release Plan

  • These changes should be backported to the latest testnet branch, then
    • be released in a new SDK.

Links

ma2bd added a commit that referenced this pull request Sep 13, 2026
…essing bytecodes on wasm32 (#6821)

Backport of #6820.

## Motivation

The web wallet decompresses bytecodes with `ruzstd`, whose
`StreamingDecoder` decodes a single
zstd frame and then keeps returning `Ok(0)`. Our `wasm32` code did not
account for that, as
noted in #2710:

* `decompress` looped `while !decoder.get_ref().is_empty()`, so a
bytecode made of more than one
frame made it **loop forever**, freezing the tab, instead of merely
decoding the first frame.
* `decompressed_size_at_most` measured the first frame only, so it
underestimated the size of a
  multi-frame bytecode.

Validators use the `zstd` crate, which accepts several concatenated
frames, both for the size
check in `linera-execution/src/policy.rs` and for decompression in
`linera-storage/src/lib.rs`.
Nothing that we compress produces multiple frames, but nothing rejects
them either: a module
published as two frames is valid to a validator and hangs every web
client that loads it.

## Proposal

Decode frame by frame on `wasm32`, creating a `StreamingDecoder` per
frame and stepping over
skippable frames using the length in their header, which is what `zstd`
does. The two targets
now accept the same bytecodes, so this is not a change in validation.

The frame loop is shared by `decompress` and
`decompressed_size_at_most`, and replaces the
`TODO(#2710)` comments in both.

Also drops the two `#[cfg(with_metrics)]` blocks from the `wasm32` impl:
`with_metrics` is
defined as `all(not(target_arch = "wasm32"), feature = "metrics")`, so
they were dead code, and
the one in `decompress` referred to an unqualified
`BYTECODE_DECOMPRESSION_LATENCY` that would
not have compiled had it ever been enabled.

## Test Plan

Cherry-picked from #6820 with no conflicts, then rechecked against this
branch's Rust 1.86
toolchain: `cargo test -p linera-base`, `cargo clippy -p linera-base
--all-targets
--all-features -- -D warnings`, `cargo clippy -p linera-base --target
wasm32-unknown-unknown
-- -D warnings`, and `cargo fmt --check` on `nightly-2025-04-03`.

The three unit tests that come with the fix cover a bytecode of two
frames followed by a
skippable frame: all frames are decompressed, all frames count towards
the size limit, and
trailing garbage is an error rather than a hang.

## Release Plan

- These changes should be backported to the latest `testnet` branch,
then
    - be released in a new SDK.

## Links

- #6820
- #2710
- [reviewer
checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ma2bd
ma2bd merged commit 85cadd0 into main Sep 13, 2026
45 checks passed
@ma2bd
ma2bd deleted the ma2bd/2710-multi-frame-zstd branch September 13, 2026 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bytecodes compressed into multiple frames can't be decompressed in the web wallet

1 participant