Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
### Candid

* Bug fixes:
+ Scope the decode fast paths of a map to their own half of an entry, so a map entry's key and value each decode under their own declared type. `deserialize_map` derives a big-integer fast path from the map's value type and a text fast path from its key type; each previously stayed active for the whole entry. The big-integer path is now cleared for the duration of the key and restored for the value, the text path is cleared for the duration of the value, and the value's expected and wire types are re-established on every entry. Each half of an entry therefore goes through its own type's entry point, keeping its own encoding (SLEB128 for `int`, LEB128 for `nat`) and its own subtype check, for every combination of key and value type — including a value whose type shares an encoding with the key's, such as `blob` against `text`. The wire format is unchanged, and entries whose key and value types agree decode identically.
+ Share the undecoded argument queue behind a reference count. The option/backtracking path snapshots the deserializer to restore on a subtype mismatch, which previously copied the whole queue of remaining arguments on every present `opt`. The queue is only mutated at the top level, so it is now shared rather than copied and the snapshot is a refcount bump. Skipping many present optional arguments is no longer superlinear in the argument count; the decode result is unchanged.

## 2026-08-14
Expand Down
48 changes: 39 additions & 9 deletions rust/candid/src/de.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1301,6 +1301,8 @@ impl<'de> de::Deserializer<'de> for &mut Deserializer<'de> {
expect,
wire,
key_text_fast,
#[cfg(feature = "bignum")]
value_bignum_fast,
},
));
self.text_fast_path = false;
Expand Down Expand Up @@ -1448,6 +1450,8 @@ enum Style {
expect: (Type, Type),
wire: (Type, Type),
key_text_fast: bool,
#[cfg(feature = "bignum")]
value_bignum_fast: Option<BigNumFastPath>,
},
}

Expand Down Expand Up @@ -1764,13 +1768,15 @@ impl<'de> de::MapAccess<'de> for Compound<'_, 'de> {
ref expect,
ref wire,
key_text_fast,
#[cfg(feature = "bignum")]
value_bignum_fast,
} => {
if *len == 0 {
return Ok(None);
}
*len -= 1;
#[cfg(feature = "bignum")]
let any_fast = key_text_fast || self.de.bignum_vec_fast_path.is_some();
let any_fast = key_text_fast || value_bignum_fast.is_some();
#[cfg(not(feature = "bignum"))]
let any_fast = key_text_fast;
if !any_fast {
Expand All @@ -1782,6 +1788,15 @@ impl<'de> de::MapAccess<'de> for Compound<'_, 'de> {
// of this (inner) map, leading to a "Type mismatch" when deserializing
// those keys.
self.de.text_fast_path = key_text_fast;
// The bignum fast path is derived from the map's *value* type and applies
// only to values, so it stays off for the duration of the key. This keeps
// a key decoding under its own declared type, through the regular
// deserialize_* entry point and its subtype check, whatever the value type
// happens to be. next_value_seed restores it for the value.
#[cfg(feature = "bignum")]
{
self.de.bignum_vec_fast_path = None;
}
if !key_text_fast {
self.de.expect_type = expect.0.clone();
self.de.wire_type = wire.0.clone();
Expand All @@ -1796,21 +1811,36 @@ impl<'de> de::MapAccess<'de> for Compound<'_, 'de> {
V: de::DeserializeSeed<'de>,
{
match &self.style {
Style::Map { expect, wire, .. } => {
Style::Map {
expect,
wire,
#[cfg(feature = "bignum")]
value_bignum_fast,
..
} => {
#[cfg(feature = "bignum")]
let any_fast = self.de.text_fast_path || self.de.bignum_vec_fast_path.is_some();
let value_bignum_fast = *value_bignum_fast;
#[cfg(feature = "bignum")]
let any_fast = self.de.text_fast_path || value_bignum_fast.is_some();
#[cfg(not(feature = "bignum"))]
let any_fast = self.de.text_fast_path;
if !any_fast {
self.de.add_cost(3)?;
}
// Re-establish the value's types unconditionally: the key is decoded under
// the key type, and the bignum fast path still reads according to
// wire_type, so the value needs its own types in place here.
self.de.expect_type = expect.1.clone();
self.de.wire_type = wire.1.clone();
// The text fast path is the mirror image: it is derived from the map's key
// type, so it too applies only to its own half of the entry. Left on, it
// lets a value be read as text without its subtype check -- `blob` shares
// text's length-prefixed encoding, so it would be accepted in a text
// value's place. next_key_seed sets it again for the following key.
self.de.text_fast_path = false;
#[cfg(feature = "bignum")]
let value_fast = self.de.bignum_vec_fast_path.is_some();
#[cfg(not(feature = "bignum"))]
let value_fast = false;
if !value_fast {
self.de.expect_type = expect.1.clone();
self.de.wire_type = wire.1.clone();
{
self.de.bignum_vec_fast_path = value_bignum_fast;
}
seed.deserialize(&mut *self.de)
}
Expand Down
96 changes: 96 additions & 0 deletions rust/candid/tests/serde.rs
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,102 @@ fn test_nested_map_non_text_key() {
assert_eq!(outer, decoded);
}

/// A map entry's key and value each decode under their own declared type: the key
/// through the key type and the value through the value type, independently of each
/// other. In particular the bignum fast path, which `deserialize_map` derives from the
/// value type, applies only to values, so a key keeps its own encoding (SLEB128 for
/// `int`, LEB128 for `nat`) and its own subtype check for every combination of key and
/// value type.
#[test]
fn test_map_key_and_value_decode_under_own_type() {
use std::collections::BTreeMap;

fn round_trip<K, V>(k: K, v: V)
where
K: CandidType + for<'a> Deserialize<'a> + Ord + std::fmt::Debug + Clone,
V: CandidType + for<'a> Deserialize<'a> + PartialEq + std::fmt::Debug + Clone,
{
let mut m: BTreeMap<K, V> = BTreeMap::new();
m.insert(k, v);
let bytes = encode_one(&m).unwrap();
let decoded: BTreeMap<K, V> = decode_one_with_config(&bytes, &get_config()).unwrap();
assert_eq!(
m.keys().collect::<Vec<_>>(),
decoded.keys().collect::<Vec<_>>()
);
assert_eq!(
m.values().collect::<Vec<_>>(),
decoded.values().collect::<Vec<_>>()
);
}

// Signed keys keep their sign whatever the value type is.
for k in [-1i64, -2, -100, -128, -1000, -2147483648] {
round_trip(Int::from(k), Nat::from(42u64));
round_trip(Int::from(k), Int::from(-7));
round_trip(Int::from(k), 42u32);
}
// Signed values keep their sign whatever the key type is.
for v in [-1i64, -2, -128, -2147483648] {
round_trip(Nat::from(5u64), Int::from(v));
round_trip(Int::from(9), Int::from(v));
round_trip("k".to_string(), Int::from(v));
}
// Above u64, where the big-integer representation is used.
let big_neg: Int = "-99999999999999999999999999".parse().unwrap();
let big_nat: Nat = "99999999999999999999999999".parse().unwrap();
round_trip(big_neg.clone(), big_nat.clone());
round_trip(big_neg.clone(), big_neg.clone());
round_trip(big_nat.clone(), big_neg.clone());

// Distinct keys stay distinct entries.
let mut m: BTreeMap<Int, Nat> = BTreeMap::new();
m.insert(Int::from(127), Nat::from(1000u64));
m.insert(Int::from(-1), Nat::from(1u64));
let bytes = encode_one(&m).unwrap();
assert_eq!(
decode_one_with_config::<BTreeMap<Int, Nat>>(&bytes, &get_config()).unwrap(),
m
);

// A key's subtype check runs: int is not a subtype of nat, whatever the value type.
let mut int_key: BTreeMap<Int, Nat> = BTreeMap::new();
int_key.insert(Int::from(5), Nat::from(7u64));
let bytes = encode_one(&int_key).unwrap();
assert!(decode_one_with_config::<BTreeMap<Nat, Nat>>(&bytes, &get_config()).is_err());

let mut int_key: BTreeMap<Int, Int> = BTreeMap::new();
int_key.insert(Int::from(5), Int::from(7));
let bytes = encode_one(&int_key).unwrap();
assert!(decode_one_with_config::<BTreeMap<Nat, Int>>(&bytes, &get_config()).is_err());

// nat is a subtype of int, so widening a key is still accepted.
let mut nat_key: BTreeMap<Nat, Nat> = BTreeMap::new();
nat_key.insert(Nat::from(5u64), Nat::from(7u64));
let bytes = encode_one(&nat_key).unwrap();
let widened = decode_one_with_config::<BTreeMap<Int, Int>>(&bytes, &get_config()).unwrap();
assert_eq!(widened[&Int::from(5)], Int::from(7));

// The mirror image of the same property: a text *key* must not let the value skip
// its own subtype check. `blob` shares text's length-prefixed encoding, so a value
// read under the key's text fast path would be accepted in a text value's place.
let mut blob_val: BTreeMap<String, Vec<u8>> = BTreeMap::new();
blob_val.insert("k".to_string(), b"hello".to_vec());
let bytes = encode_one(&blob_val).unwrap();
assert!(decode_one_with_config::<BTreeMap<String, String>>(&bytes, &get_config()).is_err());
// Rejected for the same reason outside a map and under a non-text key.
let mut blob_val2: BTreeMap<u32, Vec<u8>> = BTreeMap::new();
blob_val2.insert(1, b"hello".to_vec());
let bytes2 = encode_one(&blob_val2).unwrap();
assert!(decode_one_with_config::<BTreeMap<u32, String>>(&bytes2, &get_config()).is_err());

// Text keys and text values still round-trip, including a text value under a
// non-text key and a non-text value under a text key.
round_trip("k".to_string(), "v".to_string());
round_trip(Nat::from(1u64), "v".to_string());
round_trip("k".to_string(), Nat::from(1u64));
}

/// Regression test: elements decoded through the bulk primitive-vec fast path must
/// report the same `is_human_readable()` answer as the main deserializer, which is
/// `false` because candid is a binary format.
Expand Down
Loading