Skip to content

fix(serializing): correct two's-complement unwrap in quaternion decompression - #1067

Open
ddelgado95 wants to merge 1 commit into
FirstGearGames:mainfrom
ddelgado95:fix/quaternion-negative-unwrap
Open

fix(serializing): correct two's-complement unwrap in quaternion decompression#1067
ddelgado95 wants to merge 1 commit into
FirstGearGames:mainfrom
ddelgado95:fix/quaternion-negative-unwrap

Conversation

@ddelgado95

Copy link
Copy Markdown

Problem

ScaleToUint encodes each smaller quaternion component as a two's-complement value masked to BitsPerAxis bits, but ScaleToFloat unwraps negatives by subtracting Maximum * 2 — which is 2 * IntScale steps, not the IntMask + 1 steps the mask actually wrapped by. Every negative component therefore decodes exactly two quantization steps too high. Positive components are unaffected.

Worked example (Quaternion32, IntScale = 511, IntMask = 1023):

  • -511 steps encodes as (uint)(-511) & 1023 = 513
  • correct decode: 513 - 1024 = -511
  • current decode: 513 * M/511 - 2M = M * (513 - 1022)/511 = -509

Quaternion64Compression.ScaleToFloat_H/_L carry the same off-by-two.

Evidence

Round-tripping 200,001 values across [-Maximum, +Maximum] through ScaleToUint and back:

decode worst error, negatives worst error, positives
before 0.00345941 0.00069195
after 0.00069195 0.00069195

0.00069195 is exactly half a quantization step — the theoretical optimum. That positives already sit there while negatives are 5× worse is what localizes the defect to the negative decode path; the encoder is correct.

Small magnitudes could flip sign outright:

raw      -> encoded -> before    / after
-0.00100 ->    1023 -> +0.00138  / -0.00138
-0.50000 ->     663 -> -0.49677  / -0.49954
-0.70000 ->     518 -> -0.69742  / -0.70019

Impact

Quaternion32 is the default AutoPackType.Packed path, used by RigidbodyState in every prediction reconcile and by NetworkTransform rotation sync. The constant +0.00277 bias lands as roughly a 0.3° rotation error on clients only — the server never decodes its own state.

Because it only appears on orientations where a smaller component is negative, it presents as intermittent jitter rather than a reproducible axis bug. In a predicted rigidbody it also couples into position: the client builds thrust in a frame tilted ~0.3° from the server's, so lateral thrust acquires a vertical component the server never simulates, and the reconcile pulls it back each tick — visible as a persistent offset under held thrust and a hop on taps.

Compatibility

The encoder is untouched, so this is wire-compatible in both directions. Only the decode of already-negative components changes.

…pression

ScaleToUint encodes the smaller components as two's complement masked to
BitsPerAxis bits, but ScaleToFloat unwrapped negatives by subtracting
Maximum * 2 -- that is 2 * IntScale steps, not the IntMask + 1 steps the
mask actually wrapped by. Every negative component decoded exactly two
quantization steps high; positives were unaffected.

Round-tripping 200k values through Quaternion32:

  before: worst error 0.00345941 on negatives, 0.00069195 on positives
  after:  worst error 0.00069195 on both (== half a step, the optimum)

Small magnitudes could even flip sign: -0.001 decoded as +0.00138.

Quaternion64Compression.ScaleToFloat_H/_L carried the same off-by-two.

The encoder is untouched, so this is wire-compatible in both directions;
only the decode of already-negative components changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant