Skip to content

Restore z-score normalization for pi0_fast_libero configs#971

Open
0xadvait wants to merge 1 commit into
Physical-Intelligence:mainfrom
0xadvait:fix/pi0-fast-libero-quantile-norm
Open

Restore z-score normalization for pi0_fast_libero configs#971
0xadvait wants to merge 1 commit into
Physical-Intelligence:mainfrom
0xadvait:fix/pi0-fast-libero-quantile-norm

Conversation

@0xadvait

Copy link
Copy Markdown

Problem

Evaluating the official pi0_fast_libero checkpoint on current main gives ~0% success on LIBERO (the arm spins in place). #849 has six independent reports, and one user bisected the last working commit to e458066.

Root cause

At e458066, LeRobotLiberoDataConfig built its DataConfig without setting use_quantile_norm, so the pi0-FAST LIBERO configs trained and served with z-score normalization. Current main's DataConfigFactory.create_base_config force-sets use_quantile_norm = model_type != PI0, which silently flipped pi0_fast_libero and pi0_fast_libero_low_mem_finetune to quantile normalization. The released checkpoint was trained with z-score stats, so the state input is mis-normalized going in and the action chunk is mis-unnormalized coming out. The scale mismatch on actions is what produces the spinning-arm behavior. DROID FAST configs are unaffected; they used quantile normalization both before and after that change.

Evidence

Deterministic probe: the official checkpoint, a fixed synthetic LIBERO observation, greedy FAST decoding, run at e458066 and at current main. Preprocessed image hashes and transform sequences are identical at both commits, but:

  • post-normalization state vector differs by up to 10.36
  • FAST token ids diverge at 5 positions
  • action chunks differ with max abs diff 2.64

With this PR applied to main, the same probe reproduces e458066's post-normalization state, token ids, and both action chunks exactly (max abs diff 0 on all four checks, two infer calls each).

Change

  • DataConfigFactory gains use_quantile_norm_override: bool | None (default None keeps the model-type default for every existing config).
  • The two pi0_fast_libero configs pin the override to False with a comment explaining the checkpoint compatibility constraint.

Flag matrix after the change: pi0_fast_libero False, pi0_fast_libero_low_mem_finetune False, pi0_fast_droid True, pi0_libero False, pi05_libero True.

ruff check and ruff format --check pass. data_loader_test.py: 4 passed, plus test_with_real_dataset which fails identically on clean main (pre-existing dataset-fetch issue, unrelated).

Caveat

Anyone who finetuned pi0-FAST on LIBERO with current main trained under quantile normalization; for those checkpoints the override makes the choice explicit and per-config instead of a silent default flip.

Fixes #849

The quantile-norm default for non-PI0 models silently changed
normalization for the released pi0_fast_libero checkpoint, which was
trained with z-score stats. Serving it with quantile normalization
mis-scales the state input and the action unnormalization, which is why
LIBERO evals with that checkpoint dropped to roughly 0% success. Adds
an explicit per-config override and pins the two pi0_fast_libero
configs back to z-score normalization; DROID FAST and pi05 configs are
unchanged.

Fixes Physical-Intelligence#849
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pi0 fast libero checkpoint no longer works with new repo?

1 participant