Skip to content

[Regression] Qwen3.5 saved checkpoint is not correct with save_pretrained API since version 5.4.0 #45216

@xin3he

Description

@xin3he

System Info

transformers == 5.3.0 works well
transformers ==5.4.0 returns Unexpected model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight in loaded safetensors file

Who can help?

@zucchini-nlp

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction

import transformers

name = "Qwen/Qwen3.5-0.8B"
model = transformers.Qwen3_5ForConditionalGeneration.from_pretrained(name, trust_remote_code=True)
model.save_pretrained("./qwen-35")

from safetensors.torch import save_file, load_file

loaded = load_file("./qwen-35/model.safetensors")
assert not 'model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight' in loaded, "Unexpected model.language_model.language_model.language_model.layers.7.self_attn.v_proj.weight in loaded safetensors file"

Expected behavior

assert pass

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions