Skip to content

Port ERNIE-4.5-21B-A3B-Base (ernie4_5_moe) - #223

Open
DarthCeltic wants to merge 2 commits into
aifoundry-org:mainfrom
DarthCeltic:port-ernie45-moe
Open

Port ERNIE-4.5-21B-A3B-Base (ernie4_5_moe)#223
DarthCeltic wants to merge 2 commits into
aifoundry-org:mainfrom
DarthCeltic:port-ernie45-moe

Conversation

@DarthCeltic

Copy link
Copy Markdown
Contributor

Summary

  • Adds baidu/ERNIE-4.5-21B-A3B-Base-PT to the most_models_ported track (identity ernie4_5_moe), introducing the ERNIE 4.5 MoE execution family to the board (21B total params / 3B active per token — "A3B", 64 routed experts + 2 shared experts, 28 layers, 131K context).
  • Distinct from the already-claimed ernie45_03b identity (PR feat(llama_cpp_et): add ERNIE-4.5-0.3B, CodeShell-7B-Chat, Cohere2 causal-LM ports #185): that port covers ERNIE-4.5-0.3B, a small dense model via the separate Ernie4_5Model converter class. This port targets Ernie4_5MoeModel (a subclass), a genuinely different code path — expert routing, shared experts, different tensor shapes — and a distinct architecture/identity.
  • No converter fix was needed: Ernie4_5MoeModel is already correctly registered in this repo's convert_hf_to_gguf.py, so this converted cleanly on the first attempt with the stock, unmodified converter — same clean-conversion category as smallthinker and afmoe.
  • Self-converted to Q8_0 GGUF (389 tensors, 23.2 GB, sha256=2d18e66097f598c8fb5f3d15fd2fba9e6f916652985426bf0e7e1b478a625d48), hosted on Hugging Face (darthceltic85/ernie45-21b-a3b-base-gguf) since it far exceeds GitHub's 2 GB release-asset limit.
  • Verified live: model loads cleanly on the CPU backend (arch = ernie4_5-moe, 64 experts / 6 used + shared experts, correct MoE tensor layout, clean compute graph) and real llama-perplexity against WikiText-2 raw gives PPL = 6.8926 +/- 1.25185 — a good, in-range result for a 21.83B-parameter model.

Test plan

  • benchmark_config in the claim JSON exactly matches .github/ci/benchmark_config.json's new ernie45_21b_a3b_base entry
  • Source revision is a valid 40-char hex commit SHA
  • RECIPE.md exists and documents both the clean conversion and the distinction from the existing dense ERNIE-4.5 identity
  • All new/modified JSON files are valid JSON
  • Model loads and produces a real, in-range perplexity result (not fabricated)

Adds baidu/ERNIE-4.5-21B-A3B-Base-PT to the most_models_ported track,
introducing the ERNIE 4.5 MoE execution family to the board (21B total
params / 3B active per token, 64 routed experts + 2 shared, 28 layers).

Distinct from the already-claimed ernie45_03b identity (PR aifoundry-org#185), which
covers a small dense ERNIE-4.5-0.3B model via a separate converter
class. This port targets Ernie4_5MoeModel, a genuinely different
architecture path (expert routing, shared experts).

No converter fix was needed -- Ernie4_5MoeModel is already correctly
registered, so this converted cleanly on the first attempt with the
stock, unmodified convert_hf_to_gguf.py.

PPL = 6.8926 +/- 1.25185 on WikiText-2 raw, confirmed via a real
llama-perplexity run on CPU backend. GGUF (23.2 GB) hosted on Hugging
Face (darthceltic85/ernie45-21b-a3b-base-gguf) since it far exceeds
GitHubs 2GB release-asset limit.
@DarthCeltic
DarthCeltic requested a review from AFOliveira as a code owner July 26, 2026 10:05
@github-actions github-actions Bot added track: model-ports Global approved model ports prize track: community Community support and ecosystem contributions labels Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

track: community Community support and ecosystem contributions track: model-ports Global approved model ports prize

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant