aifoundry-org · karabambus · Jul 25, 2026
@@ -315,6 +315,9 @@
     },
     "smolvlm_500m": {
       "config": "ported_models/llama_cpp_et/benchmarks/smolvlm_500m.json"
+    },
+    "phi3_mini": {
+      "config": "ported_models/phi3/benchmarks/phi3.json"
     }
   }
 }
@@ -0,0 +1,46 @@
+# Phi-3-mini-4k-Instruct Model Card
+
+- Reference family: **Phi-3** (Microsoft), decoder-only transformer. New `llama.cpp`
+  execution family `LLM_ARCH_PHI3` — not a seed/registered family, and distinct from `phi2`.
+- Hugging Face base (weights): `microsoft/Phi-3-mini-4k-instruct`, `mit`.
+- Pinned GGUF artifact: `bartowski/Phi-3-mini-4k-instruct-GGUF` at
+  `e1447f6da0be91f91683c5d19f938d4f51122d88`, file
+  `Phi-3-mini-4k-instruct-Q8_0.gguf`
+  (sha256 `0ac8ee48aeebf7d1b354691fd1e29e91c32ad88bbad10ad45ac880dcd4372a47`,
+  4,061,221,376 bytes).
+- Benchmark model id: `phi3_mini`. Runner: `llama_server` (shared `llama.cpp-et`).
+
+Weights are not committed. The board CI downloads the pinned Q8_0 GGUF by url + sha256
+(`artifacts.json`).
+
+## Architecture (Phi-3-mini-4k)
+
+32 layers, hidden 3072, FFN 8192, 32 attention heads, 32 KV heads (head_dim 96), vocab 32064,
+RoPE theta 10000, RMSNorm, gated **SwiGLU** FFN, context 4096.
+
+Phi-3 is a llama-shaped model (RMSNorm + rotary attention + SwiGLU) that fuses the QKV
+projection (`wqkv`) and the gate/up projection into single matrices, split by view/reshape.
+Handled by `src/models/phi3.cpp`. No LayerNorm, no bias, no ALiBi, no partial rotary. (The
+4k-instruct variant uses standard RoPE — no long-rope scaling.)
+
+## Op coverage on ET (confidence: MED-HIGH)
+
+| Op | ET kernel | Status |
+|----|-----------|--------|
+| `RMS_NORM` | `et-kernels/src/rms_norm_f32.c` | seed-proven |
+| `ROPE` (NEOX, full, n_dims=96) | `et-kernels/src/rope_f32.c` | seed-proven |
+| `MUL_MAT` (Q8_0 × F32; fused QKV / gate-up) | `et-kernels/src/mul_mat_*.c` | seed-proven |
+| `GLU` (SwiGLU) | `et-kernels/src/glu_f32.c` | seed-proven |
+| `SOFT_MAX`, `GET_ROWS`, `ADD`, `MUL`, `CONT`, `VIEW` (QKV/gate-up split) | ggml-et | seed-proven |
+
+Phi-3's op set is a subset of the seed-exercised set — the fused QKV and gate/up matrices are
+split with `VIEW`/`CONT` (metadata + copy, both proven), so there are **no
+implemented-but-unproven ops**. At 4 GB it is the largest port in the set; the ET board runs
+larger models (the seed leaderboard includes 8B), so size is not expected to be a constraint.
+
+## Model-ports track compliance
+
+- New standalone root `ported_models/phi3/`; only regular files added beneath it.
+- Claim: `ported_models/submissions/model_ports/phi3.json`.
+- New execution family `phi3` — not in `baseline_port_roots`, distinct from `phi2`.
+- One benchmark entry `phi3_mini` added to `.github/ci/benchmark_config.json`.
@@ -0,0 +1,33 @@
+# Phi-3-mini-4k-Instruct — ET-SoC1 model port
+
+Port of **Phi-3-mini-4k-Instruct** (Microsoft, MIT) to ET-SoC1 through the shared
+`llama.cpp-et` runtime, for the "Most Models Ported by One Individual" track.
+
+Phi-3 is a new `llama.cpp` execution family (`LLM_ARCH_PHI3`), already implemented in the
+committed framework submodule (`src/models/phi3.cpp`) — distinct from the `phi2` family. This
+port adds no framework code — it pins the Q8_0 GGUF, wires the `llama_server` benchmark, and
+files the track claim.
+
+Confidence is **MED-HIGH**: the graph (RMSNorm + RoPE + SwiGLU, with fused QKV/gate-up split
+by `VIEW`/`CONT`) is a subset of the seed-proven ET op set — no unproven ops. At 4 GB it is
+the largest port here; the board already runs 8B seed models, so size is not expected to
+matter. See `MODEL.md` for the per-op mapping.
+
+## Layout
+
+```
+ported_models/phi3/
+├── MODEL.md                              model card, architecture, op-coverage analysis
+├── README.md                            this file
+├── artifacts.json                       GGUF + shared framework artifact wiring
+├── benchmarks/phi3.json                 llama_server board benchmark config
+└── docs/
+    ├── RECIPE.md                        end-to-end reproduce recipe
+    ├── HF_REFERENCES.md                 pinned provenance
+    ├── proposed_identity_entry.json     stage-1 registry entry (maintainer adds)
+    └── proposed_reference_contract.json stage-1 correctness contract (maintainer adds)
+```
+
+Plus, outside this root:
+- `ported_models/submissions/model_ports/phi3.json` — the track claim.
+- one `phi3_mini` entry in `.github/ci/benchmark_config.json`.
@@ -0,0 +1,80 @@
+{
+  "artifacts": {
+    "llama_cpp_source": {
+      "kind": "framework_source",
+      "framework": "llama.cpp-et",
+      "submodule_path": "ported_models/llama_cpp_et/src/llama.cpp-et",
+      "license": "MIT",
+      "license_file": "LICENSE",
+      "upstream": {
+        "type": "git",
+        "url": "https://github.com/aifoundry-org/llama.cpp.git",
+        "branch": "et",
+        "revision": "cc4049d86b14e4ef72f827f3bb767b577f18fbcd",
+        "note": "Informational only. CI builds from the committed submodule_path; the phi3 port reuses the same shared framework source as the other llama.cpp-et models and does not repin it."
+      },
+      "note": "Framework source comes from the committed shared submodule. The phi3 port adds no framework code; Phi-3 (LLM_ARCH_PHI3, src/models/phi3.cpp) is already implemented in this pinned revision."
+    },
+    "llama_cpp_build": {
+      "kind": "framework_workdir",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_WORKDIR",
+      "local_cache": "local-artifacts/frameworks/llama.cpp-et/build-et",
+      "build": {
+        "cmake": "cmake",
+        "configure_args": [
+          "-DGGML_ET=ON",
+          "-DCMAKE_BUILD_TYPE=Release",
+          "-DCMAKE_CXX_STANDARD_LIBRARIES=-lglog"
+        ],
+        "build_args": ["--config", "Release"],
+        "targets": ["llama-server", "llama-perplexity", "llama-bench"]
+      },
+      "note": "Out-of-tree CMake build of the committed shared llama_cpp_source submodule."
+    },
+    "llama_server": {
+      "kind": "framework_binary",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_SERVER",
+      "relative_to": "llama_cpp_build",
+      "relative_path": "bin/llama-server"
+    },
+    "llama_perplexity": {
+      "kind": "framework_binary",
+      "framework": "llama.cpp-et",
+      "env": "LLAMA_CPP_ET_PERPLEXITY",
+      "relative_to": "llama_cpp_build",
+      "relative_path": "bin/llama-perplexity"
+    },
+    "wikitext2_raw_test": {
+      "kind": "validation_dataset",
+      "dataset": "wikitext-2-raw-v1",
+      "split": "test",
+      "filename": "wiki.test.raw",
+      "env": "WIKITEXT_RAW_PATH",
+      "local_cache": "local-artifacts/datasets/wikitext-2-raw/wiki.test.raw",
+      "source_url": "https://cosmo.zip/pub/datasets/wikitext-2-raw/wiki.test.raw",
+      "sha256": "173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08",
+      "note": "WikiText-2 raw test corpus. Shared perplexity corpus, identical to the other llama.cpp-et models."
+    },
+    "phi3_mini_q8_gguf": {
+      "kind": "model",
+      "framework": "llama.cpp-et",
+      "variant": "Phi-3-mini-4k-instruct-Q8_0",
+      "filename": "Phi-3-mini-4k-instruct-Q8_0.gguf",
+      "env": "PHI3_MINI_MODEL_PATH",
+      "source": {
+        "type": "huggingface",
+        "repo": "bartowski/Phi-3-mini-4k-instruct-GGUF",
+        "revision": "e1447f6da0be91f91683c5d19f938d4f51122d88",
+        "filename": "Phi-3-mini-4k-instruct-Q8_0.gguf",
+        "url": "https://huggingface.co/bartowski/Phi-3-mini-4k-instruct-GGUF/resolve/e1447f6da0be91f91683c5d19f938d4f51122d88/Phi-3-mini-4k-instruct-Q8_0.gguf"
+      },
+      "sha256": "0ac8ee48aeebf7d1b354691fd1e29e91c32ad88bbad10ad45ac880dcd4372a47",
+      "size_bytes": 4061221376,
+      "local_cache": "local-artifacts/models/phi3_mini/Phi-3-mini-4k-instruct-Q8_0.gguf",
+      "board_path": "/data/models/Phi-3-mini-4k-instruct-Q8_0.gguf",
+      "note": "Weights are not committed. The board CI downloads the GGUF from the pinned Hugging Face revision by url + sha256. Base model microsoft/Phi-3-mini-4k-instruct (MIT)."
+    }
+  }
+}
@@ -0,0 +1,53 @@
+{
+  "runner": "llama_server",
+  "board": true,
+  "framework": {
+    "name": "llama.cpp-et",
+    "runner": "llama_server",
+    "source_artifact": "llama_cpp_source"
+  },
+  "artifacts_file": "../artifacts.json",
+  "reference_contract": ".github/ci/reference/phi3_mini.json",
+  "canonical_variant": "Phi-3-mini-4k-instruct-Q8_0",
+  "score": {
+    "metric": "tokens_per_second",
+    "label": "Decode tokens/s",
+    "higher_is_better": true
+  },
+  "llama_server": {
+    "source_artifact": "llama_cpp_source",
+    "model_artifact": "phi3_mini_q8_gguf",
+    "server_artifact": "llama_server",
+    "workdir_artifact": "llama_cpp_build",
+    "host": "127.0.0.1",
+    "port": 18081,
+    "device": "ET",
+    "gpu_layers": 99,
+    "ctx_size": 2048,
+    "batch_size": 256,
+    "ubatch_size": 128,
+    "parallel": 1,
+    "cache_ram_mib": 0,
+    "ready_timeout_s": 180,
+    "request_timeout_s": 300,
+    "flash_attn": false,
+    "api": "completion",
+    "prompt": "Repeat this token sequence without commentary: OK OK OK OK OK OK OK OK OK OK",
+    "max_tokens": 96,
+    "temperature": 0,
+    "ignore_eos": true,
+    "min_completion_tokens": 32,
+    "perplexity": {
+      "enabled": true,
+      "perplexity_artifact": "llama_perplexity",
+      "corpus_artifact": "wikitext2_raw_test",
+      "ctx_size": 128,
+      "batch_size": 128,
+      "ubatch_size": 128,
+      "timeout_s": 300,
+      "min_ppl": 1.0,
+      "max_ppl": 1000.0,
+      "chunks": 4
+    }
+  }
+}
@@ -0,0 +1,27 @@
+# Phi-3-mini-4k-Instruct — Hugging Face provenance
+
+Pinned so the model identity is reproducible (per `docs/HF_REFERENCES.md` submission rule).
+
+## Base model (weights)
+
+| Field | Value |
+|-------|-------|
+| Repo | `microsoft/Phi-3-mini-4k-instruct` |
+| License | `mit` |
+| Architecture | `Phi3ForCausalLM` (32L, hidden 3072, FFN 8192, 32 heads / 32 KV, head_dim 96, vocab 32064, RoPE θ=10000, RMSNorm, SwiGLU, fused QKV + gate/up, ctx 4096) |
+
+## Benchmark artifact (Q8_0 GGUF)
+
+| Field | Value |
+|-------|-------|
+| Repo | `bartowski/Phi-3-mini-4k-instruct-GGUF` |
+| Revision | `e1447f6da0be91f91683c5d19f938d4f51122d88` |
+| Filename | `Phi-3-mini-4k-instruct-Q8_0.gguf` |
+| sha256 | `0ac8ee48aeebf7d1b354691fd1e29e91c32ad88bbad10ad45ac880dcd4372a47` |
+| Size | `4061221376` bytes |
+| URL | `https://huggingface.co/bartowski/Phi-3-mini-4k-instruct-GGUF/resolve/e1447f6da0be91f91683c5d19f938d4f51122d88/Phi-3-mini-4k-instruct-Q8_0.gguf` |
+
+Direct Q8_0 conversion of Phi-3-mini-4k-Instruct — no custom quantization, packing, or shape
+change. The Microsoft GGUF repo ships only fp16/q4, so the Q8_0 is pinned from the widely-used
+bartowski conversion. The sha256 above is the Hugging Face LFS object id of the pinned
+revision; verify with `sha256sum` after download.
@@ -0,0 +1,69 @@
+# `phi3` (`phi3_mini`) — model-ports reproduce recipe
+
+Ports **Phi-3-mini-4k-Instruct** (Microsoft, MIT) to ET-SoC1 via the shared `llama.cpp-et`
+runtime. Phi-3 is a new execution family (`LLM_ARCH_PHI3`) already implemented in the committed
+framework — no framework change is made here. Two-stage `docs/SUBMISSION_GUIDE.md` flow.
+
+## 0. Provenance
+
+- Base weights: `microsoft/Phi-3-mini-4k-instruct`, MIT.
+- Benchmark GGUF: `bartowski/Phi-3-mini-4k-instruct-GGUF` @
+  `e1447f6da0be91f91683c5d19f938d4f51122d88`, `Phi-3-mini-4k-instruct-Q8_0.gguf`,
+  sha256 `0ac8ee48aeebf7d1b354691fd1e29e91c32ad88bbad10ad45ac880dcd4372a47`,
+  4,061,221,376 bytes. See `docs/HF_REFERENCES.md` and `artifacts.json`.
+- No custom quantization/packing (Microsoft's own GGUF ships only fp16/q4; Q8_0 pinned from
+  the standard bartowski conversion).
+
+## 1. Stage 1 — identity + contract approval (maintainer)
+
+- `docs/proposed_identity_entry.json` → add to `data/model-port-identities.json`.
+- `docs/proposed_reference_contract.json` → add as `.github/ci/reference/phi3_mini.json`.
+
+Precomputed hashes in the identity entry:
+
+- `benchmark_config_sha256` = `canonical_sha256(effective phi3_mini model config)`:
+  ```
+  python3 - <<'PY'
+  import hashlib, json
+  cfg = json.load(open("ported_models/phi3/benchmarks/phi3.json"))
+  cfg["config"] = "ported_models/phi3/benchmarks/phi3.json"
+  print(hashlib.sha256(json.dumps(cfg, sort_keys=True, separators=(",",":")).encode()).hexdigest())
+  PY
+  ```
+- `validation_contract_sha256` = `sha256sum .github/ci/reference/phi3_mini.json` after the
+  maintainer commits the contract (byte-identical to `docs/proposed_reference_contract.json`).
+
+## 2. Stage 2 — implementation PR (this port)
+
+- New standalone root `ported_models/phi3/`.
+- Claim `ported_models/submissions/model_ports/phi3.json`.
+- One `phi3_mini` entry in `.github/ci/benchmark_config.json`:
+  ```json
+  "phi3_mini": { "config": "ported_models/phi3/benchmarks/phi3.json" }
+  ```
+
+## 3. Build (CI, reproducible by hand)
+
+```
+cmake -S ported_models/llama_cpp_et/src/llama.cpp-et -B build-et \
+  -DGGML_ET=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD_LIBRARIES=-lglog
+cmake --build build-et --config Release --target llama-server llama-perplexity
+
+build-et/bin/llama-server --model Phi-3-mini-4k-instruct-Q8_0.gguf \
+  --device ET --gpu-layers 99 --ctx-size 2048 --batch-size 256 --ubatch-size 128 \
+  --host 127.0.0.1 --port 18081
+```
+
+## 4. Board correctness + metric
+
+Main-owned `.github/ci/reference/phi3_mini.json` contract: decode validation (≥32 tokens,
+temp 0, device ET, full offload, host-CPU agreement 1.0), zero op fallbacks, WikiText-2
+perplexity within bound (ET-vs-CPU relative diff ≤ 0.01), metric decode `tokens_per_second`.
+
+## 5. Op-coverage rationale (confidence MED-HIGH)
+
+Phi-3 ops — `RMS_NORM`, `ROPE`-NEOX (full, n_dims=96), `MUL_MAT` (Q8_0×F32; fused QKV /
+gate-up), `GLU` SwiGLU, `SOFT_MAX`, `GET_ROWS`, `ADD`, `MUL`, `CONT`, `VIEW` — all have real ET
+kernels. The fused QKV and gate/up matrices are split with `VIEW`/`CONT` (metadata + copy, both
+proven), so there are no implemented-but-unproven ops. Largest port in the set (4 GB); the
+board runs 8B seed models, so size is not expected to constrain. See `MODEL.md`.
@@ -0,0 +1,17 @@
+{
+  "_comment": "Stage-1 registry entry for the phi3 identity. A maintainer adds this object to the identities list in data/model-port-identities.json to make the identity eligible. The two sha256 values are precomputed against this port; recompute per docs/RECIPE.md if any file changes. Drop the _comment key before committing.",
+  "identity_id": "phi3",
+  "execution_family": "phi3",
+  "benchmark_models": ["phi3_mini"],
+  "aliases": ["phi-3-mini-4k-instruct"],
+  "eligible": true,
+  "canonical_source": {
+    "repo": "bartowski/Phi-3-mini-4k-instruct-GGUF",
+    "revision": "e1447f6da0be91f91683c5d19f938d4f51122d88",
+    "license": "mit"
+  },
+  "approved_runner": "llama_server",
+  "benchmark_config_sha256": "c8c37874e6c221d68a330da80abe5a1ae77fac26a57c0a050d78a552213a9c31",
+  "validation_contract": ".github/ci/reference/phi3_mini.json",
+  "validation_contract_sha256": "c9e8c38f8b4a7906a693627dda93141e5baba2d2b39682b26635f708622d9768"
+}