Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/ci/benchmark_config.json
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,53 @@
},
"smolvlm_500m": {
"config": "ported_models/llama_cpp_et/benchmarks/smolvlm_500m.json"
},
"yolov10n_hf_reference": {
"canonical_variant": "yolov10n_hf_full",
"source": "ported_models/yolov10n_hf_reference/src/leaderboard_full.c",
"bench_dir": "yolov10n-bench",
"manifest": "yolov10n_variants.txt",
"score": {
"metric": "kernel_wait_s",
"label": "Full-graph kernel wait",
"higher_is_better": false,
"baseline_variant": "yolov10n_hf_full"
},
"build": {
"opt": "-O2",
"defines": [
"-std=gnu11",
"-funroll-loops",
"-fno-tree-loop-distribute-patterns",
"-fno-strict-aliasing",
"-fno-fast-math",
"-ffp-contract=off",
"-fno-tree-vectorize",
"-DYR_PMC",
"-Wl,--defsym=STACK_SIZE=4096"
]
},
"artifacts": {
"port_runtime": {
"kind": "framework_source",
"submodule_path": "ported_models/yolov10n_hf_reference/src"
}
},
"mem_size": "0x03030000",
"region_size": "0x00400000",
"dump_size": "0x022A0000",
"file_loads": [
{
"address": "0x022A0000",
"paths": ["yolov10n_hf_reference/inputs.bin"],
"required": true
},
{
"address": "0x02750000",
"paths": ["yolov10n_hf_reference/weights.bin"],
"required": true
}
]
}
}
}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
!ported_models/yolo_e2e/assets/**/*.bin
!ported_models/dncnn/assets/
!ported_models/dncnn/assets/**/*.bin
!ported_models/yolov10n_hf_reference/assets/**/*.bin
!ported_models/dncnn/assets/**/*.npy
!ported_models/dncnn/refs/
!ported_models/dncnn/refs/**/*.npy
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ Start here:
- [`docs/opinionated_porting_options/martin.md`](docs/opinionated_porting_options/martin.md): ET-SoC1 board mental model, correctness footguns, and
performance playbook.
- `ported_models/yolo/`
- `ported_models/yolov10n_hf_reference/` (separate pinned-ONNX scalar FP32 correctness path)
- `ported_models/llama_cpp_et/`
- `ported_models/ggonnx/`

Expand Down
1 change: 1 addition & 0 deletions docs/HF_REFERENCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Hugging Face base model.
|--------------|-------------------|----------|-------------|---------|
| `dncnn` | `deepinv/dncnn` | `3bb1f2a95321781343331069776c3eba98707a56` | `bsd-3-clause` | `dncnn_sigma2_gray.pth` |
| `yolo` | `kadirnar/yolov10n` | `9fa42234fbcdb13b78fa57ebaac6c50e6dd2eb21` | `agpl-3.0` | `yolov10n.pt` |
| `yolov10n_hf_reference` | `onnx-community/yolov10n` | `57657320425ee34056408a57ad9d29c4d4815bd8` | `agpl-3.0` | `onnx/model.onnx` (direct source; no re-export) |
| `lfm25` | `LiquidAI/LFM2.5-1.2B-Instruct-GGUF` | `047e06635fbe71469926b35ea414537245218200` | `other` | `LFM2.5-1.2B-Instruct-Q8_0.gguf` |
| `llama32_1b` | `lmstudio-community/Llama-3.2-1B-Instruct-GGUF` | `199151125cf15a129ab3b548b26afeed976df066` | `llama3.2` | `Llama-3.2-1B-Instruct-Q8_0.gguf` |
| `gemma3n_e2b` | `ggml-org/gemma-3n-E2B-it-GGUF` | `989cffaba23976934324f5e3abfabe31b30eb73b` | `gemma` | `gemma-3n-E2B-it-Q8_0.gguf` |
Expand Down
1 change: 1 addition & 0 deletions docs/THIRD_PARTY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ inventory; per-port `THIRD_PARTY.md` files hold the detailed records.
|-----------|------|------|---------|-------|
| ET `llama.cpp` fork | `ported_models/llama_cpp_et/src/llama.cpp-et` | git submodule (pointer) | MIT | Upstream `aifoundry-org/llama.cpp`, branch `et`. License travels with the submodule. See `ported_models/llama_cpp_et/THIRD_PARTY.md`. |
| GGONNX | `ported_models/ggonnx/src/ggonnx` | vendored source | Pending (expected Apache-2.0) | Upstream `marty1885/ggonnx` had no LICENSE at vendoring time; license grant being secured with the author. See `ported_models/ggonnx/THIRD_PARTY.md`. |
| YOLOv10n pinned ONNX | `local-artifacts/yolov10n_hf_reference/model.onnx` (not committed) | downloaded model graph and weights | AGPL-3.0 | `onnx-community/yolov10n` revision `57657320425ee34056408a57ad9d29c4d4815bd8`; see `ported_models/yolov10n_hf_reference/THIRD_PARTY.md`. |
| Model weights (GGUF, ONNX) | not committed | downloaded at runtime | Per upstream model card | Fetched on the board host from Hugging Face / source URLs declared in each port's `artifacts.json`; each model retains its own license. |

## How licensing is structured here
Expand Down
217 changes: 217 additions & 0 deletions ported_models/yolov10n_hf_reference/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,217 @@
# YOLOv10n pinned-ONNX scalar reference

This directory contains a readable FP32 C port of one exact Hugging Face ONNX
artifact. It is a correctness and workshop baseline for ET-SoC1, not an
optimized submission.

| Source fact | Pinned value |
|---|---|
| Repository | `onnx-community/yolov10n` |
| Revision | `57657320425ee34056408a57ad9d29c4d4815bd8` |
| File | `onnx/model.onnx` |
| Size | 9,386,116 bytes |
| SHA-256 | `a77dd863933f184a19e84361c64b788228a7c7dacc2c78939239a96ad3efca3b` |
| License | AGPL-3.0 |

The model is ONNX opset 13. Its interface is `images` FP32
`[1,3,640,640]` to `output0` FP32 `[1,300,6]`. The graph has 308 nodes,
187 initializers, 83 Conv nodes, and 22 operator types. It includes decode and
NMS-free Top-300 selection; it does not contain a `NonMaxSuppression` node.

Every ONNX node is kept as a separate scalar operation. There is no VPU,
TFMA, fusion, tiling, threading, fast-math, or other performance
transformation.

## Model parts

| Part | ONNX nodes | Purpose |
|---|---:|---|
| Stem | `N000:N005` | Two stride-2 Conv/SiLU steps |
| Backbone | `N006:N090` | C2f-style features and P3/P4/P5 downsampling |
| SPPF and partial attention | `N091:N128` | Spatial pyramid pooling and attention/FFN |
| Neck | `N129:N207` | Top-down and bottom-up multiscale fusion |
| Three-scale head | `N208:N270` | P3/P4/P5 box and class branches |
| DFL and decode | `N271:N288` | Distribution expectation, boxes, and class sigmoid |
| Top-300 selection | `N289:N307` | Two TopK stages and final `[box, score, class]` rows |

The first five parts are learned feature extraction and detection. The last
two are output transformation implemented inside the ONNX. Preprocessing,
score filtering, label names, drawing, and mapping boxes back to the original
image are outside the model.

`manifests/layers.tsv` provides stable `Nxxx`, high-level `Lxxx`, and
per-operator IDs. `manifests/graph_inventory.json` contains every node,
attribute, inferred tensor, and initializer.

## Supported workflows

There are only two execution modes:

1. `generate_full_graph.py` creates an end-to-end `N000:N307` package.
2. `capture_range.py` creates one node or any contiguous range, such as
`N003:N003` or `N271:N288`.

Both use the same hand-written runtime in `src/ref_runtime.c`. Generated ONNX
instrumentation, headers, weights, inputs, goldens, ELFs, dumps, and logs stay
under ignored `local-artifacts/`.

### 1. Set up and verify the pinned model

Run from the repository root:

```bash
PORT=ported_models/yolov10n_hf_reference
PY=local-artifacts/yolov10n_hf_reference/venv/bin/python

"$PORT/tools/setup_host_env.sh"
"$PY" "$PORT/tools/download_model.py"
"$PY" "$PORT/tools/inspect_onnx.py"
"$PY" "$PORT/tools/pack_initializers.py"
```

The environment is pinned in `requirements-host.txt` and supports Python
3.8–3.11. The download and every generator reject the wrong model checksum.

### 2. Run the full graph on the host

```bash
"$PY" "$PORT/tools/generate_full_graph.py" --name deterministic
"$PORT/scripts/run_host_full.sh" \
local-artifacts/yolov10n_hf_reference/full_graph/deterministic
```

This compares 16 architecture checkpoints and final `output0` against ONNX
Runtime with graph optimization disabled. FP32 values use
`abs(actual-reference) <= 5e-5 + 1e-4*abs(reference)`, except for the
documented `N288` decode checkpoint override. INT64 comparisons are exact.

### 3. Run one layer or a small range

This example captures and checks nodes `N003:N005`:

```bash
NAME=n003_n005
RANGE=local-artifacts/yolov10n_hf_reference/ranges/$NAME

"$PY" "$PORT/tools/capture_range.py" \
--range N003:N005 \
--name "$NAME"
"$PORT/scripts/run_host_range.sh" "$RANGE"
```

Every output of every selected node is retained and compared. Boundary
tensors are captured from the same pinned model through ONNX Runtime. A
single node is selected by repeating it or using it once:

```bash
"$PY" "$PORT/tools/capture_range.py" \
--range N003 \
--name n003
```

### 4. Build and run that range in `sys_emu`

The ET compiler requires a valid `ET_PLATFORM` or `ET_INSTALL`. Set
`LAUNCHER` to the system-emulator launcher installed on the ET host.

```bash
ELF="$RANGE/yolov10n_hf_range.elf"
RUN=local-artifacts/yolov10n_hf_reference/results/sys_emu_$NAME

"$PORT/scripts/build_et_slice.sh" "$RANGE" "$ELF"
"$PORT/scripts/run_et_slice.sh" \
--device sys_emu \
--slice-dir "$RANGE" \
--elf "$ELF" \
--launcher "$LAUNCHER" \
--output-dir "$RUN" \
--outer-timeout 1800 \
--launcher-timeout 1740
"$PORT/scripts/validate_device_run.sh" "$RANGE" "$RUN" sys_emu
```

`sys_emu` is intentionally slow. Use it for one layer or a bounded range,
not for routine end-to-end inference. The validator checks the saved command,
ELF, input and weight identities, every selected output, and the PMC record.

### 5. Run the complete graph on ET-SoC1

On a configured ET board host:

```bash
FULL=local-artifacts/yolov10n_hf_reference/full_graph/deterministic
ELF="$FULL/yolov10n_hf_full.elf"
RUN=local-artifacts/yolov10n_hf_reference/results/full_board
MODEL=local-artifacts/yolov10n_hf_reference/model.onnx

"$PORT/scripts/build_et_full.sh" "$FULL" "$ELF"
"$PORT/scripts/run_et_full.sh" \
--device soc1sim \
--full-dir "$FULL" \
--elf "$ELF" \
--launcher "$LAUNCHER" \
--output-dir "$RUN"
"$PORT/scripts/validate_et_full.sh" \
"$FULL" "$RUN" soc1sim "$MODEL" 1
```

The real board path obtains the repository board lock, resets ET-SoC1, and
stores hash-bound run evidence. It never registers the port with the
leaderboard.

## PMCs

`src/ref_pmc.h` programs and reads these counters:

- `hpmcounter3`: minion cycles
- `hpmcounter4/5`: retired instructions on thread 0/1
- `hpmcounter6`: L2 miss requests
- `hpmcounter7`: minion I-cache requests
- `hpmcounter8`: I-cache ET-link requests

A range has one PMC interval around exactly its selected nodes. Full
execution has seven intervals matching the model-parts table above. Input
loading, launcher startup, dumping, and host comparison are outside those
intervals. `validate_device_run.sh` and `validate_et_full.sh` decode and check
the records automatically; `tools/decode_pmc.py` is available for manual
inspection.

Simulator PMCs prove execution and instrumentation. Use real-board PMCs for
performance conclusions.

## Current validation

| Path | Result |
|---|---|
| Full host, deterministic input | PASS, all 308 nodes and 16 checkpoints |
| Full host, checked real image | PASS, strict `output0` 0/1,800 mismatches |
| Host arbitrary ranges | PASS, including every supported operator |
| `sys_emu` bounded ranges | PASS, selected outputs and PMC records |
| ET-SoC1 full real image | PASS, strict `output0`, 16 checkpoints, seven PMCs |

The measured full-board kernel wait for this unoptimized scalar baseline was
721.396 seconds. This is correctness evidence, not a target latency.
`manifests/board_full_summary_strict.json` is the compact, hash-bound record;
raw binaries and logs are deliberately not committed.

## Repository contents

- `src/`: scalar runtime, host/ET runners, and PMC support.
- `tools/`: model download, graph generation, comparison, preprocessing, and
evidence utilities.
- `scripts/`: small host and ET build/run/validate entry points.
- `manifests/`: pinned graph, layer, weight, execution, and board facts.
- `tools/tests/`: tamper and contract regression tests.
- [THIRD_PARTY.md](THIRD_PARTY.md): upstream artifact and license record.

Run lightweight source checks with:

```bash
python3 -m compileall -q "$PORT/tools"
for script in "$PORT"/scripts/*.sh "$PORT"/tools/*.sh; do
bash -n "$script"
done
```

The complete graph and weight package are generated locally because the model
artifact remains under its upstream AGPL-3.0 terms.
16 changes: 16 additions & 0 deletions ported_models/yolov10n_hf_reference/THIRD_PARTY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Third-party model artifact

The model weights and ONNX graph are not part of this repository's Apache-2.0
licensed source and are not committed.

| Component | Source | Revision | File | SHA-256 | License |
|---|---|---|---|---|---|
| YOLOv10n ONNX | `onnx-community/yolov10n` | `57657320425ee34056408a57ad9d29c4d4815bd8` | `onnx/model.onnx` | `a77dd863933f184a19e84361c64b788228a7c7dacc2c78939239a96ad3efca3b` | AGPL-3.0 |

Resolve URL:
`https://huggingface.co/onnx-community/yolov10n/resolve/57657320425ee34056408a57ad9d29c4d4815bd8/onnx/model.onnx?download=true`.

`tools/download_model.py` downloads the pinned artifact into the ignored
`local-artifacts/` cache and verifies its 9,386,116-byte size and checksum
before it is used. The port does not re-export from PyTorch: this ONNX file is
the sole source of topology, weights, tensor names, shapes, and golden outputs.
23 changes: 23 additions & 0 deletions ported_models/yolov10n_hf_reference/artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"schema_version": 1,
"artifacts": {
"yolov10n_onnx": {
"kind": "model",
"format": "onnx",
"dtype": "fp32",
"source": {
"type": "huggingface",
"repo": "onnx-community/yolov10n",
"revision": "57657320425ee34056408a57ad9d29c4d4815bd8",
"filename": "onnx/model.onnx",
"url": "https://huggingface.co/onnx-community/yolov10n/resolve/57657320425ee34056408a57ad9d29c4d4815bd8/onnx/model.onnx?download=true"
},
"size_bytes": 9386116,
"sha256": "a77dd863933f184a19e84361c64b788228a7c7dacc2c78939239a96ad3efca3b",
"license": "AGPL-3.0",
"local_cache": "local-artifacts/yolov10n_hf_reference/model.onnx",
"source_of_truth": true,
"export": "none"
}
}
}
Binary file not shown.
Binary file not shown.
Loading
Loading