Skip to content

Add NVDLA nv_small partitions (a/c/m/o/p) on asap7 - #68

Merged
mguthaus merged 10 commits into
mainfrom
add-nvdla
Apr 19, 2026
Merged

mguthaus merged 10 commits into
mainfrom
add-nvdla

Conversation

@mguthaus

Copy link
Copy Markdown
Contributor

Summary

Adds NVIDIA's NVDLA (Deep Learning Accelerator, nv_small config) as an asap7 benchmark to HighTide. NVDLA is split into 5 partitions (a, c, m, o, p), each a separate top-level design sharing one source tree and a pool of 57 FakeRAM LEF/LIB pairs. Split out from closed PR #37, adapted to the current Bazel-primary flow.

  • Upstream: nvdla/hw#nv_small @ 771f20cc (pinned submodule)
  • Release-only RTL for now (dev-mode wiring deferred — see Wire NVDLA dev-mode RTL regeneration into Bazel #67)
  • SYNTH_HIERARCHICAL=1 on all partitions
  • partition_o runs two clocks (nvdla_core_clk 2000 ps, nvdla_falcon_clk 2500 ps); others use a single 1500 ps nvdla_core_clk

Per-partition verification (NRP Nautilus K8s, full RTL→GDS)

Partition Top Clocks FakeRAMs K8s status Elapsed
a NV_NVDLA_partition_a core 1500 ps 2 Succeeded first-try
c NV_NVDLA_partition_c core 1500 ps 84 macros Succeeded 8531s (3 retunes)
m NV_NVDLA_partition_m core 1500 ps 0 Succeeded first-try
o NV_NVDLA_partition_o core 2000 ps + falcon 2500 ps 9 Succeeded 15639s
p NV_NVDLA_partition_p core 1500 ps 4 Succeeded 5734s

partition_c tuning journey (the hard one — 84 macros on an asap7 die)

# util / addon / halo Furthest stage Failure
1 50 / 0.25 / 2 post-CTS DPL DPL-0036: clkbuf couldn't legalize near cbuf_ram_bank31
2 45 / 0.30 / 5 MPL floorplan MPL-0003: no valid tilings (halo×84 ate tiling budget)
3 48 / 0.25 / 3 GRT GRT-0116: placed but routed-congested
4 40 / 0.30 / 3 6_final Succeeded, 42% final util

Traded peak util for flow completion. PPA sweep is a follow-up.

Fixes / deviations from closed PR #37

  • New BUILD.bazel files: closed PR had none. Per-partition hightide_design() calls, per-partition SRAM filegroups, shared designs/src/NVDLA/BUILD.bazel exposing :rtl, :rtl_release, :include.
  • stage_data = {\"synth\": [\"//designs/src/NVDLA:include\"]} on each partition so yosys can resolve .vh headers in the Bazel sandbox (otherwise fails with Can't open include file 'simulate_x_tick.vh').
  • Fixed verilog.mk — was find \$(VMOD) -name \"*.vh\" where \$(VMOD) is undefined. Changed to find \$(BENCH_DESIGN_HOME)/src/NVDLA/vmod -name \"*.vh\".
  • Removed ~545 .vcp files — upstream tmake preprocessor inputs, never consumed by the flow. Source tree drops from 1072 → 527 vmod files.
  • Deduplicated set_ideal_network block in partition_c/constraint.sdc (was repeated 2×).

Test plan

  • All 5 partitions clean bazel build :partition_X_final on Nautilus K8s
  • Dev-mode regeneration not wired — tracked in Wire NVDLA dev-mode RTL regeneration into Bazel #67
  • PPA sweep on partition_c (currently 42% util) — follow-up
  • partition_o SDC uses set_false_path between clocks instead of set_clock_groups -asynchronous — works end-to-end, idiomatic cleanup is a nit

Follow-ups

mguthaus added 10 commits April 18, 2026 19:34
NVDLA (NVIDIA Deep Learning Accelerator, nv_small config) sources are
pinned at github.com/nvdla/hw#nv_small commit 771f20cc.

dev/setup.sh installs OpenJDK 11, OpenSSL 1.0.2u, Perl 5.10.1,
Python 2.7.18, and SystemC 2.3.0 locally under dev/packages/, then
runs `perl tools/bin/tmake -build vmod` and substitutes a pre-baked
NV_NVDLA_cfgrom.v (the upstream cfgrom file is auto-generated and
contains an enum literal yosys parses incorrectly).

Pre-generated release RTL is committed in a follow-up commit so the
dev path is opt-in.
Commits the vmod/ tree generated by `tmake -build vmod` against the
nv_small project. Synthesis consumes vmod/{rams/synth,vlibs,nvdla}
plus macros.v (per verilog.mk); vmod/rams/{model,fpga} are
simulation-only and would not be generated by the dev path either.

macros.v provides FakeRAM wrappers (RAMDP_*/RAMPDP_*) bridging the
NVDLA RAM port names to the asap7 fakeram pins.

verilog.mk: fix VERILOG_INCLUDE_DIRS — original used an undefined
$(VMOD), should glob under vmod/ to pick up include/ and vlibs/ .vh
files (NV_HWACC_NVDLA_tick_defines.vh, simulate_x_tick.vh,
assertion_*.vh).

BUILD.bazel exposes :rtl as a release-only filegroup. Dev-mode
generation is intentionally not wired as a Bazel genrule because
setup.sh downloads multi-hundred-MB toolchains and runs for hours;
run setup.sh manually and re-stage vmod/ to refresh.

.vcp source files (the pre-tmake `vlib` inputs, identical content to
the .v outputs) were trimmed — only .v files are read by the flow.
~57 fakeram_<W>x<D>_1r1w pairs covering the union of macros referenced
across all 5 partitions. Per-partition subsets are exposed as
:sram_lefs_{a,c,o,p} / :sram_libs_{a,c,o,p} filegroups (mirroring the
ADDITIONAL_LEFS/LIBS lists in each partition's config.mk). partition_m
references no FakeRAM so it pulls neither.

A :sram_lefs_all / :sram_libs_all pair is also exposed for any
consumer that wants the full set.
Five top-level partitions of NVDLA nv_small. All share the same vmod
RTL via DESIGN_NICKNAME=NVDLA + DESIGN_RESULTS_NAME=NVDLA_partition_X
in config.mk; on the Bazel side each partition is a separate
hightide_design() that pulls //designs/src/NVDLA:rtl with its own SDC
and FakeRAM subset.

Per-partition tuning (from closed PR #37):
  a: util=55, density+0.20, halo=5, FakeRAM={256x16,272x16}
  c: util=50, density+0.25, halo=2, FakeRAM={64x256,64x16,6x128,
                                             66x8,11x128}
  m: util=55, density+0.20, halo=5, no FakeRAM
  o: util=45, density+0.15, halo=7, FakeRAM={18x128,8x256,4x256,
                                             7x256,66x64,15x80,22x60,
                                             32x128,9x80}, two clocks
                                             (core 2000ps + falcon 2500ps)
  p: util=45, density+0.25, halo=5, FakeRAM={16x160,65x160,14x80,66x80}

All partitions: SYNTH_HIERARCHICAL=1, TNS_END_PERCENT=100, single
1500ps nvdla_core_clk except partition_o.
The release PR copied the reset/clock ideal-network list twice; remove
the duplicate block. Cosmetic cleanup, no functional change.
Yosys on Nautilus failed with "Can't open include file
'simulate_x_tick.vh'" because VERILOG_INCLUDE_DIRS pointed at dirs
that were never staged into the Bazel action sandbox. The :rtl
filegroup ships .v files only, so headers lived outside the inputs.

- designs/src/NVDLA:include now globs both vmod/include/*.vh and
  vmod/vlibs/*.vh (vlibs is on VERILOG_INCLUDE_DIRS too).
- Each partition BUILD.bazel passes //designs/src/NVDLA:include via
  stage_data['synth'] (same pattern bp_processor uses for its header
  tree).
partition_c synth/floorplan/place/CTS all completed on Nautilus, but
post-CTS detailed-placement legalization failed on 1/140726 cells
(a clkbuf adjacent to cbuf_ram_bank31). Align the tuning knobs with
partition_a/p, which both legalized end-to-end:

  CORE_UTILIZATION        50 -> 45
  PLACE_DENSITY_LB_ADDON  0.25 -> 0.30
  MACRO_PLACE_HALO        2 2 -> 5 5
halo=2 failed DPL legalization (1 clkbuf near cbuf_ram_bank31 had no
slot). halo=5 then failed MPL-0003 "no valid tilings" with 84
macros — too much halo area. Try an intermediate halo with a small
die bump:

  CORE_UTILIZATION        50 -> 48 (keep die almost the same)
  PLACE_DENSITY_LB_ADDON  0.30 -> 0.25 (back to original)
  MACRO_PLACE_HALO        5 5 -> 3 3

If this also fails, STOP and document in REPORT per PROMPT3.
Attempt 3 (util=48 halo=3) reached global routing but hit
GRT-0116 "finished with congestion". Enlarge the die:

  CORE_UTILIZATION        48 -> 40
  PLACE_DENSITY_LB_ADDON  0.25 -> 0.30
  MACRO_PLACE_HALO        3 3 (unchanged)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant