feat(setup): install ATOM on bare metal and fold it into the 12h demos; fix the failures of its 12h release run

docs/compatibility.rst

@@ -54,7 +54,8 @@ The following table lists the minimum requirements for running Hyperloom.
 | Python              | >= 3.10                                                |
 +---------------------+--------------------------------------------------------+
 | Inference Framework | SGLang (>= 0.5.12), vLLM (>= 0.21.0),                  |
-|                     | ATOM (preinstalled; see below), plus ``custom``        |
+|                     | ATOM (preinstalled or setup-installed; see below),     |
+|                     | plus ``custom``                                        |
 |                     | benchmark script                                       |
 +---------------------+--------------------------------------------------------+
 | Kernel Languages    | HIP, Triton, FlyDSL                                    |
@@ -131,7 +132,7 @@ The following inference frameworks are supported:
        frameworks within one session
    * - ATOM
      - 7.2.4 (recorded image stack)
-     - AMD out-of-tree engine, launched as ``python3 -m atom.entrypoints.openai_server``. Supports Docker or direct execution in a preinstalled ATOM/ROCm torch environment. The recorded image-based stack used ``rocm/atom-dev:v0.1.7-rc0`` on MI355X, not a universal ATOM pip-version minimum. Other builds require local validation. An unset or empty ``KERNEL_OPT_BACKEND_ORDER`` resolves to GEAK here as it does for every framework, and explicit values are preserved.
+     - AMD out-of-tree engine, launched as ``python3 -m atom.entrypoints.openai_server``. Supports Docker, direct execution in a preinstalled ATOM/ROCm torch environment, or a bare-metal source install with ``--install-framework atom``. The recorded image-based stack used ``rocm/atom-dev:v0.1.7-rc0`` on MI355X, not a universal ATOM pip-version minimum. Other builds require local validation. An unset or empty ``KERNEL_OPT_BACKEND_ORDER`` resolves to GEAK here as it does for every framework, and explicit values are preserved.
    * - ``custom``
      - Host-defined
      - Escape hatch for your own benchmark script; Hyperloom does not manage the server lifecycle. Requires ``HYPERLOOM_BENCHMARK_BACKEND=bypass`` plus ``--framework-path`` (or ``FRAMEWORK_REPO_PATH``) and ``--benchmark-scripts-dir`` (or ``HYPERLOOM_BYPASS_SCRIPTS_DIR``); the CLI exits with status 2 when any of the three is missing.
@@ -188,8 +189,15 @@ Hyperloom does not install ROCm or torch itself.
 
 For ATOM, ``baremetal`` means running directly in the development machine's
 selected Python environment, including when the development platform itself is
-a container; it does not start an additional Docker container. ATOM must already
-be installed. Verify a real ``import atom`` and a non-empty ``torch.version.hip``
+a container; it does not start an additional Docker container. ATOM is either
+already installed there or installed by setup with ``--install-framework atom``,
+which builds AITER and then ATOM (default ``ATOM_REF``: the commit
+``rocm/atom-dev:v0.1.7-rc0`` was built from) against the existing ROCm torch in
+that Python. That Python must not also serve SGLang or vLLM: ATOM registers
+plugins that both engines load by default, so setup refuses to install ATOM
+where either imports (including a ROCm 10 vLLM venv built over that Python),
+and refuses to install SGLang, shared vLLM or ROCm 10 source vLLM where ATOM
+imports. Run ATOM in a separate container instead. Verify a real ``import atom`` and a non-empty ``torch.version.hip``
 with that Python, keep its executable first on ``PATH`` for Magpie's ``python3``
 launch, and use ``PYTHON`` with ``INFERENCE_OPTIMIZER_FORCE_PYTHON=1`` if pinning
 the interpreter. Keep any activated venv consistent; ``/opt/venv`` is not required.
@@ -199,9 +207,9 @@ Run ``python -m hyperloom.inference_optimizer.setup --check-only --
 selected interpreter first. Only after approval, repeat without ``--check-only``:
 ``none`` skips framework installation but can still write configuration and
 apply ROCm hotfixes. Preserve the selected ``USER_DATA_PATH``; an existing setup
-need not be repeated. Hyperloom does not install ATOM and does not assert a
-minimum ATOM package version derived from a Docker tag. Import checks establish
-local prerequisites, not end-to-end validation of every stack.
+need not be repeated. Hyperloom does not assert a minimum ATOM package version
+derived from a Docker tag. Import checks establish local prerequisites, not
+end-to-end validation of every stack.
 
 .. list-table::
    :header-rows: 1

docs/install/install.md

@@ -132,7 +132,7 @@ The backend runs `install_baremetal.sh` in five phases:
 
 1. **Base preflight**: Checks ROCm, GPU arch, ROCm torch, torch/triton alignment,
    and serving framework imports.
-2. **Framework install**: Optionally installs the SGLang or vLLM framework layer.
+2. **Framework install**: Optionally installs the SGLang, vLLM or ATOM framework layer.
 3. **ROCm hotfix**: Applies the profiler hotfix when the ROCm stack is eligible,
    covering both `/opt/rocm/lib` and PyTorch's bundled `torch/lib/`.
 4. **Credentials**: Resolves LLM gateway credentials into `.env`.
@@ -396,7 +396,7 @@ set of accepted shapes, including split entrypoints and self-hosted gateways.
 Make sure the host already provides the required base environment:
 
 - ROCm runtime and a ROCm-built torch.
-- A serving framework (SGLang or vLLM) importable in the active Python.
+- A serving framework (SGLang, vLLM or ATOM) importable in the active Python.
 - `git` for the dependency checkouts the optimization skill performs.
 
 With that in place, open the repository root in the agent and paste a launch

docs/reference/environment-variables.md

@@ -721,8 +721,9 @@ The following variables configure framework source discovery and path overrides.
 | `INFERENCE_`<br>`OPTIMIZER`<br>`_STRICT_PATHS`                | `1` when CLI bootstraps                                                | When `1`, missing path env raises instead of falling back to discovery. Set by the CLI at session start; do not override unless debugging.              |
 | `HYPERLOOM_`<br>`SGLANG_PA`<br>`TCH_EXACT`<br>`_VERSIONS`           | Unset                                                                  | Pin the sglang server-patch step to specific upstream versions; advanced compatibility option.                                                          |
 | `HYPERLOOM_`<br>`ENABLE`<br>`_PATCH`                          | `1`                                                                    | Set to `0` to skip the in-place server patch step (useful when the upstream is already pre-patched).                                                    |
-| `HYPERLOOM_`<br>`SKIP_FRAME`<br>`WORK_CHECK`                  | Unset (check enabled)                                                  | Truthy skips the `optimize` preflight gate that requires the selected serving framework to be importable and a ROCm build. Last resort: when the server runs elsewhere, set `BENCHMARK_BASE_URL` instead, which exempts the check and configures the supported path. The gate already stays out of the way for `xdit`/`custom` (server-less), external multi-node, and any framework `install_baremetal.sh` cannot install (`atom`), where it warns instead of blocking. |
+| `HYPERLOOM_`<br>`SKIP_FRAME`<br>`WORK_CHECK`                  | Unset (check enabled)                                                  | Truthy skips the `optimize` preflight gate that requires the selected serving framework to be importable and a ROCm build. Last resort: when the server runs elsewhere, set `BENCHMARK_BASE_URL` instead, which exempts the check and configures the supported path. The gate already stays out of the way for `xdit`/`custom` (server-less) and external multi-node. |
 | `AITER_REF` | Unset | Optional bare-metal AITER install pin. When unset, the installer selects the newest tag compatible with the installed torch/triton stack. |
+| `ATOM_REF` | `fe1099b15ddc52e6873e1932935f722864cdeee0` | Full commit SHA that `--install-framework atom` checks out from `ATOM_REPO` (default `https://github.com/ROCm/ATOM.git`) into `ATOM_ROOT` (default `<deps root>/atom`). The default is the commit `rocm/atom-dev:v0.1.7-rc0` was built from. |
 
 ---
 

examples/README.md

@@ -124,7 +124,7 @@ The backend runs `install_baremetal.sh` in five phases:
 
 1. **Base preflight**: checks ROCm, GPU arch, ROCm torch, torch/triton alignment,
    and serving framework imports.
-2. **Framework install**: optionally installs the SGLang or vLLM framework layer.
+2. **Framework install**: optionally installs the SGLang, vLLM or ATOM framework layer.
 3. **ROCm hotfix**: applies the profiler hotfix when the ROCm stack is eligible.
 4. **Credentials**: resolves LLM gateway credentials into `.env`.
 5. **Runtime env**: persists bare-metal runtime vars (framework, ROCm/venv roots,
@@ -175,21 +175,16 @@ run:
 
 - [`3h`](hyperloom-qwen3-8b-3h/SKILL.md) — Qwen3-8B, short no-kernel run; best
   for a first end-to-end check.
-- [`12h`](hyperloom-qwen3-14b-fp8-12h/SKILL.md) — Qwen3-14B-FP8, medium-length FP8 run.
-- [`12h forge`](hyperloom-qwen3-14b-fp8-12h-forge/SKILL.md) — the same run on the
-  KernelForge kernel backend.
-- [`12h atom`](hyperloom-qwen3-14b-fp8-12h-atom/SKILL.md) — the same run on ATOM,
-  using the user's selected `docker` or `baremetal` mode from setup; if no mode is
-  selected, ask before continuing. `baremetal` uses the development machine's
-  existing ATOM/ROCm Python environment, even if that platform is a container
-  (no additional Docker). Setup uses `--install-framework none
-  --frameworks atom --require-frameworks`, first with `--check-only` and then
-  with approval if setup is needed. The KERNEL_AGENT phase runs GEAK, the same
-  backend the other frameworks get when none is named.
-- [`12h atom forge`](hyperloom-qwen3-14b-fp8-12h-atom-forge/SKILL.md) — the ATOM
-  run with `KERNEL_OPT_BACKEND_ORDER=forge`, so the KERNEL_AGENT phase is the
-  per-kernel KernelForge backend. Same workload, budget and phase split as
-  `12h atom`, so the two are directly comparable.
+- [`12h`](hyperloom-qwen3-14b-fp8-12h/SKILL.md) — Qwen3-14B-FP8, medium-length FP8
+  run on SGLang, vLLM or ATOM. For ATOM it follows its ATOM section, using the
+  user's selected `docker` or `baremetal` mode from setup; if no mode is selected,
+  ask before continuing. `baremetal` uses the development machine's ATOM/ROCm
+  Python environment, even if that platform is a container (no additional
+  Docker); setup verifies an existing ATOM with `--install-framework none
+  --frameworks atom --require-frameworks` or, with approval, installs it with
+  `--install-framework atom`.
+- [`12h forge`](hyperloom-qwen3-14b-fp8-12h-forge/SKILL.md) — the same run, on any
+  of the three frameworks, with the KernelForge kernel backend.
 - [`custom advanced`](hyperloom-custom-advanced/SKILL.md) — user-selected model,
   framework, TP/EP, concurrency, ISL/OSL, precision, budget, phase toggles, and
   advanced CLI flags.