Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![Tests](https://github.com/AMD-AGI/Hyperloom/actions/workflows/tests-coverage.yml/badge.svg)](https://github.com/AMD-AGI/Hyperloom/actions/workflows/tests-coverage.yml)
[![Lint](https://github.com/AMD-AGI/Hyperloom/actions/workflows/lint.yml/badge.svg)](https://github.com/AMD-AGI/Hyperloom/actions/workflows/lint.yml)
[![Version](https://img.shields.io/badge/version-1.1.1-blue)](https://github.com/AMD-AGI/Hyperloom/blob/main/pyproject.toml)
[![Version](https://img.shields.io/badge/version-1.1.2-blue)](https://github.com/AMD-AGI/Hyperloom/blob/main/pyproject.toml)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](https://github.com/AMD-AGI/Hyperloom/blob/main/LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://github.com/AMD-AGI/Hyperloom/blob/main/pyproject.toml)

Expand Down
25 changes: 13 additions & 12 deletions docs/compatibility.rst
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The following table lists the minimum requirements for running Hyperloom.
+---------------------+--------------------------------------------------------+
| Operating System | Ubuntu 24.04 (recommended); Ubuntu 22.04 (legacy) |
+---------------------+--------------------------------------------------------+
| ROCm Version | 7.2.x |
| ROCm Version | 7.2.x, 10.0 |
+---------------------+--------------------------------------------------------+
| Python | >= 3.10 |
+---------------------+--------------------------------------------------------+
Expand All @@ -75,7 +75,7 @@ The following table lists the validated Hyperloom version and component combinat
+-------------------+---------------------------+------------------------+------------------------------------+---------------+-------------+-----------------------------+
| Hyperloom version | Component | GPU | ROCm version | Ubuntu | Python | GitHub |
+===================+===========================+========================+====================================+===============+=============+=============================+
| 1.1.1 | `TraceLens 1.0.0`_ | Hardware-agnostic | No dependency | OS-independent| >= 3.6 | |tracelens-github| |
| 1.1.2 | `TraceLens 1.0.0`_ | Hardware-agnostic | No dependency | OS-independent| >= 3.6 | |tracelens-github| |
+ +---------------------------+------------------------+------------------------------------+---------------+-------------+-----------------------------+
| | `GEAK 4.0.0`_ | MI300X, MI325X, MI355X | 6.4.x, 7.0.x, 7.1.x, 7.2.x, 10.0.0 | 22.04, 24.04 | 3.8, 3.12 | |geak-github| |
+ +---------------------------+------------------------+------------------------------------+---------------+-------------+-----------------------------+
Expand Down Expand Up @@ -160,8 +160,9 @@ mirror, set the registry prefix accordingly.
* - ``rocm/atom-dev:v0.1.7-rc0``
- MI355X (verified); MI300X / MI325X untested

The vLLM image entrypoint is ``vllm serve``, so override it (for example
``--entrypoint tail``) when starting a long-running Hyperloom container.
The images default to ``/bin/bash``, which exits at once in a detached
container, so override the entrypoint (for example ``--entrypoint tail``) when
starting a long-running Hyperloom container.

The ``rocm/vllm`` image serves an installed wheel, so ``install.sh`` turns its
``/app/vllm`` checkout into the patchable source tree: it pins the tree to vLLM
Expand Down Expand Up @@ -226,11 +227,11 @@ overridable via env (``SGLANG_REF``, ``SGLANG_ROCM_EXTRA``, ``VLLM_VERSION``,
``VLLM_ROCM_VARIANT``, ``VLLM_INSTALL_METHOD``, ``VLLM_REPO``, ``VLLM_SOURCE_REF``,
``VLLM_ROOT``) for hosts that need a different pinned stack.

The table above is the validated combination, and ROCm 7.2.x under a single
``/opt/rocm`` prefix is the layout to prefer. A host where ROCm arrives as
TheRock's pip wheels instead, split across the ``_rocm_sdk_*`` namespace
packages, is handled rather than validated: the bare-metal installer probes
those packages for library resolution and, before a framework source build,
supplies the devel headers and toolchain root from them, so setup does not fail
on that layout. Only ROCm 7.0.x and 7.2.x have a published ``amd-sglang`` wheel;
any other stack falls back to a source install. See :doc:`/install/install`.
The table above is the validated combination. ROCm 7.2.x is validated under a
single ``/opt/rocm`` prefix. ROCm 10.0 arrives as TheRock's pip wheels, split
across the ``_rocm_sdk_*`` namespace packages, which is the layout the ``rocm10``
images are built from and the one the ROCm 10 routes are validated on: the
bare-metal installer probes those packages for library resolution and, before a
framework source build, supplies the devel headers and toolchain root from them.
Only ROCm 7.0.x and 7.2.x have a published ``amd-sglang`` wheel; any other stack
falls back to a source install. See :doc:`/install/install`.
2 changes: 1 addition & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"""

# Keep in sync with pyproject.toml [project].version.
version_number = "1.1.1"
version_number = "1.1.2"

html_theme = "rocm_docs_theme"
html_theme_options = {
Expand Down
34 changes: 17 additions & 17 deletions docs/install/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Before installing Hyperloom, ensure the following requirements are met.
From the agent terminal in that workspace, install the published release wheel:

```bash
pip install hyperloom-inference-optimizer==1.1.1 --target .
pip install hyperloom-inference-optimizer==1.1.2 --target .
```

It is normal for the current directory to contain many Python package directories
Expand Down Expand Up @@ -106,19 +106,20 @@ Requirements:
- Ubuntu 24.04 is the recommended host OS for bare-metal setup. vLLM 0.28.0+
requires Ubuntu 24.04 or newer; on Ubuntu 22.04, downgrade vLLM (for example
``VLLM_VERSION=0.27.1``) or use Docker mode instead.
- ROCm runtime and ROCm torch are already installed. ROCm 7.2.x is the
recommended and validated baseline; see
{doc}`Compatibility </compatibility>` for the combination the release is
- ROCm runtime and ROCm torch are already installed. ROCm 7.2.x and ROCm 10.0
are the validated stacks; on ROCm 10.0, bare-metal vLLM is built from source.
See {doc}`Compatibility </compatibility>` for the combination the release is
tested against. Other ROCm versions are not validated.
- `git` is available for dependency checkouts.
- A serving framework is either already installed, or setup might install one.
- **Base Python on this GPU host** — the interpreter `install_baremetal.sh`
resolves, not a child venv:
- Python 3.10+ for SGLang and general operation.
- Exactly Python 3.12 when setup installs vLLM. vLLM ROCm wheels are
built for 3.12 only. vLLM defaults to an isolated framework venv, but that
venv is created from the base interpreter and inherits its version; isolated
mode does not relax the requirement.
- Exactly Python 3.12 when setup installs vLLM from the ROCm 7.2.x wheel,
which is built for 3.12 only; the ROCm 10.0 source build accepts Python
>= 3.10, < 3.15. vLLM defaults to an isolated framework venv, but that venv
is created from the base interpreter and inherits its version; isolated mode
does not relax the requirement.

In this scenario, `/hyperloom-setup` runs the packaged setup backend on the host:

Expand Down Expand Up @@ -146,15 +147,14 @@ and a ROCm-built torch. Two framework-install details are worth knowing:
best-effort, trying `libopenmpi3t64` (Ubuntu 24.04's 64-bit `time_t` name) and
`libopenmpi3`. It skips silently when the library already resolves, when `apt`
is unavailable, or when not running as root.
- **ROCm as pip wheels**: the validated baseline is ROCm 7.2.x under a single
`/opt/rocm` prefix, and that is the layout to prefer. ROCm can instead arrive
as TheRock's wheels, split across the `_rocm_sdk_core`,
`_rocm_sdk_libraries` and `_rocm_sdk_devel` namespace packages. The installer
handles that layout so it does not fail setup on it, but such a host is not a
tested configuration: Phase 1 probes all three packages, including their
`rocm_sysdeps` and `host-math` subdirectories, so the gate does not report
libraries as missing that the loader does resolve at runtime; and before a
source build, which needs hipBLAS/hipSPARSE/thrust headers, it adds
- **ROCm as pip wheels**: ROCm 7.2.x is validated under a single `/opt/rocm`
prefix. ROCm 10.0 arrives as TheRock's wheels, split across the
`_rocm_sdk_core`, `_rocm_sdk_libraries` and `_rocm_sdk_devel` namespace
packages, which is the layout the `rocm10` images are built from and the one
the ROCm 10 routes are validated on. Phase 1 probes all three packages,
including their `rocm_sysdeps` and `host-math` subdirectories, so the gate
does not report libraries as missing that the loader does resolve at runtime;
and before a source build, which needs hipBLAS/hipSPARSE/thrust headers, it adds
`rocm-sdk-devel` pinned to the installed `rocm-sdk-core` version, expands it
with `rocm-sdk init`, and exports the root `rocm-sdk path --root` reports so
the compiler and its include tree come from the same package. All of this is a
Expand Down
1 change: 1 addition & 0 deletions docs/install/slurm.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ optimization. Model names must exist in your key's catalog:
|---|---|---|
| `HL_SHM_SIZE` | `64g` | docker `--shm-size`; raise it for high concurrency. |
| `HL_CONTAINER_RUNTIME` | `auto` | Force `docker` or `pyxis`. |
| `HL_ORPHAN_MIN_AGE_S` | `900` | Minimum age in seconds before a container whose `CLAW_SESSION_ID` has no live client is reclaimed as orphaned; younger containers are left alone. |
| `HL_GPU_TYPE_OVERRIDE` | — | Override `--gpu-type` (lowercase) when hardware differs from the table row. |
| `HL_SHARED_MOUNT` | `/path` | Shared FS bind-mounted into the container. |
| `HL_DATA_ROOT` | `<shared-mount>/hyperloom-slurm` | Artifact root. |
Expand Down
1 change: 1 addition & 0 deletions docs/reference/environment-variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ The following variables configure filesystem paths for Hyperloom's runtime depen
| `HYPERLOOM`<br>`_CACHE_`<br>`DIR` | No | `$REPO_ROOT`<br>`/.cache` | Writable, repo-local base for auto-cloned open-source deps (TraceLens, Magpie, etc.), cloned per revision as `<name>@<sha>`. Not under `$TMPDIR` so a reaper cannot wipe it mid-run. |
| `KERNELFORGE`<br>`_PROJECT_`<br>`ROOT` | No | `$USER_DATA_PATH/kernelforge`, else `~/.cache/hyperloom/kernelforge` | Writable root for forge's own state and for resource-tree overrides. Holds the learned knowledge base (`knowledge_base/<backend>/learned/`), the tuning DB, postmortems and `forge_experiments/`. A subtree placed here also **overrides the copy packaged inside `kernelforge`** — a `serving_patches/` or `examples/` directory under this root wins over the shipped one, which is the supported way to try a patch or a task without editing site-packages. Must be writable: it deliberately never resolves to the installed package directory or to the cwd. **This is the replacement for the removed `FORGE_PATH`**, which nothing reads any more — a stale `FORGE_PATH` is still forwarded (the `FORGE_` prefix is on the dotenv allowlist) and then ignored. |
| `SKIP_FORGE`<br>`_PROFILING` | No | Unset (the extra is installed) | Set to `1` to make `install.sh` skip `pip install -e "$REPO_ROOT[forge-profiling]"`. That extra is rocprof-compute's own dependency set (~20 wheels, including the exact `kaleido==0.2.1` / `astunparse==1.6.2` pins ROCm 7.2.x requires); without it forge's profiler degrades to the lightweight PMC path instead of System Speed-of-Light + roofline. Installed by default on purpose — the previous gate made this a silent skip on every pod. |
| `ROCPC_VENV` | No | `/opt/rocprof-compute-venv` | Private venv that rocprof-compute's analyze mode runs in, created by `install.sh` from the tool's own `requirements.txt` so its exact pins cannot displace the serving image's numpy and pandas. `rocpc_profile.py` derives the same path. Without the venv, analyze degrades and profiling still collects. |
| `MAGPIE_PATH` | No | Resolved from installed `Magpie` package unless explicitly set | Magpie package root for benchmark wrappers and patch inspection. |
| `INFERENCE_`<br>`OPTIMIZER`<br>`_MODEL_PATH_ROOTS` | No | Built-in model roots such as `/models` and `/shared_nfs` | `os.pathsep`-separated allowlist for absolute model paths restored from `state.json` during a resume. HuggingFace-style repo IDs remain allowed. Set this when production models live outside the built-in roots. |
| `INFERENCE_`<br>`OPTIMI`<br>`ZER_SES`<br>`SION_DIR` | No (multi-node) | Unset | Last-resort session root for multi-node crash-log collection. Point it at one session dir, never at `$USER_DATA_PATH`. |
Expand Down
Loading
Loading