Skip to content

Add prgenv-gnu-dpcpp: Intel DPC++ SYCL for GH200 aarch64#331

Open
dganellari wants to merge 5 commits into
eth-cscs:mainfrom
dganellari:prgenv-gnu-dpcpp
Open

Add prgenv-gnu-dpcpp: Intel DPC++ SYCL for GH200 aarch64#331
dganellari wants to merge 5 commits into
eth-cscs:mainfrom
dganellari:prgenv-gnu-dpcpp

Conversation

@dganellari

Copy link
Copy Markdown
Member

Add experimental recipe that extends prgenv-gnu/25.11 with Intel DPC++ (SYCL compiler with CUDA backend) for GH200 on Alps aarch64 systems.

The Intel oneAPI binary distribution is x86_64 only, so DPC++ is built from source using a custom Spack package (llvmdpcpp) that fetches the intel/llvm sycl branch.

Changes:

  • New recipe at recipes/prgenv-gnu-dpcpp/25.11/gh200/
  • Custom Spack package repo/packages/llvmdpcpp/package.py
  • Spack v1.1.1, spack-packages 7178c89 (2026-03-09)
  • Registered in config.yaml (build only, no deploy)

Tested on Santis and Clariden up to 2262 nodes (9048 GPUs).

Add experimental recipe that extends prgenv-gnu/25.11 with Intel DPC++
(SYCL compiler with CUDA backend) for GH200 on Alps aarch64 systems.

The Intel oneAPI binary distribution is x86_64 only, so DPC++ is built
from source using a custom Spack package (llvmdpcpp) that fetches the
intel/llvm sycl branch.

Changes:
- New recipe at recipes/prgenv-gnu-dpcpp/25.11/gh200/
- Custom Spack package repo/packages/llvmdpcpp/package.py
- Spack v1.1.1, spack-packages 7178c89 (2026-03-09)
- Registered in config.yaml (build only, no deploy)

Tested on Santis and Clariden up to 2262 nodes (9048 GPUs).
@bcumming

Copy link
Copy Markdown
Member

cscs-ci run alps;system=daint;uarch=gh200;uenv=prgenv-gnu-dpcpp:25.11

Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/environments.yaml Outdated
Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/environments.yaml Outdated
Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/environments.yaml Outdated
Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/environments.yaml Outdated
Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/compilers.yaml Outdated
Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/extra/reframe.yaml Outdated
@msimberg

Copy link
Copy Markdown
Collaborator

cscs-ci run alps;system=daint;uarch=gh200;uenv=prgenv-gnu-dpcpp:25.11

@dganellari

Copy link
Copy Markdown
Member Author

@msimberg seems like it failed at some NCCL test.

Comment thread recipes/prgenv-gnu-dpcpp/25.11/gh200/environments.yaml Outdated
@msimberg

Copy link
Copy Markdown
Collaborator

cscs-ci run alps;system=daint;uarch=gh200;uenv=prgenv-gnu-dpcpp:25.11

@msimberg

Copy link
Copy Markdown
Collaborator

Oh, the llvmdpcpp package doesn't pin the commit but just uses the latest commit on the sycl branch. This makes the build very unreproducible, and I'm guessing this is why the build is failing now, but didn't before.

@dganellari do you have a way to find out which commit was actually used for the previous (successful) build? If yes, could you check that and pin the commit? If no, let's go with the last successful build and just deploy that instead (with a big TODO on the version saying it should be pinned).

@msimberg

Copy link
Copy Markdown
Collaborator

I deployed the latest build to daint:

$ uenv image find prgenv-gnu-dpcpp
uenv                       arch   system  id                size(MB)  date
prgenv-gnu-dpcpp/25.11:v1  gh200  daint   e8bc614187bd742a   5,769    2026-06-19

@dganellari will look into the llvmdpcpp version/commit still so let's keep the pr open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants