Skip to content

bazel: add bazelisk run //:cmake — dependency prefix for plain CMake builds#10822

Open
oharboe wants to merge 27 commits into
The-OpenROAD-Project:masterfrom
oharboe:bazel-cmake-deps
Open

bazel: add bazelisk run //:cmake — dependency prefix for plain CMake builds#10822
oharboe wants to merge 27 commits into
The-OpenROAD-Project:masterfrom
oharboe:bazel-cmake-deps

Conversation

@oharboe

@oharboe oharboe commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #10812 (the diff shows both until that merges; review this PR from the `bazel: add bazelisk run //:cmake commit).

Problem

Building OpenROAD with CMake requires etc/DependencyInstaller.sh: sudo, distro-specific package lists, and from-source builds of boost, or-tools, abseil, swig, bison, spdlog and more into /usr/local or ~/.local. The bazel module graph already carries every one of those dependencies at pinned versions and builds them hermetically.

Change

A bazelisk run //:cmake target that materializes those dependencies into <workspace>/deps for a plain CMake workflow:

bazelisk run //:cmake
cmake -DCMAKE_TOOLCHAIN_FILE=deps/toolchain.cmake -B build .
cmake --build build -j$(nproc)

deps/ (~1 GB, gitignored, regenerated on every run) contains:

  • the merged header tree and every static archive of the dependency closure (boost 1.89, abseil, or-tools 9.15 with scip/highs/glpk/soplex/bliss/protobuf, spdlog, fmt, eigen, lemon, cudd, zlib, yaml-cpp, googletest, openmp),
  • the build tools: swig 4.3.0, bison 3.3.2, flex 2.6.4,
  • Tcl 9 (archive, headers, and its script library at lib/tcl9.0),
  • a hermetic CPython 3.13 (interpreter, headers, libpython, stdlib),
  • the hermetic clang/libc++ toolchain from bazel: switch C++ toolchain to hermetic-llvm; add etc/bazel-hermetic #10812 behind generated bin/cc/bin/c++ wrappers, and a toolchain.cmake that points CMake at all of the above.

Host requirements: bazelisk, cmake ≥ 3.16, ninja or make, git, bash. No sudo, no compiler packages, nothing written outside the workspace. This is additive and opt-in: DependencyInstaller.sh/Build.sh are untouched, and the CMake workflow afterwards is plain cmake — bazel is only the provisioning tool.

Mechanics

  • Compiler/linker flags are extracted from the resolved C++ toolchain at analysis time (cc_common.get_memory_inefficient_command_line, the rules_foreign_cc technique) and rewritten to relocatable paths, so the wrappers cannot drift from the bazel toolchain when the llvm module is bumped. Build-type flags (-O, -g, -std=, warnings) are left to CMake.
  • Archives are linked through a single interface target carrying a --start-group pool; order becomes irrelevant and the absl/or-tools/scip/protobuf interdependencies need no per-package attribution. The seven or-tools alwayslink archives are --whole-archived.
  • Package config shims (Boost, absl, ortools, spdlog, fmt, Eigen3, LEMON, GTest, yaml-cpp) live in cmake-deps/shims/; TCL, ZLIB, CUDD, SWIG, BISON, FLEX, Python3, OpenMP are steered through their existing Find modules by cache variables in toolchain.cmake. Dependency versions are parsed from MODULE.bazel into a generated deps-versions.cmake.
  • The multiplexed ~190 MB LLVM binary is staged once; the tool names are recreated as relative symlinks.

One change to existing CMake files: src/gui/CMakeLists.txt now only requires OpenGL when the GUI is actually built (Qt5_FOUND AND BUILD_GUI); OpenGL::GL is only referenced inside that branch, and the unconditional find_package(OpenGL REQUIRED) made GUI-less hosts install OpenGL headers for a CLI build.

Scope

  • Linux x86_64. macOS needs a platform-conditional pool file (ld64 has no --start-group/--whole-archive) and a different C++ runtime story; deliberately a follow-up.
  • The GUI is not part of the prefix: host Qt5 is a libstdc++ build and cannot link against libc++ archives. toolchain.cmake sets BUILD_GUI=OFF; the build degrades to CLI exactly like a host without Qt.
  • The produced binary is a clang/libc++ build, matching the bazel build.

Verification

On Ubuntu (cmake 4.2.3, ninja), from a clean checkout:

$ bazelisk run //:cmake
$ cmake -DCMAKE_TOOLCHAIN_FILE=deps/toolchain.cmake -G Ninja -B build .
-- Configuring done (8.1s)
$ cmake --build build -j8
[3239/3239] Linking CXX executable ...
$ ldd build/bin/openroad
        libdl / libpthread / libm / librt / libc / libutil (host glibc)
        libpython3.13.so.1.0 => .../deps/python/lib/libpython3.13.so.1.0
  • Every Found ... in the configure log resolves under deps/ (TCL, SWIG 4.3.0, Boost 1.89.0, absl, GTest 1.17.0, Python3 3.13.11 Development+Embed, ZLIB 1.3.1, spdlog 1.15.1, BISON 3.3.2, FLEX 2.6.4, CUDD, OpenMP 5.1, OR-Tools 9.15, Eigen3, LEMON, yaml-cpp, fmt). CMakeCache audit: nothing resolved from /usr besides bash, git and optional Doxygen.
  • Zero compile errors and zero toolchain-flag warnings across the 3239-step build.
  • Smoke: openroad -version; a utl tcl regression (logger_redirection.tcl); from openroad import Tech via -python; a gtest unit binary (TestMetrics) — all pass with TCL_LIBRARY=deps/lib/tcl9.0 PYTHONHOME=deps/python.
  • bazelisk run //:cmake re-runs are idempotent (deps/ wiped and recreated; a stamp file guards against deleting a foreign directory).
  • bazelisk build //:openroad analysis and bazelisk test //:lint_test unaffected.

Type of Change

  • New feature (build infrastructure)

  • I have verified that the local build succeeds

  • My code follows the repository's formatting guidelines

  • I have signed my commits (DCO).

oharboe added 20 commits July 4, 2026 15:00
Replace toolchains_llvm + prebuilt LLVM release binaries with
hermetic-llvm (BCR module 'llvm'): statically linked LLVM 22.1.8
binaries and a zero-sysroot cc_toolchain. No host compiler, linker,
headers or libraries are involved; the toolchain runs unmodified on
hosts without libxml2.so.2 (Ubuntu 25.10+, Arch, Fedora 41+, where
prebuilt ld.lld could not start) and on non-FHS distros such as NixOS.

clang-tidy now comes from @llvm//tools:clang-tidy.

tcl_lang needs a one-line patch: tclZipfs.c's '#include "crypt.h"'
resolved to glibc's crypt.h instead of the vendored minizip header once
glibc headers became explicit -isystem directories. Drop the override
when a fixed tcl_lang lands in BCR.

Tip of the hat to @dzbarsky for pointing us at hermetic-llvm:
bazel-contrib/toolchains_llvm#795 (comment)

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Launches bazel with an allowlisted PATH (no host compilers, linkers or
interpreters), BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, and only the
workspace bazelrc, so builds fail instead of silently reaching for host
tools the build should provide. Proxy and cache environment variables
pass through; the bazelisk-managed bazel binary is resolved portably.

First finding: rules_python's legacy py_binary stub needs a host
python3 ('#!/usr/bin/env python3') before the hermetic interpreter
takes over. bootstrap_impl=script uses a shell stub instead.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
- type -P instead of command -v: builtins (pwd, test, true, printf)
  returned bare names and produced self-referential toolbox symlinks.
- Detect wrappers by bazelisk name or shebang instead of requiring ELF,
  so native macOS (Mach-O) bazel binaries are symlinked directly.
- Preserve SSH_AUTH_SOCK, SSL_CERT_FILE/DIR and TMPDIR alongside proxy
  and cache variables.
- Drop exec so the EXIT trap runs and the toolbox directory is not
  leaked on every invocation.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
std::filesystem (src/tcl_readline_setup.cc) requires macOS 10.15. The
previous 10.13 floor was never enforced: toolchains_llvm ignored
--macos_minimum_os and compiled against the host SDK default.
hermetic-llvm honors it, failing the Mac build with
"'path' is unavailable: introduced in macOS 10.15".

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
With bootstrap_impl=script, a py_binary (tclint) invoked via realpath
cannot locate its runfiles after the lint scripts cd to the workspace.
Export the runfiles root so nested tools resolve it from the
environment. No-op under 'bazel test', where the runner sets it.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
hermetic-llvm ships a deliberately minimal macOS SDK subset; frameworks
beyond its six defaults are opt-in via the osx.frameworks extension
tag. Qt's bootstrap failed on the first missing one
(ApplicationServices/ApplicationServices.h). List the desktop
frameworks qtbase links against.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
ATS (inside the ApplicationServices umbrella) ships .tbd symlinks into
the FontServices private framework. Without it in the subset they
dangle, and bazel rejects the whole sysroot as an action input
('The file type ... is not supported'). Zero dangling symlinks remain
in the extracted SDK after this.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
hermetic-llvm's SDK extraction excludes usr/include/cups unless
PrintCore is listed explicitly; naming only its ApplicationServices
umbrella left PDEPluginInterface.h's '#import <cups/ppd.h>' dangling
when Qt compiles Objective-C++.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
AppKit's NSColor.h includes CoreImage/CIColor.h unconditionally.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Suggestions from review:
- tools/bazel is bazelisk's wrapper convention, so the pruned
  environment is applied automatically to every bazelisk invocation
  (Go bazelisk; the npm package does not implement the convention).
  $BAZEL_REAL is used when bazelisk provides it; direct invocation and
  BAZELISK_SKIP_WRAPPER=1 remain available.
- BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1 moves to .bazelrc so it applies
  to unwrapped invocations too.
- --host_action_env=PATH complements --action_env for exec-config
  actions.

DISPLAY/WAYLAND_DISPLAY/XAUTHORITY pass through for 'bazel run' of GUI
targets.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
AppKit reaches all three on macOS: NSImageView.h includes
Symbols/NSSymbolEffect.h unconditionally, the NSText headers take the
non-UIKit __has_include branch into the UIFoundation private framework,
and NSSharingService pulls CloudKit.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Jenkins surfaced three problems with the first wrapper cut:

- --action_env=PATH broke test actions: the harness uses tee/file and
  standalone_python tests invoke host python3 (bazel's own
  test-setup.sh also uses file). Actions now keep bazel's strict
  default PATH; per-action pruning returns when the harness stops
  using host tools (see the PR cleanup list).
- An --action_env PATH naming a per-run mktemp directory would also
  fragment action cache keys across runs and machines.
- The npm bazelisk implements the tools/bazel wrapper convention too,
  so the wrapper's own 'bazelisk --version' probe re-entered the
  wrapper; probe with BAZELISK_SKIP_WRAPPER=1 and pass flags-only
  invocations (e.g. --version) through verbatim.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
boost::icl's exclusive_less_than comparator violates strict weak
ordering (boostorg/icl#51). libstdc++ tolerates it; libc++'s std::map
does not: interval_set::operator-= subtracts only the first overlapping
interval and leaves phantom coverage (boostorg/icl#55). With the
hermetic toolchain's libc++, pad placed IO filler cells on top of pads
(seven src/pad tests) and grt's overlapping_edges golden diverged.

Carry upstream boostorg/icl@7de3f55655 ('Refactored interval lookup
operations', The-OpenROAD-Project#54, merged after boost 1.90), trimmed to include/, as a
single_version_override patch. Verified: an isolated interval_set
subtraction repro is wrong with libc++ headers and correct with the
patch at -O0 through -O3; all 61 //src/pad tests and
//src/grt/test:overlapping_edges pass. Drop the patch when a boost.icl
release containing the fix lands in BCR.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
hermetic-llvm injects Linux kernel and glibc headers as the first
-isystem entries, outranking -isystem/-I directories of libraries that
deliberately shadow libc headers. BCR sed (gnulib's stdio.h
replacement) fails to compile (SETLOCALE_NULL_MAX, _GL_ATTRIBUTE_*
undeclared) — breaking test/orfs/gcd targets on Jenkins — and tcl's
vendored minizip crypt.h loses to glibc's crypt.h. Host sysroots
provide libc via the default search path, searched after all user
includes; carry a patch switching the two cc_args to -idirafter to
restore that precedence. Verified: unpatched BCR sed 4.9 builds;
//:openroad builds; 327 pad/grt/odb tests pass. Propose upstream after
this stabilizes; drop the patch when released.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Review feedback: bazelisk auto-invoking tools/bazel forced the pruned
environment on everyone — a developer whose bazel is not
bazelisk-managed got a silent exit, and repository rules of
dependencies legitimately probe for host tools the wrapper prunes
(GNU Make's configure looks for ld in bazel-orfs's gnumake rule,
failing the Jenkins orfs targets). etc/bazel is invoked explicitly;
the toolchain-autodetection guard stays in .bazelrc for everyone.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
libc++ marks floating-point std::to_chars (used by std::format in
OpenSTA) as introduced in macOS 13.3. The SDK's libc++ headers enforce
availability annotations; earlier runs compiled against header sets
with annotations disabled, so the 10.15 floor only appeared
sufficient.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Qt's Cocoa QPA plugin (qnsview.cpp) includes MetalKit/MetalKit.h.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
MetalKit's MTKModel.h includes ModelIO/ModelIO.h; the MetalKit+ModelIO
header closure resolves entirely within the subset now (verified by
scanning their framework imports against the extracted tree).

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
The wrapper built its pruned-PATH toolbox in a fresh mktemp directory on
every invocation and passed it as --repo_env=PATH. That value is part of
bazel's analysis cache key, so every run invalidated loading/analysis and
re-ran repository rules, recompiling work that was already done.

The toolbox is now a fixed per-checkout directory, tmp/bazel-toolbox under
the workspace root (gitignored via tmp/): the --repo_env value is
byte-identical across runs, per-user and per-checkout with no /tmp
cross-user collisions, and not on a RAM-backed tmpfs. Entries are
refreshed with ln -sfn (atomic replace) and entries no longer in the
allowlist are pruned, so the allowlist remains the contract. The EXIT
cleanup trap is gone, which also lets the wrapper exec bazel directly.

Verified: second 'etc/bazel build //src/utl' is 1 internal action / 0.8s
with no re-analysis, invoked from a subdirectory to confirm the path does
not depend on the caller's working directory.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
…e builds

Materialize every dependency of the CMake build into <workspace>/deps
from the pinned bazel module graph, as an alternative to
etc/DependencyInstaller.sh:

  bazelisk run //:cmake
  cmake -DCMAKE_TOOLCHAIN_FILE=deps/toolchain.cmake -B build .
  cmake --build build -j$(nproc)

deps/ contains the merged header tree, every static archive, the build
tools (swig, bison, flex), the Tcl 9 script library, a hermetic CPython
(headers, libpython, stdlib), CMake package config shims, and the
hermetic clang/libc++ toolchain behind generated bin/cc, bin/c++
wrappers. Host requirements shrink to bazelisk, cmake, ninja or make,
git and bash: no sudo, no compiler packages, nothing installed outside
the workspace. Linux x86_64; the GUI is not part of the prefix (host
Qt5 is a libstdc++ build and cannot link against libc++ archives).

Compiler and linker flags are extracted from the resolved C++ toolchain
at analysis time (cc_common.get_memory_inefficient_command_line), so
the wrappers cannot drift from what bazel itself uses when the llvm
module is bumped. Archives are linked through one interface target
carrying a --start-group pool, making archive order irrelevant; the
seven or-tools alwayslink archives are wrapped in --whole-archive.
Dependency versions are parsed out of MODULE.bazel into a generated
deps-versions.cmake consumed by the shims.

The existing DependencyInstaller.sh/Build.sh flow is unaffected. The
only change to existing CMake files is gating src/gui's
find_package(OpenGL REQUIRED) behind Qt5_FOUND AND BUILD_GUI: OpenGL::GL
is only referenced inside that branch, and requiring OpenGL on GUI-less
hosts made every non-GUI build demand OpenGL development headers.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe oharboe requested review from a team as code owners July 6, 2026 06:40
@oharboe oharboe requested review from gadfort and maliberty July 6, 2026 06:40

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to materialize OpenROAD's third-party dependencies into a local deps/ folder, enabling developers to build OpenROAD using plain CMake on Linux x86_64 without running DependencyInstaller.sh or installing host compilers. This is achieved via a new Bazel rule (cmake_deps_bundle) that packages headers, static archives, the hermetic clang toolchain, and config shims into a relocatable prefix, alongside a wrapper script (etc/bazel) to run Bazel in a pruned environment. The review feedback focuses on improving robustness, including safeguarding the destination path in materialize.sh when run outside of Bazel, avoiding fragile Bzlmod canonical repository names (like googletest+) by copying archives to stable paths, enhancing shell-quoting robustness for toolchain flags containing literal $ characters, and adopting idiomatic with open(...) blocks in Python scripts to prevent unclosed file descriptors.

Comment thread cmake-deps/materialize.sh Outdated
Comment thread cmake-deps/shims/GTestConfig.cmake Outdated
Comment thread cmake-deps/defs.bzl
Comment thread cmake-deps/defs.bzl Outdated
Comment thread cmake-deps/assemble_bundle.py Outdated
Comment thread cmake-deps/assemble_bundle.py Outdated
oharboe added 4 commits July 6, 2026 09:11
With the variable unset or empty (the script run directly rather than
via bazel run), the default destination degenerated to /deps. The stamp
check would still have refused to delete anything, but fail early with
a clear message instead of relying on that backstop.

Review feedback from gemini-code-assist on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
GTestConfig.cmake embedded the bzlmod canonical repository name
(lib/pool/googletest+/...), which is not stable across bazel versions.
A new stable_lib_modules attribute copies a module's archives to
lib/<basename>; the shim now points at lib/libgtest.a and
lib/libgtest_main.a. The generated deps-pool.cmake is unaffected: it is
produced from the actual archive paths at build time.

Review feedback from gemini-code-assist on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
_shell_quote left $ unescaped inside double quotes, so a toolchain flag
or define containing a literal $ would be expanded by bash when the
wrapper runs. Escape all $ and restore only the ${R} placeholder.

Review feedback from gemini-code-assist on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Review feedback from gemini-code-assist on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Load cc_common and CcInfo from @rules_cc//cc/common (native symbols are
no longer global per buildifier v8), let buildifier reorder the rule
attribute dict and BUILD attributes, and annotate the two intentional
/external/ path strings in the bison/flex env rewrite the same way
bazel/bison.bzl does.

Fixes the Buildifier lint and Buildifier format CI failures on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe

oharboe commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

@maliberty @hzeller What do you think about this idea?

Get CMake users onto bazel dependencies without forcing them to drink the Bazel cool-aid, just address the DependencyInstaller.sh hell + move onto a single source of truth for dependencies.

Mostly, per Matt, OpenROAD developers sit in their favorite IDE, fix+compile+debug, repeat..

The Bazel IDE experience isn't as good always. As an example, the IDE, intellisense and build is fighting over the same bazel-* folders, single threading them.

@oharboe

oharboe commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to materialize OpenROAD's third-party dependencies and a hermetic LLVM toolchain into a local deps/ folder, enabling plain CMake builds without requiring host compilers or global dependency installers. It replaces toolchains_llvm with hermetic-llvm (llvm), updates macOS requirements, and adds a pruned wrapper script (etc/bazel) to run Bazel hermetically. Feedback on these changes suggests making the MODULE.bazel version parser in assemble_bundle.py more robust against reformatting, avoiding global injection of preprocessor defines via compiler wrappers to prevent macro namespace pollution, and adding a fallback in etc/bazel to handle non-bazelisk wrapper scripts gracefully instead of failing unconditionally.

Comment thread cmake-deps/assemble_bundle.py Outdated
Comment thread cmake-deps/defs.bzl
Comment thread etc/bazel
The MODULE.bazel version parser assumed single-line bazel_dep calls
with name before version; buildifier reformatting or argument
reordering would silently drop versions. Match each call block as a
whole and search name/version within it. Output is byte-identical for
the current MODULE.bazel.

Review feedback from gemini-code-assist on The-OpenROAD-Project#10822.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
oharboe added a commit to oharboe/OpenROAD that referenced this pull request Jul 6, 2026
Review discussion on The-OpenROAD-Project#10822: per-shim INTERFACE_COMPILE_DEFINITIONS
cannot reach dependencies consumed through plain cache variables (Tcl,
zlib), and a define set differing between a header's TU and its archive
(_FILE_OFFSET_BITS=64) is an ABI break. Global injection is bazel's
semantics for these defines; record that reasoning at the collection
site.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Review discussion on The-OpenROAD-Project#10822: per-shim INTERFACE_COMPILE_DEFINITIONS
cannot reach dependencies consumed through plain cache variables (Tcl,
zlib), and a define set differing between a header's TU and its archive
(_FILE_OFFSET_BITS=64) is an ABI break. Global injection is bazel's
semantics for these defines; record that reasoning at the collection
site.

Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe oharboe force-pushed the bazel-cmake-deps branch from 0e84375 to 0d2cade Compare July 6, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant