Skip to content

Derive runtime search dirs from foreign cc deps#1536

Draft
jsun-splunk wants to merge 5 commits into
bazel-contrib:mainfrom
jsun-splunk:jsun-runtime-libs
Draft

Derive runtime search dirs from foreign cc deps#1536
jsun-splunk wants to merge 5 commits into
bazel-contrib:mainfrom
jsun-splunk:jsun-runtime-libs

Conversation

@jsun-splunk
Copy link
Copy Markdown
Contributor

@jsun-splunk jsun-splunk commented May 4, 2026

Foreign-built binaries and shared libraries can depend on shared libraries from deps or dynamic_deps. Some upstream builds also install companion shared libraries under out_bin_dir and out_lib_dir without runtime search paths between those outputs. Add opt-in derivation so foreign_cc outputs can resolve dependency libraries and, when requested, companion libraries from the same foreign build without loader environment variables such as LD_LIBRARY_PATH.

Public attrs:

  • runtime_library_search_directories
  • include_self_runtime_library_search_directories
  • runtime_library_search_mode
  • additional_dynamic_runtime_library_search_origins
  • additional_executable_runtime_library_search_origins

runtime_library_search_directories accepts:

  • auto: follow
    @rules_foreign_cc//foreign_cc/settings:runtime_library_search_directories
  • enabled: force runtime search directory derivation on for this target
  • disabled: force runtime search directory derivation off for this target

The global build setting defaults to disabled, so the default behavior remains disabled unless a target opts in or the build setting is enabled.

configure_make(
    name = "python",
    out_binaries = ["python3.10"],
    out_shared_libs = ["libpython3.10.so"],
    out_data_dirs = ["lib"],
    runtime_library_search_directories = "enabled",
    additional_dynamic_runtime_library_search_origins = [
        "lib/python3.10/lib-dynload",
    ],
)
configure_make(
    name = "openssl",
    out_shared_libs = ["libssl.so", "libcrypto.so"],
    runtime_library_search_directories = "enabled",
    include_self_runtime_library_search_directories = True,
)
make(
    name = "runtime_app",
    out_binaries = ["runtime_app"],
    runtime_library_search_directories = "auto",
    runtime_library_search_mode = "executable",
    executable_ldflags_vars = ["LDFLAGS"],
    deps = [":runtime_lib"],
)
cmake(
    name = "runtime_lib",
    out_shared_libs = ["libruntime.so"],
    runtime_library_search_directories = "enabled",
    runtime_library_search_mode = "shared",
    deps = [":leaf_lib"],
)

Default origins are derived from declared output File.short_path values. Shared-library link actions use the directories of declared shared-library outputs. Executable link actions use the directories of declared binary outputs. Additional origins are install-tree-relative paths joined under the rule install root.

The implementation passes a runtime_search_context from the common framework into linker flag derivation. It uses declared output files for default origins, the framework install root for additional origins, and LibraryToLink short paths for dependency shared libraries. It also adds _solib sibling paths for Bazel-created solib symlinks and skips runtime search derivation for Windows C++ toolchains.

Supported exported rules are cmake, make, and configure_make. ninja and meson inherit the attrs as best-effort support. boost_build does not expose them because its b2 interface does not map cleanly onto the common linker flag plumbing used here.

Unit tests cover mode filtering, default origins, additional origins, self-output search paths, _solib paths, deduplication, and auto resolution through the global build setting. Example integration tests build app -> libmiddle.so -> libleaf.so chains with cmake, make, and configure_make.

This PR depends on #1538

@jsun-splunk jsun-splunk force-pushed the jsun-runtime-libs branch 8 times, most recently from b9adfbf to 641ee6a Compare May 11, 2026 04:44
The transitive matrix dynamic_deps cases exercise cc_shared_library
providers produced by rules_cc. Looking up those providers through
bazel_features.globals.CcSharedLibraryInfo can use the wrong provider key
for that path and fail analysis before the matrix tests run.

Load CcSharedLibraryInfo through the rules_cc compatibility proxy so
foreign_cc.dynamic_deps consumes the same provider identity as rules_cc.
The pkg-config toolchain setup emits BUILD files for the glib_dev,
glib_src, and gettext_runtime repositories. These generated BUILD files
instantiate cc_import directly, but commit fd05b5a added the Bazel 8 CI
config with --incompatible_disable_autoloads_in_main_repo, so cc_import
is no longer implicitly available when those repositories are analyzed.

Load cc_import from @rules_cc//cc:defs.bzl in each generated BUILD file
so the repositories are self-contained under the Bazel 8 config.
Add an integration matrix for testing native cc_library and foreign_cc
producer combinations. The matrix covers the main ways a consumer can
receive a transitive native or foreign dependency:

- static: the app consumes a static libarchive producer through deps.
  Libarchive may be native cc_library or foreign_cc, and zlib may be native,
  foreign_cc, wrapped shared, or foreign shared.

- direct: the app consumes a foreign_cc-built shared libarchive producer
  directly through deps. This covers foreign shared libarchive variants across
  the supported zlib producer shapes.

- dynamic_deps: the app consumes a native cc_shared_library libarchive producer
  through dynamic_deps.

- native wrapper: the app consumes native shared libarchive through a
  cc_library wrapper in deps.

Exercise these producer shapes from both cc_binary and CMake consumers so
the same graph is checked from native and foreign build paths.

Add linkage tests that inspect the built app and, when libarchive is shared,
the built libarchive library, then verify their recorded dynamic dependencies
match the expected static/shared link shape.

Add provider parity tests that compare the CcInfo exposed by matching native
and foreign producer shapes.

Include libarchive example targets to exercise a larger transitive graph
through zlib and the native/foreign combinations used by the matrix.
@jsun-splunk jsun-splunk force-pushed the jsun-runtime-libs branch 2 times, most recently from 05161ba to 2ecebfa Compare May 18, 2026 08:11
jsun-splunk and others added 2 commits May 19, 2026 08:52
Foreign-built binaries and shared libraries can depend on shared libraries from `deps` or `dynamic_deps`. Some upstream builds also install companion shared libraries under `out_bin_dir` and `out_lib_dir` without runtime search paths between those outputs. Add opt-in derivation so foreign_cc outputs can resolve dependency libraries and, when requested, companion libraries from the same foreign build without loader environment variables such as `LD_LIBRARY_PATH`.

Public attrs:

- `runtime_library_search_directories`
- `include_self_runtime_library_search_directories`
- `runtime_library_search_mode`
- `additional_dynamic_runtime_library_search_origins`
- `additional_executable_runtime_library_search_origins`

`runtime_library_search_directories` accepts:

- `auto`: follow
  `@rules_foreign_cc//foreign_cc/settings:runtime_library_search_directories`
- `enabled`: force runtime search directory derivation on for this target
- `disabled`: force runtime search directory derivation off for this target

The global build setting defaults to `disabled`, so the default behavior remains disabled unless a target opts in or the build setting is set to `enabled`.

```python
configure_make(
    name = "python",
    out_binaries = ["python3.10"],
    out_shared_libs = ["libpython3.10.so"],
    out_data_dirs = ["lib"],
    runtime_library_search_directories = "enabled",
    additional_dynamic_runtime_library_search_origins = [
        "lib/python3.10/lib-dynload",
    ],
)
```

```python
configure_make(
    name = "openssl",
    out_shared_libs = ["libssl.so", "libcrypto.so"],
    runtime_library_search_directories = "enabled",
    include_self_runtime_library_search_directories = True,
)
```

```python
make(
    name = "runtime_app",
    out_binaries = ["runtime_app"],
    runtime_library_search_directories = "auto",
    runtime_library_search_mode = "executable",
    executable_ldflags_vars = ["LDFLAGS"],
    deps = [":runtime_lib"],
)
```

```python
cmake(
    name = "runtime_lib",
    out_shared_libs = ["libruntime.so"],
    runtime_library_search_directories = "enabled",
    runtime_library_search_mode = "shared",
    deps = [":leaf_lib"],
)
```

Default origins are derived from declared output `File.short_path` values. Shared-library link actions use the directories of declared shared-library outputs. Executable link actions use the directories of declared binary outputs. Additional origins are install-tree-relative paths joined under the rule install root.

The implementation passes a `runtime_search_context` from the common framework into linker flag derivation. It uses declared output files for default origins, the framework install root for additional origins, and `LibraryToLink` short paths for dependency shared libraries. It also adds `_solib` sibling paths for Bazel-created solib symlinks and skips runtime search derivation for Windows C++ toolchains.

Supported exported rules are `cmake`, `make`, and `configure_make`. `ninja` and `meson` inherit the attrs as best-effort support. `boost_build` does not expose them because its `b2` interface does not map cleanly onto the common linker flag plumbing used here.

Unit tests cover mode filtering, default origins, additional origins, self-output search paths, `_solib` paths, deduplication, and `auto` resolution through the global build setting. Example integration tests build `app -> libmiddle.so -> libleaf.so` chains with `cmake`, `make`, and `configure_make`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant