Derive runtime search dirs from foreign cc deps#1536
Draft
jsun-splunk wants to merge 5 commits into
Draft
Conversation
b9adfbf to
641ee6a
Compare
The transitive matrix dynamic_deps cases exercise cc_shared_library providers produced by rules_cc. Looking up those providers through bazel_features.globals.CcSharedLibraryInfo can use the wrong provider key for that path and fail analysis before the matrix tests run. Load CcSharedLibraryInfo through the rules_cc compatibility proxy so foreign_cc.dynamic_deps consumes the same provider identity as rules_cc.
The pkg-config toolchain setup emits BUILD files for the glib_dev, glib_src, and gettext_runtime repositories. These generated BUILD files instantiate cc_import directly, but commit fd05b5a added the Bazel 8 CI config with --incompatible_disable_autoloads_in_main_repo, so cc_import is no longer implicitly available when those repositories are analyzed. Load cc_import from @rules_cc//cc:defs.bzl in each generated BUILD file so the repositories are self-contained under the Bazel 8 config.
Add an integration matrix for testing native cc_library and foreign_cc producer combinations. The matrix covers the main ways a consumer can receive a transitive native or foreign dependency: - static: the app consumes a static libarchive producer through deps. Libarchive may be native cc_library or foreign_cc, and zlib may be native, foreign_cc, wrapped shared, or foreign shared. - direct: the app consumes a foreign_cc-built shared libarchive producer directly through deps. This covers foreign shared libarchive variants across the supported zlib producer shapes. - dynamic_deps: the app consumes a native cc_shared_library libarchive producer through dynamic_deps. - native wrapper: the app consumes native shared libarchive through a cc_library wrapper in deps. Exercise these producer shapes from both cc_binary and CMake consumers so the same graph is checked from native and foreign build paths. Add linkage tests that inspect the built app and, when libarchive is shared, the built libarchive library, then verify their recorded dynamic dependencies match the expected static/shared link shape. Add provider parity tests that compare the CcInfo exposed by matching native and foreign producer shapes. Include libarchive example targets to exercise a larger transitive graph through zlib and the native/foreign combinations used by the matrix.
05161ba to
2ecebfa
Compare
Foreign-built binaries and shared libraries can depend on shared libraries from `deps` or `dynamic_deps`. Some upstream builds also install companion shared libraries under `out_bin_dir` and `out_lib_dir` without runtime search paths between those outputs. Add opt-in derivation so foreign_cc outputs can resolve dependency libraries and, when requested, companion libraries from the same foreign build without loader environment variables such as `LD_LIBRARY_PATH`.
Public attrs:
- `runtime_library_search_directories`
- `include_self_runtime_library_search_directories`
- `runtime_library_search_mode`
- `additional_dynamic_runtime_library_search_origins`
- `additional_executable_runtime_library_search_origins`
`runtime_library_search_directories` accepts:
- `auto`: follow
`@rules_foreign_cc//foreign_cc/settings:runtime_library_search_directories`
- `enabled`: force runtime search directory derivation on for this target
- `disabled`: force runtime search directory derivation off for this target
The global build setting defaults to `disabled`, so the default behavior remains disabled unless a target opts in or the build setting is set to `enabled`.
```python
configure_make(
name = "python",
out_binaries = ["python3.10"],
out_shared_libs = ["libpython3.10.so"],
out_data_dirs = ["lib"],
runtime_library_search_directories = "enabled",
additional_dynamic_runtime_library_search_origins = [
"lib/python3.10/lib-dynload",
],
)
```
```python
configure_make(
name = "openssl",
out_shared_libs = ["libssl.so", "libcrypto.so"],
runtime_library_search_directories = "enabled",
include_self_runtime_library_search_directories = True,
)
```
```python
make(
name = "runtime_app",
out_binaries = ["runtime_app"],
runtime_library_search_directories = "auto",
runtime_library_search_mode = "executable",
executable_ldflags_vars = ["LDFLAGS"],
deps = [":runtime_lib"],
)
```
```python
cmake(
name = "runtime_lib",
out_shared_libs = ["libruntime.so"],
runtime_library_search_directories = "enabled",
runtime_library_search_mode = "shared",
deps = [":leaf_lib"],
)
```
Default origins are derived from declared output `File.short_path` values. Shared-library link actions use the directories of declared shared-library outputs. Executable link actions use the directories of declared binary outputs. Additional origins are install-tree-relative paths joined under the rule install root.
The implementation passes a `runtime_search_context` from the common framework into linker flag derivation. It uses declared output files for default origins, the framework install root for additional origins, and `LibraryToLink` short paths for dependency shared libraries. It also adds `_solib` sibling paths for Bazel-created solib symlinks and skips runtime search derivation for Windows C++ toolchains.
Supported exported rules are `cmake`, `make`, and `configure_make`. `ninja` and `meson` inherit the attrs as best-effort support. `boost_build` does not expose them because its `b2` interface does not map cleanly onto the common linker flag plumbing used here.
Unit tests cover mode filtering, default origins, additional origins, self-output search paths, `_solib` paths, deduplication, and `auto` resolution through the global build setting. Example integration tests build `app -> libmiddle.so -> libleaf.so` chains with `cmake`, `make`, and `configure_make`.
2ecebfa to
35f722c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Foreign-built binaries and shared libraries can depend on shared libraries from
depsordynamic_deps. Some upstream builds also install companion shared libraries underout_bin_dirandout_lib_dirwithout runtime search paths between those outputs. Add opt-in derivation so foreign_cc outputs can resolve dependency libraries and, when requested, companion libraries from the same foreign build without loader environment variables such asLD_LIBRARY_PATH.Public attrs:
runtime_library_search_directoriesinclude_self_runtime_library_search_directoriesruntime_library_search_modeadditional_dynamic_runtime_library_search_originsadditional_executable_runtime_library_search_originsruntime_library_search_directoriesaccepts:auto: follow@rules_foreign_cc//foreign_cc/settings:runtime_library_search_directoriesenabled: force runtime search directory derivation on for this targetdisabled: force runtime search directory derivation off for this targetThe global build setting defaults to
disabled, so the default behavior remains disabled unless a target opts in or the build setting is enabled.Default origins are derived from declared output
File.short_pathvalues. Shared-library link actions use the directories of declared shared-library outputs. Executable link actions use the directories of declared binary outputs. Additional origins are install-tree-relative paths joined under the rule install root.The implementation passes a
runtime_search_contextfrom the common framework into linker flag derivation. It uses declared output files for default origins, the framework install root for additional origins, andLibraryToLinkshort paths for dependency shared libraries. It also adds_solibsibling paths for Bazel-created solib symlinks and skips runtime search derivation for Windows C++ toolchains.Supported exported rules are
cmake,make, andconfigure_make.ninjaandmesoninherit the attrs as best-effort support.boost_builddoes not expose them because itsb2interface does not map cleanly onto the common linker flag plumbing used here.Unit tests cover mode filtering, default origins, additional origins, self-output search paths,
_solibpaths, deduplication, andautoresolution through the global build setting. Example integration tests buildapp -> libmiddle.so -> libleaf.sochains withcmake,make, andconfigure_make.This PR depends on #1538