test(graphics): one coverage matrix, run per machine, with a third outcome by Sunrisepeak · Pull Request #543 · openxlings/xim-pkgindex

Sunrisepeak · 2026-08-07T07:29:54Z

Verification of this stack was three scripts covering one slice each --
verify-host-link.sh (NVIDIA only), selfcontained-check.sh (empty host), and
whatever got typed that day. Each had its own setup, none knew about the others,
and the union was never reported. So "the ecosystem works" rested on one machine
with one GPU, and every other cell was untested in a way that produced no output.

verify-stack.sh creates a subos, installs graphics, and walks the matrix:
software rendering, NVIDIA proprietary (delegated to the provenance verifier),
radeonsi / iris / nouveau, WSL2 d3d12, Vulkan, X11, Wayland, a real GUI
application, and empty-host self-containment.

THE POINT IS THE THIRD OUTCOME. A cell that could not be exercised here is
printed, counted, and listed again in the summary WITH ITS REASON. Skips do not
fail the run -- treating "I have no AMD GPU" as a failure would make the script
useless to everyone -- but they are never silent, because "we lack that hardware"
and "it works" must not look alike. The summary's skip list is the recruitment
list: no one machine has NVIDIA, AMD, Intel and WSL2 at once, so coverage is the
union of runs by different people, and --json exists so those runs can be
aggregated.

Writing it immediately caught two false passes in itself, both the exact shape it
exists to prevent:

nouveau reported PASS on this host. MESA_LOADER_DRIVER_OVERRIDE=nouveau
rendered fine -- on llvmpipe -- and the cell only checked RESULT=ok. A hardware
cell now asserts the renderer is not the software fallback, and nouveau is
correctly reported as not-exercisable while the proprietary nvidia.ko owns the
GPU.
xlings install graphics printed "0 package(s)" as a pass. That is what a
re-run looks like; it now says "already satisfied" instead of a count that
reads as coverage -- the same trap B line: interpose every glvnd entry point, with provenance in the proof #532 hit in CI.

First run on an RTX 4080 / driver 550.144.03: pass 13, fail 0, not-exercised 6
(amd, intel, WSL2, Vulkan, Wayland, and the empty-host check which is currently
INCONCLUSIVE by its own control run).

…tcome Verification of this stack was three scripts covering one slice each -- verify-host-link.sh (NVIDIA only), selfcontained-check.sh (empty host), and whatever got typed that day. Each had its own setup, none knew about the others, and the union was never reported. So "the ecosystem works" rested on one machine with one GPU, and every other cell was untested in a way that produced no output. verify-stack.sh creates a subos, installs `graphics`, and walks the matrix: software rendering, NVIDIA proprietary (delegated to the provenance verifier), radeonsi / iris / nouveau, WSL2 d3d12, Vulkan, X11, Wayland, a real GUI application, and empty-host self-containment. THE POINT IS THE THIRD OUTCOME. A cell that could not be exercised here is printed, counted, and listed again in the summary WITH ITS REASON. Skips do not fail the run -- treating "I have no AMD GPU" as a failure would make the script useless to everyone -- but they are never silent, because "we lack that hardware" and "it works" must not look alike. The summary's skip list is the recruitment list: no one machine has NVIDIA, AMD, Intel and WSL2 at once, so coverage is the union of runs by different people, and --json exists so those runs can be aggregated. Writing it immediately caught two false passes in itself, both the exact shape it exists to prevent: * `nouveau` reported PASS on this host. MESA_LOADER_DRIVER_OVERRIDE=nouveau rendered fine -- on llvmpipe -- and the cell only checked RESULT=ok. A hardware cell now asserts the renderer is not the software fallback, and nouveau is correctly reported as not-exercisable while the proprietary nvidia.ko owns the GPU. * `xlings install graphics` printed "0 package(s)" as a pass. That is what a re-run looks like; it now says "already satisfied" instead of a count that reads as coverage -- the same trap #532 hit in CI. First run on an RTX 4080 / driver 550.144.03: pass 13, fail 0, not-exercised 6 (amd, intel, WSL2, Vulkan, Wayland, and the empty-host check which is currently INCONCLUSIVE by its own control run).

…d to end verify-stack.sh reported it as a non-fatal unresolved dlopen under the real-GUI cell, which is the only place it could have shown up: libXinerama is on no DT_NEEDED path -- mesa does not need it, nothing in the rendering closure does -- and a toolkit dlopens it to ask where the monitors are. It appears in no dependency graph derived from ELF metadata, and the surfaceless probe that was this stack's acceptance criterion could never miss it. godot printed libXinerama.so.1: cannot open shared object file and started anyway, falling back to single-screen geometry. A non-fatal dlopen failure survives every test that only asks "did it run". Built with build-in-subos.sh (leak check: no host references), published to a new xlings-res/libXinerama, recipe added, pulled into `graphics`. Verified by reinstalling and running godot: the line is gone, and it still reports OpenGL API 3.3.0 NVIDIA 550.144.03 on the RTX 4080. Also fixes build-in-subos.sh, which is why the build failed the first time. It spliced only `<dep>/lib/pkgconfig` into PKG_CONFIG_LIBDIR. A PROTOCOL-ONLY package installs no library and puts its .pc in `share/pkgconfig` -- xorgproto ships 40 of them there, xcb-proto likewise -- so every protocol package was invisible to pkg-config while its HEADERS were still spliced in via -I. That combination is why the gap lasted: mesa builds fine because it includes the headers and never asks pkg-config for a protocol module; libXinerama does ask, and died on `XINERAMA_CFLAGS ... no such package` for a package sitting right there with its headers already on the command line. GLOBAL mirror only, stated in the recipe rather than papered over: `gtc` can publish a release into an existing GitCode project but cannot create the project, and xlings-res/libXinerama does not exist there yet. A CN URL pointing at a missing project fails at download time instead of falling back, which is worse than not having one.

…read The matrix reported Vulkan as an empty cell: mesa builds RADV and rewrites its ICD manifest to an absolute path in our payload, and nothing ever read it, because an ICD is a DRIVER and a driver is loaded BY a loader. No libvulkan.so.1 meant no vulkaninfo and a dead zink in a payload that ships it. vulkan-headers 1.4.313 and vulkan-loader (from tag vulkan-sdk-1.4.313.0), both built in a subos and leak-checked, published to xlings-res, pulled into `graphics`. Verified: lib/libvulkan.so.1 reaches <subos>/lib and its DT_NEEDED closure resolves with 0 unresolved under our own loader. Discovery needed no new declaration -- the loader searches $XDG_DATA_DIRS/vulkan/icd.d and mesa's config() already prepends its share directory there. VK_DRIVER_FILES would have been wrong: it is an override that suppresses system discovery. ## And a bug in build-in-subos.sh that produced a mislabeled payload The download cache was keyed on the URL's basename. Every GitHub archive URL is `.../archive/refs/tags/v<tag>.tar.gz`, so the basename carries the tag and nothing about the project -- and Vulkan-Headers and Vulkan-Loader are both released as v1.4.313. The loader build found the headers' tarball already in $SRC, skipped the download, then configured, built, staged, LEAK-CHECKED and packaged the wrong source. Every step reported success and the artifact was published to xlings-res as `vulkan-loader` while containing Vulkan-Headers. It was caught only because the installed package had no lib/ directory. The release has been deleted and replaced with the real loader; the cache is now keyed on NAME-VERSION. Two smaller things the same build surfaced: Vulkan-Loader's tags are `vulkan-sdk-<x>` and not `v<x>` (GitHub's archive endpoint answers 200 for a ref that is not the tag you meant), and xrandr.pc Requires.private xrender, which was simply missing from --deps -- a missing dependency, not a tooling fault.

…at a pass Adding a Vulkan loader immediately broke two of the matrix's own cells, both by being too weak. Recording them because each is the shape the matrix exists for. **"software rendering (llvmpipe)" started failing with** zink Vulkan 1.3 (NVIDIA GeForce RTX 4080 (NVIDIA_PROPRIETARY)) Selecting the mesa vendor is not selecting SOFTWARE. With a working libvulkan.so.1 present, mesa switched to zink -- its GL-over-Vulkan driver, a real and welcome capability, and emphatically not the CPU path this cell is named for. It now forces LIBGL_ALWAYS_SOFTWARE and asserts the renderer is one of llvmpipe/softpipe/swrast. (That zink came alive at all is the loader paying for itself: the recipe predicted "zink is dead in a payload that ships it".) **"Vulkan loader + ICDs" reported PASS with "0 ICD manifest(s)".** A loader with none of OUR ICDs in the subos is not Vulkan support -- it is a loader that finds the HOST's ICDs and succeeds. Same boundary the GL side needed interposers for, one API over, and it looks like a pass from every angle except asking whose ICD answered. Zero ICDs is now a failure with that sentence as its message. Which then named the real defect: mesa's ICD manifest lived only in its payload. The loader reads $XDG_DATA_DIRS/vulkan/icd.d and mesa puts ${subosdir}/share on that list -- so a manifest that never reaches the subos is never found. graphics.declare_vulkan_icd() places them, the same shared-directory shape the glvnd vendor JSON uses and for the same reason. Verified: radeon_icd.x86_64.json now lands in <subos>/share/vulkan/icd.d, the cell passes on OUR ICD, and the software cell is back to llvmpipe. Matrix on this host: pass 13, fail 0, not-exercised 6.

…-contained The tool the last round said to write, written. It dlopens a glvnd vendor directly and reports whether the object loaded and whether __egl_Main is there, which is the one question every other elimination had left. First run answered it, and not at the step I predicted: DLOPEN=fail ERR=libXau.so.6: cannot open shared object file The vendor never loads at all; __egl_Main is never reached. With LD_LIBRARY_PATH=<subos>/lib it loads and the entry point is present -- so the missing thing is a library, not an entry point. The chain, each link measured: libxcb.so.1 NEEDs libXau.so.6 libxcb's RUNPATH is only $ORIGIN -- its own payload directory libXau.so.6 lives in a DIFFERENT payload DT_RUNPATH is not transitive, so nothing above libxcb can help it => libXau is resolved from the HOST <subos>/lib does contain libXau.so.6. Nothing on libxcb's search path points there. The file is present and the path is not. So the stack has never been self-contained, and the gap is a SECOND-LEVEL dependency. Outside a container nobody notices, because every Linux machine has libXau in its ld.so.cache. The empty-host container has no cache, the vendor fails to load, and that surfaces as zero vendors and EGL_BAD_PARAMETER -- which is what S1 has been reporting. This is exactly what the S3 assertion was written to catch, one layer lower than expected: not a GL renderer coming from the host, but an X11 second-level dependency. I was wrong to write that S1's failure was uninformative. The control run failing too does not mean the test is broken; it means both runs failed for the same real defect. The INCONCLUSIVE gate is still worth keeping -- it turns a message that would blame the closure into an honest "could not tell" -- but this time the closure really is incomplete. Not fixed here: elfpatch writes a full closure RPATH onto executables and leaves payload libraries with $ORIGIN. Either each payload library's RPATH should cover its own closure, or <subos>/lib should be appended to it -- which is precisely what §B1 just did for the interposer, and this shows the same reasoning applies to the whole stack. That is a blast-radius decision, not a drive-by edit.

…id pass `exports.runtime.libdirs` is what a package OFFERS its dependents. Something has to CONSUME it, and that something is `elfpatch` — which no recipe in this index, or in mcpp-index, ever called. `elfpatch.closure_lib_paths()` is a public, documented libxpkg API that had zero callers in the entire ecosystem. So `libxcb.so.1` shipped as DT_NEEDED libXau.so.6, libXdmcp.so.6, libc.so.6 DT_RUNPATH $ORIGIN with libXau in a different payload. It resolved anyway — from the host's /etc/ld.so.cache, on every machine that has libxau, which is every desktop Linux. The stack looked self-contained and never was. The subos link directory does not cover this, and the reason is one sentence of ld.so's search order: if an object has DT_RUNPATH, no ancestor's DT_RPATH is consulted for its dependencies. A consumer with `<subos>/lib` in a transitive DT_RPATH cannot serve libxcb's search for libXau, because libxcb has a RUNPATH of its own. Measured on a sealed bwrap with no /usr at all: as shipped EGL_CLIENT_EXTENSIONS= , surfaceless refused 0x300c exit 1 sealed GL_RENDERER=llvmpipe (LLVM 20.1.7) PIXEL=336699 exit 0 libs/selfcontain.lua wraps the closure patch; 28 recipes call it from install(). Under-declared direct deps are completed at the same time — a closure is only as complete as the deps list it is computed from, and `runtime_deps` is direct, not transitive. glibc is patched by nothing: it is the root, and rewriting the loader's own payload is how this was nearly broken while being investigated. The verify-stack cell that has reported INCONCLUSIVE since it was written now reads `✓ empty-host self-containment — S1-S4 pass`. Both arms of that A/B had been failing for the same real defect, which is why it never accused anything. pass 14 fail 0 not-exercised-here 5

… both CI registers every changed recipe a second time under `local:`. fontconfig declared bare `expat@2.6.2`, so the moment this PR also touched expat the name had two candidates and the install died with a candidate list — a failure caused by the SHAPE of the PR, not by either recipe. 62 bare dep names remain index-wide (cairo, glib, libpng, harfbuzz...). Each is latent in exactly the same way: fine until some unrelated change happens to hit the depender and the dependee together.

…uccess Every recipe ends `os.mv(srcdir, install_dir); return true` with the move unchecked. When the extracted source directory is already gone the move does nothing, install() returns true, and xlings prints a tick over an EMPTY payload. Reproduced locally, not inferred: xim:libffi@3.4.4 installed as a dependency -- consumes the srcdir local:libffi@3.4.4 ✓ done, 1 package installed -- payload directory empty The only complaint came from the *config* hook two steps later, about pkgconfig globs — the error named the wrong subsystem because by then nothing remembered that the payload never arrived. That is what CI is currently reporting for pkgs/l/libffi.lua. The check cannot fire in CI's install test, and the comment says so: under `config --add-xpkg` every `xim.pkgindex.*` import is a no-op proxy, so that job does not exercise selfcontain.seal at all. Cell 6 of verify-stack.sh is the evidence that the seal works; a green linux-install-test is not.

…xist `config --add-xpkg` registers the changed recipe under `local:` while the published one stays under `xim:`. Two candidates for one package is a state that never exists after merge, and it breaks the run in two ways that both read as bugs in the diff: 1. AMBIGUITY. Any recipe naming a dep without a namespace now has two candidates. Published `xim:fontconfig` says `expat@2.6.2`, so a PR that merely touched expat broke fontconfig AND graphics. Whether a PR passed depended on which OTHER packages it happened to touch. 2. DOUBLE INSTALL FROM ONE EXTRACTION. `xim:libffi` arrives as another package's dependency and its hook MOVES the extracted tree into place. `local:libffi` then has no download artifact, so no extraction; the recipe's os.mv finds nothing, install() returns true, and xlings prints a tick over an EMPTY payload. The only complaint came from the config hook two steps later, about pkgconfig globs. So the recipe is written OVER the published one, in place, in the same namespace. Deleting the published copy instead was tried and is NOT equivalent: it also leaves one candidate, but removes the `xim:` NAME, so every self-qualifying dep (`xim:expat@2.6.2`, `xim:libffi@>=3.4`) stops resolving — 3 failures became 3 different failures. Overlay keeps the name. libs/ is overlaid into the same index for the reason the local-index copy already documents: a recipe imports `xim.pkgindex.*` from the index it was loaded from, and a missing libs/ turns every helper call into a truthy no-op. That blindness was described in this file and never fixed; the same job could not execute selfcontain.seal at all. Falls back to --add-xpkg when the index path is a symlink (a developer pointing their home at the checkout) so the copy can never write into the source tree. Validated against a CI-shaped home (index populated from origin/main, not the working copy): 21 packages, 0 failures, including the four that were red.

These shipped GLOBAL-only with a comment saying gtc could publish a release but not create a GitCode project. That was wrong — `gtc repo create` exists. The actual blocker was different and only shows up on an EMPTY project: tagging a release fails with `main is not exist`, because there is no branch to target. Pushing a README first is what makes the release possible. Created xlings-res/{libXinerama,vulkan-headers,vulkan-loader} on GitCode, pushed the same README the GitHub mirror carries, published the tag and uploaded the payload. Verified by DOWNLOADING each asset from the CN URL and comparing sha256 to the artifact the recipe pins — not by checking that the URL exists. All three match byte for byte.

The previous commit overlaid every changed recipe into the index. That is right for a change to a PUBLISHED package and wrong for one the PR ADDS: the package is not in the index, so asking for it by its index name makes xlings say 'xim:libXinerama' not in current index; refreshing index... and the refresh re-fetches the whole index, overwriting the file just placed there. The install then fails with `not found` — all three new packages died in 21 seconds. Overlay now requires the published copy to exist. That is exactly the case that produces the duplicate candidate, and the only case overlay can serve. A new package keeps the --add-xpkg / local: path, where it is unambiguous anyway: nothing published shares the name. This is also the namespace rule the index already follows — new package referenced bare, changed published one with xim:. Validated on the union of both failure sets — the 3 new packages that just failed and the 4 that were red before: 7 tested, 0 failures.

Every bump(xlings) PR touches pkgs/x/xlings.lua, so the lifecycle test installs xlings and then tries to remove it: [error] xlings only has one version installed (xim:xlings@2026.8.8.1); cannot remove the running binary itself. That refusal is correct and deliberate -- there is a separate `xlings self uninstall` built for exactly this -- so the test should not assert on it. It is new to bump PRs and nothing about xlings caused it. Before #543 a changed recipe was registered with `config --add-xpkg` and installed as `local:xlings`, which the running-binary guard does not match. #543 made CI overlay the recipe into the index instead, so it installs as `xim:xlings` and the guard now fires. The evidence is in the two logs: PR #541 shows `local:xlings@2026.8.7.1` and passed, PR #548 shows `xim:xlings@2026.8.8.1` and failed, with the same pinned client (2026.8.6.3) on both. Left unfixed this is red on every future release bump. Keyed on the exact diagnostic, like the config-type tolerance above it.

…eck that runs (#549) Split out of the single large PR so its CI cost matches its risk: this touches no pkgs/ file at all, so the per-package install tests skip entirely. THE CONTRACT 0 proven / 1 broken / 2 inconclusive / 3 could-not-run-here, written into .agents/tools/README.md and obeyed by eleven scripts. A caller must map 3 to "not run" and must never fold it into a pass. It exists because skip() in selfcontained-check.sh used to exit 0, and 0 is what the caller reads as "S1-S4 pass" -- so a machine without bwrap printed a tick for a check that ran nothing. Applying the contract turned up two more of the same: verify-host-link's check 4 and verify-stack's app-RPATH cell both printed a pass with no patchelf present. BUILD INPUTS, NOT JUST OUTPUTS build-in-subos.sh verified the produced payload and never what the build linked against, so a build configured entirely against /usr passed "no host references". The sealed-bwrap approach was tried first and does not work here (the subos has no POSIX userland, so clean and dirty builds fail alike); configure-log parsing instead, filtered to resolved records. It reproduces both known instances -- the host libzstd in the LLVM tree behind our mesa, and a constructed case -- and found a third: the --deps .pc rewrite patched only prefix=, leaving libdir=/usr/lib in 40 files. DECLARED DEPS vs REAL DT_NEEDED dep-closure-check.sh, run per package from posix-test.sh. D1: a soname provided by an installed package must be declared DIRECTLY, because a transitive dep does not put its libdir in the closure. D2: if the payload uses our interpreter, a soname with no provider is fatal -- our ld.so has a cache path that exists on no machine, so there is no host fallback behind it. Strictness follows the payload's own state (interpreter, or an RPATH into xpkgs), so host-integrated packages are reported and not failed. A check that fires on correct recipes gets switched off. The tooling step matters as much as the script: linux-install-test is a separate job from linux-test, so lua5.4 was absent and the check exited 3 on every run -- "not evaluated", job green. Measured after fixing it: 20 passes, 6 not-evaluated, 0 failures across a 25-package changed set. Two uninstall tolerances, each keyed on its exact diagnostic rather than on the package type alone: a type="config" package registers no xvm version (11 of the 12 in this index call xvm.add zero times), and a bump PR cannot remove the xlings it is running. The latter is new since #543 -- overlaying the recipe into the index made it install as xim:xlings rather than local:xlings, which the running-binary guard matches. Left alone it is red on every future release bump. Also stops posix-test.sh from printing hundreds of "ignored null byte in input" warnings: it read ELF magic with head -c4 inside a command substitution while walking the whole store. Co-authored-by: sunrisepeak <x.d2learn.org@gmail.com>

speak-agent and others added 11 commits August 7, 2026 15:29

Sunrisepeak merged commit c554682 into main Aug 7, 2026
11 checks passed

Sunrisepeak mentioned this pull request Aug 7, 2026

ci: a three-state contract for verification scripts, and a closure check that runs #549

Merged

Navigation Menu

Uh oh!

test(graphics): one coverage matrix, run per machine, with a third outcome - #543

test(graphics): one coverage matrix, run per machine, with a third outcome#543
Sunrisepeak merged 11 commits into
mainfrom
feat/verify-stack-matrix

Sunrisepeak commented Aug 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Sunrisepeak commented Aug 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants