Skip to content

Make uv lockfiles with local find_links portable across checkouts - #23710

Merged
benjyw merged 5 commits into
pantsbuild:mainfrom
martvald:martin/uv-portable-local-find-links
Nov 6, 2026
Merged

benjyw merged 5 commits into
pantsbuild:mainfrom
martvald:martin/uv-portable-local-find-links

Conversation

@martvald

Copy link
Copy Markdown
Contributor

Description:
With resolver = "uv", local find_links (file://%(buildroot)s/wheels) puts absolute checkout paths in the lockfile, so it only works where generated and misses cache elsewhere. path_mappings isn't supported for uv.

Issue is reproduced in martvald/pants-example-python:

  • commit 1: local wheel with pex
  • commit 2: uv with absolute path in lockfile

Three things leak the checkout path or machine state, one fix per commit:

  1. Absolute find-links paths in uv.lock. uv writes find-links paths as given. Find-links under the buildroot are now passed to uv as relative paths, with their files in the sandbox. URLs and paths outside the buildroot are unchanged.

    # before
    source = { registry = "/home/me/repo/wheels" }
    wheels = [{ path = "/home/me/repo/wheels/my_wheel-0.1.0-py3-none-any.whl" }]
    # after
    source = { registry = "wheels" }
    wheels = [{ path = "my_wheel-0.1.0-py3-none-any.whl" }]
  2. uv_cache.json baked into PEXes. For local wheels, uv writes <dist>.dist-info/uv_cache.json with the file's mtime:

    {"timestamp":{"secs_since_epoch":1792100808,"nanos_since_epoch":679638970},"commit":null,...}

    Pex copies it into the PEX, so the same PEX differed between CI and dev. Nothing else did. We now delete it after uv sync. Pex could arguably skip it, like INSTALLER.

  3. Resolver args on PEXes with no requirements. pytest_runner.pex and local_dists.pex resolve nothing but still got --find-links=file:///<checkout>/.... They now return early, like the resolve-subsetting branch.

On our repo it solved all of our 89 tests missing cache with uv, all tests using repo hosted .whl files.

goal Pex, 2.33.0 uv, 2.33.0 pex, this PR uv, this PR
lint 190/216 187/217 197/216 212/217
test 246/246 157/246 246/246 246/246
package 773/786 774/787 774/786 778/787

LLM notice:

This PR was written primarily by Claude Code. I directed the testing and reviewed the result.

@jsirois

jsirois commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

@martvald a head's up would have been nice. I'm fixing this in Pex and pex.rc presently and should have releases out in the next few days. That said, see: https://docs.astral.sh/uv/reference/environment/#uv_no_installer_metadata - which works.

@martvald

Copy link
Copy Markdown
Contributor Author

@martvald a head's up would have been nice. I'm fixing this in Pex and pex.rc presently and should have releases out in the next few days. That said, see: https://docs.astral.sh/uv/reference/environment/#uv_no_installer_metadata - which works.

Sorry for not reaching out first @jsirois . Great news, then I'll drop commit 2 since the proper fix is
coming in Pex. The other two are still needed on the Pants side: Pants passes find_links to uv
as absolute paths, and also passes them to PEXes with no requirements.

@jsirois

jsirois commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

@martvald I wouldn't rely on the Pex fix if I were you. Pex has to try to account for these files since it can't control the --venv-repository values its handed. That handling though could be enough today and not enough tomorrow (new metadata files introduced or new names for existing files). The verbage around https://docs.astral.sh/uv/reference/environment/#uv_no_installer_metadata and the PRs introducing it are all about stability / removing uv bookeeping from installs. That's exactly what you want and you can control it here since you create the venv in the 1st place and know its uv and know setting UV_NO_INSTALLER_METADATA=1 has the effect you want at a higher level.

@jsirois

jsirois commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

@martvald the Pex fix is out in https://github.com/pex-tool/pex/releases/tag/v2.104.1.

That said, I'll reiterate, don't rely on it. Set the env var instead.

@martvald
martvald force-pushed the martin/uv-portable-local-find-links branch from 418dc17 to 7ce2b6c Compare October 21, 2026 04:59
@martvald

Copy link
Copy Markdown
Contributor Author

@jsirois I really appreciate the effort you put into this. I have updated commit 2 to set UV_NO_INSTALLER_METADATA=1.

@martvald
martvald marked this pull request as ready for review October 28, 2026 06:03
@martvald

Copy link
Copy Markdown
Contributor Author

Ready for review. Addressed @jsirois feedback, now sets UV_NO_INSTALLER_METADATA=1. Could a maintainer approve the CI run?

@martvald
martvald force-pushed the martin/uv-portable-local-find-links branch from 7ce2b6c to 6d22aa6 Compare November 1, 2026 21:23
@tdyas
tdyas requested a review from benjyw November 2, 2026 05:03
uv records find-links paths verbatim in uv.lock. With the common
`find_links = ["file://%(buildroot)s/wheels"]`, the lockfile therefore
embedded the absolute checkout path: it only worked at the path it was
generated from, and every process that consumed it got checkout-specific
cache keys.

Rewrite find-links that point at a directory under the buildroot to
buildroot-relative paths, and add that directory's files to the sandbox
of both `uv lock` and `uv sync`. Other find-links (URLs, paths outside
the buildroot) are passed through unchanged. This gives the uv resolver
the portability that `[python-repos].path_mappings` provides for Pex.
uv writes bookkeeping files into installed dist-infos, e.g. uv_cache.json
with a local wheel's mtime. Pex copies them into PEXes built from the
venv, so those PEXes differed between checkouts and machines.

Pants creates these venvs, so it disables the bookkeeping at the source.
PEXes that resolve nothing, like pytest_runner.pex and local_dists.pex,
fell through to the pip-resolve branch and still received the
`[python-repos]` args. With `find_links = ["file://%(buildroot)s/..."]`
that put the absolute checkout path in their argv, so their cache keys
differed per checkout.

Return early when there are no requirements, as the resolve-subsetting
branch above already does.
- Check for the file:// prefix explicitly instead of stripping it blindly.
- normpath find-links paths so `..` can't escape the buildroot check.
- Return the sandbox PathGlobs directly instead of raw dirs.
- Inline the hardcoded UV_NO_INSTALLER_METADATA env var.
@martvald
martvald force-pushed the martin/uv-portable-local-find-links branch from 6d22aa6 to 2ef24ee Compare November 5, 2026 03:36
@martvald

martvald commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

@benjyw Thank you for the review. I have addressed all you comments in a new commit and rebased on origin/main.

@martvald
martvald requested a review from benjyw November 5, 2026 03:40
@benjyw
benjyw merged commit 3e6008a into pantsbuild:main Nov 6, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants