Repository navigation
Make uv lockfiles with local find_links portable across checkouts - #23710
Conversation
|
@martvald a head's up would have been nice. I'm fixing this in Pex and pex.rc presently and should have releases out in the next few days. That said, see: https://docs.astral.sh/uv/reference/environment/#uv_no_installer_metadata - which works. |
Sorry for not reaching out first @jsirois . Great news, then I'll drop commit 2 since the proper fix is |
|
@martvald I wouldn't rely on the Pex fix if I were you. Pex has to try to account for these files since it can't control the |
Like pex-tool/pex#3302. Spotted in pantsbuild/pants#23710
This issue was spotted over here: pantsbuild/pants#23710
|
@martvald the Pex fix is out in https://github.com/pex-tool/pex/releases/tag/v2.104.1. That said, I'll reiterate, don't rely on it. Set the env var instead. |
418dc17 to
7ce2b6c
Compare
|
@jsirois I really appreciate the effort you put into this. I have updated commit 2 to set |
|
Ready for review. Addressed @jsirois feedback, now sets |
7ce2b6c to
6d22aa6
Compare
uv records find-links paths verbatim in uv.lock. With the common `find_links = ["file://%(buildroot)s/wheels"]`, the lockfile therefore embedded the absolute checkout path: it only worked at the path it was generated from, and every process that consumed it got checkout-specific cache keys. Rewrite find-links that point at a directory under the buildroot to buildroot-relative paths, and add that directory's files to the sandbox of both `uv lock` and `uv sync`. Other find-links (URLs, paths outside the buildroot) are passed through unchanged. This gives the uv resolver the portability that `[python-repos].path_mappings` provides for Pex.
uv writes bookkeeping files into installed dist-infos, e.g. uv_cache.json with a local wheel's mtime. Pex copies them into PEXes built from the venv, so those PEXes differed between checkouts and machines. Pants creates these venvs, so it disables the bookkeeping at the source.
PEXes that resolve nothing, like pytest_runner.pex and local_dists.pex, fell through to the pip-resolve branch and still received the `[python-repos]` args. With `find_links = ["file://%(buildroot)s/..."]` that put the absolute checkout path in their argv, so their cache keys differed per checkout. Return early when there are no requirements, as the resolve-subsetting branch above already does.
- Check for the file:// prefix explicitly instead of stripping it blindly. - normpath find-links paths so `..` can't escape the buildroot check. - Return the sandbox PathGlobs directly instead of raw dirs. - Inline the hardcoded UV_NO_INSTALLER_METADATA env var.
6d22aa6 to
2ef24ee
Compare
|
@benjyw Thank you for the review. I have addressed all you comments in a new commit and rebased on origin/main. |
Description:
With
resolver = "uv", localfind_links(file://%(buildroot)s/wheels) puts absolute checkout paths in the lockfile, so it only works where generated and misses cache elsewhere.path_mappingsisn't supported for uv.Issue is reproduced in martvald/pants-example-python:
Three things leak the checkout path or machine state, one fix per commit:
Absolute find-links paths in
uv.lock. uv writes find-links paths as given. Find-links under the buildroot are now passed to uv as relative paths, with their files in the sandbox. URLs and paths outside the buildroot are unchanged.uv_cache.jsonbaked into PEXes. For local wheels, uv writes<dist>.dist-info/uv_cache.jsonwith the file's mtime:{"timestamp":{"secs_since_epoch":1792100808,"nanos_since_epoch":679638970},"commit":null,...}Pex copies it into the PEX, so the same PEX differed between CI and dev. Nothing else did. We now delete it after
uv sync. Pex could arguably skip it, likeINSTALLER.Resolver args on PEXes with no requirements.
pytest_runner.pexandlocal_dists.pexresolve nothing but still got--find-links=file:///<checkout>/.... They now return early, like the resolve-subsetting branch.On our repo it solved all of our 89 tests missing cache with uv, all tests using repo hosted
.whlfiles.LLM notice:
This PR was written primarily by Claude Code. I directed the testing and reviewed the result.