Add support for -r pylock.toml

[sbidoul]

• this requires a new packaging release to have Pylock.select()
• to install vcs entries from a lock file which also has other entries with hashes, we'll need Validate VCS urls in hash-checking mode using their commit hashes #11968
• to install directory entries from a lock file which also has other entries with hashes, we'll need editable cannot be installed when requiring hashes #4995 (comment)
• paths relative to lockfile location
• tests
• docs
• and address the most important TODO's

But this works.

Closes #13334

[sbidoul]

@pypa/pip-committers is everyone on board with -r pylock.toml as UX ?

[pfmoore]

Makes sense to me. Will the decision as to whether it's a requirements file or a lock file be based on the filename, or the content? Either is a plausible choice. My preference would be to base it on the filename, as per the lockfile spec. This is technically backward incompatible, but I doubt anyone is naming their requirements files pylock.toml...

Edit: I see the current implementation works off the filename. That fits with my preference.

[stonebig]

Initial try... I can have screwed-up...:

• using a WinPython-3.14.3 (dot version)
• replacing pip package per your branch
• then pip3 download --dest .\budot\WPy64-31431b0\notebook\test01 -r .\build_history\publish_202602\requir.64-3_14_3_1slimb0.txt

It gives me two strange issues:

• it downloads cython-3.2.4-cp314-cp314-win_amd64.whl from https://pypi.org/project , but I should already have it but it can be the fact I mistaken testing requirement file instead of pylock file
• it crashes at this maybe particular case:

  Using cached zstandard-0.25.0-cp314-cp314-win_amd64.whl (516 kB)
Collecting pip (from datasette==0.65.2->-r .\build_history\publish_202602\requir.64-3_14_3_1slimb0.txt (line 147))
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    pip from https://files.pythonhosted.org/packages/de/f0/c81e05b613866b76d2d1066490adf1a3dbc4ee9d9c839961c3fc8a6997af/pip-26.0.1-py3-none-any.whl (from datasette==0.65.2->-r .\build_history\publish_202602\requir.64-3_14_3_1slimb0.txt (line 147))

then, realizing it's not the .toml, redoing with pylock file:

• pip3 download --dest .\bd314\budot\WPy64-31431b0\notebook\test01 -r .\build_history\publish_202602\pylock.64-3_14_3_1slimb0.toml

output (same issue): (it's text file output, branded as .csv )

• from requirement file:
  pip_requfile_test_output.csv
• from pylock file:
  pip_lockfile_test_output.csv

[sbidoul]

@stonebig maybe try with --no-deps. If I read the output correctly, datasette depends on pip, which is not in the lock file.

[stonebig]

@stonebig maybe try with --no-deps. If I read the output correctly, datasette depends on pip, which is not in the lock file.

Yes, I realised I should do that after... maybe the error message could suggest that obvious error ?

[sbidoul]

maybe the error message could suggest that obvious error

Maybe? I don't think I'll address that in this PR, which "simply" reads requirements from a pylock file but otherwise leaves all other pip behaviors intact.

[stonebig]

Strange test:

• removing cython-3.2.4-cp314-cp314-win_amd64.whl from my personnal directory .\packages.srcreq, only leaving cython-3.2.4-py3-none-any.whl
• using pip3 download --dest .\test05 -r .\build_history\publish_202602\pylock.64-3_14_3_1slimb0.toml --no-deps --pre --no-index --trusted-host=None --find-links=.\packages.srcreq --no-cache-dir

==> this downloads wheels from pypi .... and I get again cython-3.2.4-cp314-cp314-win_amd64.whl at arrival

can't I make a fully isolated source directory of wheels with -no-deps --pre --no-index --trusted-host=None --find-links=.\packages.srcreq --no-cache-dir ?

Besides that, result matches the wheel requested... only the source of wheels seems not-constrainable to a directory

[pfmoore]

If the lock file includes PyPI URLs as the location of the wheels, then of course it will download files from PyPI. If you want to only use local files, you'll need to modify the lockfile to reflect that.

The lockfile you linked to only uses URLs, so that's what pip will use.

[notatallshaw]

is everyone on board with -r pylock.toml as UX ?

Will this work where all -r places currently work? Such as download and wheel?

Will this work with all the options users expect to work with requirements files? Constraints? Dry run?

[sbidoul]

Will this work where all -r places currently work? Such as download and wheel?

Yes absolutely. pip {wheel,download} -r pylock.toml are useful features, I think.

Will this work with all the options users expect to work with requirements files? Constraints? Dry run?

Yes everything works because the lock file is simply another source of requirements. Constraints of course must be compatible with the requirements in the lock file which already provides strong constraints, but that works.

So far the only limitations I see are support for selecting extras and dependency groups from the lock file. So my current thinking is to do

• A MVP with pip {install,wheel,download,lock} -r pylock.toml without support for extras and dependency groups (this PR)
• After that, we can consider new pip sync command that accepts a single lock file and updates the current environment to match the lockfile exactly. That is not entirely obvious since we don't have a way to record the provenance origin of installed distributions (except when the requirement was provided as a direct URLs), so pip sync would roughly be pip install -r pylock.toml --ignore-installed --no-deps + pip uninstall {everything not listed in the lock file}. That pip sync command would accept --extra, --group options likely inspired from uv.
• I would also investigate if pylock files could serve as constraint sources. I think it would mesh well with the current pip UX, and there seems to be some demand for it on the uv tracker Accept pylock.toml as a constraint file astral-sh/uv#13031.

Two more notes:

• currently, the still experimental pip lock command cannot (even in principle) produce extras and dependency groups in the lock file with the current options set
• uv pip install -r does not accept other requirements when a pylock.toml is provided, so it could in principle handle --extra and --group options. But I feel that limitation a bit awkward for -r and we don't have such --extra and --group options in pip anyway. That might be a track to explore, though, not sure.