Skip to content

Ignore malformed lockfiles in generate-lockfiles - #23751

Merged
benjyw merged 1 commit into
pantsbuild:mainfrom
A-G0D:fix/23706-generate-lockfiles-fails-with
Nov 11, 2026
Merged

benjyw merged 1 commit into
pantsbuild:mainfrom
A-G0D:fix/23706-generate-lockfiles-fails-with

Conversation

@A-G0D

@A-G0D A-G0D commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

generate-lockfiles reads the existing lockfile to plan a uv sync and to compute the dependency diff, so a malformed lockfile failed with TOMLDecodeError instead of being overwritten. Malformed existing lockfiles are now treated as absent: the sync starts from scratch and the diff sees no old dependencies. I added a regression test that corrupts an existing uv lockfile and reruns generation with --sync, and ran pants test, pants lint, and pants check on the touched files.

Fixes #23706.

I used DeepSeek V4 to draft this patch; I reproduced the bug and ran the tests locally.

`generate-lockfiles` reads the existing lockfile to plan a uv sync and
to compute the dependency diff, so a malformed lockfile raised
`TOMLDecodeError` even though the command was about to overwrite it.
Treat an unparseable existing lockfile as absent so the lockfile is
regenerated.

Fixes pantsbuild#23706

Signed-off-by: A-G0D <143042805+A-G0D@users.noreply.github.com>
@benjyw

benjyw commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Thanks for the contribution. We've just branched for 2.34.x, so merging this pull request now will come out in 2.35.x, please move the release notes updates to docs/notes/2.35.x.md if that's appropriate.

Actually the notes can stay in 2.34.x because I will backport this into the 2.34.x branch.

@benjyw
benjyw self-requested a review November 11, 2026 01:31

@benjyw benjyw left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for the fix.

This will fix the annoying case where there is a merge conflict in the lockfile, so the conflict markers malform the TOML, and you want to resolve it by overwriting it. Today you have to first resolve the conflict, at least by accepting one or the other side of it.

@benjyw benjyw added this to the 2.34.x milestone Nov 11, 2026
@benjyw
benjyw merged commit d3ef404 into pantsbuild:main Nov 11, 2026
53 of 56 checks passed
@WorkerPants

Copy link
Copy Markdown
Member

I tried to automatically cherry-pick this change back to each relevant milestone, so that it is available in those older releases of Pants.

✔️ 2.34.x

Successfully opened #23760.


Thanks again for your contributions!

🤖 Beep Boop here's my run link

@sureshjoshi

Copy link
Copy Markdown
Member

Is this a good idea? This feels suss to me.

How do lockfiles get malformed? If I think I have a lockfile, and that just gets ignored and re-built, that's a HUGE WTF to me.

If it throws an error, I can just delete the lockfile and re-run.

@benjyw

benjyw commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Well, there are two layers here - we parse the TOML ourselves for an unimportant reason (to estimate the number of deps), but we can squelch that lower down. Then we would pass malformed toml to uv and it would choke on it. Which is what happens today with a pex lockfile, so I suppose we should be consistent.

This is only relevant with generate-lockfiles --sync, where you're asking to incrementally update something that isn't a valid lockfile. With this change that will become a full regeneration (which is the same as deleting the lockfile and rerunning).

I could be convinced that it's better to take the user literally and fail, but it's annoying when you have many lockfiles with merge conflicts and you want to overwrite them if they do (but not if they don't). I suppose this could be an option, like --rewrite-malformed-lockfiles, that would be a more cautious approach.

@benjyw

benjyw commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Is this a good idea? This feels suss to me.

How do lockfiles get malformed? If I think I have a lockfile, and that just gets ignored and re-built, that's a HUGE WTF to me.

If it throws an error, I can just delete the lockfile and re-run.

OK, this takes away the one case where it differed from pex lockfiles: #23763

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

generate-lockfiles fails with TOMLDecodeError when the current lockfile is malformed

4 participants