[adamcrews]

Upgrade the python-hcl2 version bundled with the terraform-hcl2-parser subsystem from 4.3.0 to 8.1.4.

4.3.0 rejects some valid Terraform, and one unparseable file fails dependency inference for its whole terraform_module. 8.1.4 parses all of these:

• for used as an attribute name (lark.exceptions.UnexpectedToken when creating alert rules via Grafana's Terraform Provider #21845)
• in used as a block name (lark.exceptions.UnexpectedToken When creating Snowflake-Terraform data source outpus using database params #22401)
• conditional expressions split over multiple lines

python-hcl2 8.x keeps the quotes around string values by default, so source = "./foo" comes back as "\"./foo\"". With the current hcl2_parser.py, that means no source ever looks like a local path, and Pants silently infers no module dependencies at all. The parser now passes SerializationOptions(strip_string_quotes=True). default_requirements is now python-hcl2>=8.1.4,<9, and hcl2.lock was regenerated with generate_builtin_lockfiles.py, which also adds regex, a new dependency of 8.x.

As suggested in review, older releases are deprecated for one version instead of being dropped right away. A resolve set with [terraform-hcl2-parser].install_from_resolve can still provide python-hcl2 older than 8.0. Those releases have no SerializationOptions and strip the quotes themselves, so the parser falls back to plain hcl2.loads(content) and prints a marker to stderr. When dependency inference sees the marker, it logs a deprecation warning, scheduled for removal in 2.35.0.dev0, that tells users to upgrade that resolve to 8.1.4 or newer. docs/notes/2.34.x.md covers both the upgrade and the deprecation.

Pants pinned 4.3.0 in #21361 because of parse regressions in 4.3.1+. To check for regressions, I ran the patched parser over a private codebase with ~900 .tf files. 8.1.4 inferred the same 106 module dependencies as 4.3.0 and parsed every file that belongs to a terraform_module.

Testing:

• New test test_hcl_parser_handles_keywords_and_multiline_conditionals parses the three constructs above with the bundled 8.1.4.
• test_hcl_parser_wrapper_runs passes on Python 3.10 through 3.14. It and the new test also check that 8.1.4 isn't reported as a deprecated release.
• In hcl2_parser_test.py, test_main_strips_string_quotes and test_main_falls_back_for_legacy_hcl2 use stand-in hcl2 modules to check that the parser asks 8.x to strip quotes, and that it falls back and prints the marker for releases without SerializationOptions.
• test_warn_if_legacy_hcl2 checks that the marker produces the deprecation warning.
• pants fmt lint check passes on the changed files.

This code was generated with Claude code, however I am using this on my internal systems and so far it is working well in my pipelines.

[tdyas]

Thanks for fixing!

Would it make sense to just require the newer version and thus avoid the need to check for SerializationOptions?

[adamcrews]

Thanks for fixing!

Would it make sense to just require the newer version and thus avoid the need to check for SerializationOptions?

Yeah, probably. I was a little worried about breaking anyone that had this pinned, but with other similar breaking changes it looks like the pattern is to just leave a note in the release notes/upgrade instructions. I'll rework this a bit to pin to the current version or better.

[tdyas]

Thanks for fixing!
Would it make sense to just require the newer version and thus avoid the need to check for SerializationOptions?

Yeah, probably. I was a little worried about breaking anyone that had this pinned, but with other similar breaking changes it looks like the pattern is to just leave a note in the release notes/upgrade instructions. I'll rework this a bit to pin to the current version or better.

We could also deprecate for one version, and then remove in the subsequent version. Maybe add a deprecation warning using pants.base.deprecated.warn_or_error?

[adamcrews]

We could also deprecate for one version, and then remove in the subsequent version. Maybe add a deprecation warning using pants.base.deprecated.warn_or_error?

I reworked things to use the deprecation code/messaging. I'm happy with this current state and I think this is good to go now.

[tdyas]

This can be a bytes b"" constant and then the various LEGACY_HCL2_MARKER.encode() calls can be avoided.

[tdyas]

Just the change and the PR otherwise looks good to go.

[adamcrews]

Fixed & pushed.