Skip to content

Add pygamma_n back to analysis3#428

Open
charles-turner-bot wants to merge 7 commits into
ACCESS-NRI:mainfrom
charles-turner-bot:ctbot/issue-423-add-pygamma-n
Open

Add pygamma_n back to analysis3#428
charles-turner-bot wants to merge 7 commits into
ACCESS-NRI:mainfrom
charles-turner-bot:ctbot/issue-423-add-pygamma-n

Conversation

@charles-turner-bot

Copy link
Copy Markdown

Summary

  • add pygamma_n back to the analysis3 pixi environment
  • pin it to the conda-forge/label/broken 1.0.0 build, which remains compatible with the current netcdf-fortran stack
  • carry the broken label through the regenerated lock/export files and the recreate-env-yml.sh helper

Notes

  • Solving against the current non-broken pygamma_n releases failed because they require a newer netcdf-fortran than analysis3 can currently install.
  • 1.0.0 from conda-forge/label/broken solves cleanly and matches the existing compatibility constraints for this environment.

Closes #423

@charles-turner-1

Copy link
Copy Markdown
Collaborator

This should close #423. Not 100% sure why the workflows are failing but I saw a lurking 403, ie. failed auth, so I'm not sure it's a 'real' failure (ie. failure to solve).

With that said, this changes the lock file a lot, so I'd like to, at the very least, do some more careful verification & maybe see if we can minimise the changes.

@charles-turner-bot

Copy link
Copy Markdown
Author

A quick verification note on the broken-label scope, because the lockfile churn is a bit noisy.

I checked this against the current resolved outputs, not just by reading the diff:

  • environments/analysis3/pixi.lock — inspected the resolved package URLs under the analysis3 environment
  • environments/analysis3/solved.json — inspected the exported resolved package URLs

Then I filtered those resolved URLs for /label/broken/.

Result:

  • broken-label packages in pixi.lock: 1
  • broken-label packages in solved.json: 1
  • package matched in both cases: pygamma_n only

Resolved broken-label artifact:

pygamma_n 1.0.0 py312h4cfd3ad_1
https://conda.anaconda.org/conda-forge/label/broken/linux-64/pygamma_n-1.0.0-py312h4cfd3ad_1.conda

So although this PR still has a lot of general solve churn, I did verify that we are not accidentally pulling other packages from conda-forge/label/broken. The broken-label package scope in the current solve is just pygamma_n.

I agree it would be worth automating this going forward — e.g. a workflow check that fails if any resolved package URL contains /label/broken/ unless it is explicitly allowlisted.

@charles-turner-bot

Copy link
Copy Markdown
Author

A quick summary of where this ended up and why.

What changed

The main functional change is:

  • add pygamma_n back to analysis3
  • pin it to conda-forge/label/broken 1.0.0
  • keep conda-forge/label/broken in the workspace channel list, but put it last

Relevant config shape in pixi.toml is now effectively:

channels = [
  "conda-forge",
  { channel = "accessnri", exclude-newer = "0d" },
  "nodefaults",
  "rapidsai",
  "pytorch",
  "nvidia",
  "conda-forge/label/broken",
]

pygamma_n = { version = "==1.0.0", channel = "conda-forge/label/broken" }

Why it ended up this way

I first tried the simpler route of just adding pygamma_n normally, but the current non-broken releases would not solve in this environment because they require a newer netcdf-fortran stack than analysis3 can currently accept.

I also checked whether Pixi would let us scope the broken channel only at the dependency level without listing it in [workspace].channels, but it rejects that with an unavailable-channel error. So the narrowest workable setup here is:

  • keep the broken channel present in the workspace channel list
  • put it last so normal channels are preferred first
  • explicitly pin only pygamma_n to that broken-labelled channel

About the diff noise

The big lock/export churn was not caused by pulling lots of broken-channel packages. It came from a fresh solve updating many normal packages.

I explicitly verified the broken-channel scope against the current resolved outputs:

  • pixi.lock
  • solved.json

and in both cases there is exactly one package resolved from /label/broken/:

pygamma_n 1.0.0 py312h4cfd3ad_1
https://conda.anaconda.org/conda-forge/label/broken/linux-64/pygamma_n-1.0.0-py312h4cfd3ad_1.conda

So the current solve is not accidentally drawing in other broken-labelled packages.

Rebuild/export hiccup

While regenerating the exports, pixi run rebuild-env initially kept panicking while reading zero-byte metadata in arrow-1.4.0.dist-info inside the local env. That turned out to be caused by a corrupted extracted arrow package in the rattler cache, not by the lockfile itself.

After deleting that corrupted cached package and rerunning, pixi run rebuild-env succeeded cleanly and the tracked exports were regenerated from the normal workflow.

Guardrail added for future changes

I also added a PR workflow to check broken-labelled packages more directly.

The current design is intentionally simple:

  • parse environments/analysis3/pixi.toml
  • collect packages explicitly configured with a .../label/broken channel
  • parse environments/analysis3/pixi.lock
  • collect packages actually resolved from /label/broken/
  • fail if those sets do not match

So the rule becomes:

  • if a package is resolved from a broken-labelled channel, it must be explicitly declared that way in pixi.toml
  • if a package is explicitly declared broken in pixi.toml, it must actually appear broken in the lock

That should help keep future maintenance honest without making the check more complicated than it needs to be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[PACKAGE REQUEST] Add <pygamma_n> to analysis3 environment

2 participants