Notebook execution errors silently pass CI in three lecture repos — standardise `-n -W` on the sphinx-tojupyter build step

[mmcky]

Summary

Most lecture repos catch a broken notebook cell in CI, but three do not, because the ci.yml step that actually executes the notebooks — "Build Download Notebooks (sphinx-tojupyter)", jb build lectures ... --custom-builder=jupyter — is missing the -n -W flags in those three. Without -W on that step a CellExecutionError is a non-fatal warning; the later -W "Build HTML" step runs under execute_notebooks: "cache" and reuses the already-executed (possibly-errored) notebooks, so it never re-catches the failure. Net effect: a lecture whose code is broken passes CI green in those repos.

Audit (the execution step's flags)

Repo	-W on the sphinx-tojupyter execution step	Broken cell fails CI?
lecture-python.myst	✅ -n -W --keep-going	yes
lecture-python-programming	✅	yes
lecture-python-programming.zh-cn	✅	yes
lecture-python-programming.fa	✅	yes
lecture-python.zh-cn	❌ missing	no
lecture-intro.zh-cn	❌ missing	no
lecture-python-intro	❌ missing	no

It is not a translation-vs-source split: the programming family (source + editions) all catch it, the intro family masks on both the source and the edition, and the intermediate family catches on the source (myst) while the zh-cn edition dropped it. lecture-python-programming.fr has no custom-builder=jupyter step yet.

Evidence

Surfaced during the lecture-python.zh-cn resync wave (2026-07): a cold Build Cache run reported success while its execution-reports artifact held 19 notebook execution failures, and the resync only stayed safe because of a manual workaround — every fix was pushed as a new commit to force a cold rebuild, and the reports were read by hand. The repos that carry -W on the execution step (myst, programming) do not need that workaround; their CI goes red on a broken cell.

Fix

Add -n -W --keep-going to the "Build Download Notebooks (sphinx-tojupyter)" step in the three repos, matching the repos that already catch failures — no _config.yml change required, and the working repos prove the pattern:

- name: Build Download Notebooks (sphinx-tojupyter)
  run: |
    jb build lectures -n -W --keep-going --path-output ./ --builder=custom --custom-builder=jupyter

Repos to change: lecture-python.zh-cn, lecture-intro.zh-cn, lecture-python-intro. One caveat before flipping it on: any lecture with an intentionally-erroring cell needs a raises-exception tag first, or it will (correctly) start failing.

Secondary note — cache.yml

The weekly Build Cache job builds with the default HTML builder under execute_notebooks: "cache", which does not catch execution errors even with -W present (the HTML builder reuses the cache rather than re-executing — this is what let the 19 failures above pass). Aligning ci.yml fixes the PR gate, which is the priority; if the cache job should also gate on execution, that is a separate change (run the tojupyter execution builder in cache.yml, or set a raise-on-error config).

References

• Found during the intermediate lecture-python.zh-cn resync wave; recorded in project-translation#12.
• Sibling finding from the same wave (shared-asset sync, not CI): action-translation#117.

[mmcky]

Field note from applying this in lecture-python.zh-cn (QuantEcon/lecture-python.zh-cn#185): the one-line fix is not sufficient on its own, and the gap is present in the reference repo too.

Adding -n -W --keep-going to the sphinx-tojupyter step does not by itself make that step fail, because the step's exit code is not jb build's. The step is:

• jb build lectures -n -W --keep-going ... --builder=custom --custom-builder=jupyter
• mkdir -p _build/html/_notebooks
• cp -u _build/jupyter/*.ipynb _build/html/_notebooks

run under shell: bash -l {0}. GitHub only injects -eo pipefail for the bare shell: bash shorthand (which expands to bash --noprofile --norc -eo pipefail {0}); an explicit shell: bash -l {0} is a custom spec and gets neither. So a failing jb build is followed by mkdir and cp, and the step exits with cp's status.

--keep-going makes this worse rather than better: it forces Sphinx to emit output despite the errors, so the .ipynb files exist, cp succeeds, and the step reports green. Verified empirically: bash -l -c 'false; echo hi' exits 0; with set -eo pipefail it exits 1.

This affects lecture-python.myst and lecture-python-programming too — the repos this issue cites as already carrying the flag. Both have the byte-identical step with the same shell spec and no set -e, so on both, a jupyter-builder failure currently passes CI. The masking is narrower than it sounds, because the later Build HTML step is single-command and does fail correctly, so most defects are still caught downstream; the real exposure is failures unique to the jupyter builder.

Suggested amendment to the fix in this issue: add set -eo pipefail as the first line of that run: block wherever the flag is added, and retrofit it to the repos already considered aligned. Only multi-command run: blocks using an explicit shell: are affected — single-command steps are fine, since their exit code is the command's.

Applied in QuantEcon/lecture-python.zh-cn#185 (commit 31c8082), caught by Copilot review on that PR.

🤖 Generated with Claude Code