Repository navigation
Take the best of 15 timed pairs in the tapscript scaling ratio tests (issue #2613) - #2623
Merged
Merged
Conversation
pmazzocchi
reviewed
Oct 11, 2026
pmazzocchi
left a comment
Contributor
There was a problem hiding this comment.
A fresh local review cleared f2b5fa5. Nothing blocks it. Four points:
- Mutation cost.
.github/mutation/engine.toml:31andsig_hash.toml:43run all oftests/script_enginefor every mutant, under a 90-minute budget. On this head CI took 19.6 s and 21.3 s for the two ratio tests; 5-pair branches took 3.3–7.0 s. So each session reaches fewer mutants. One option is to deselect the ratio tests in those twotest-commands. That is a design choice, so it is @fametrano's to make. This is an estimate from CI logs, not a measured session. - Commit message: the 2.3–6.0 extremes are given "at load 5" there, "at loads 2 to 11" in the body, and a load of 13 in the thread. Make them agree in the squash.
- Commit message: "over 50 runs" is a count. It belongs in the body, not in what lands.
- Body: ISS 2613 is not a link.
Contributor
|
Claimed: a /btclib-pr session (pmazzocchi) is rebasing this onto main (bbb0555), running the gates and taking it to the decision. The branch holds no other change. |
…(issue #2613) The ratio of the best large run to the best small run failed at 12.0 against a limit of 10 at a 1-minute load of 20 to 23, twice, and passed on a re-run. Interleaved timing (PR #2560) was already in place. On a quiet machine the ratio is about 4, from 2.3 to 6.0 over 50 runs of each case at load 5, and none failed. Load adds the time a run is descheduled, which a longer run is more likely to suffer, so the best of more runs is what is asked of both sides: 15 instead of 5, in one helper for the two tests. The limit stays at 10 against 16 for a quadratic implementation. The failure rate at a load of 20 or more is unmeasured, so the issue stays open for it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Ferdinando Ametrano <ferdinando@ametrano.net>
pmazzocchi
force-pushed
the
tapscript-scaling-2613
branch
from
October 11, 2026 21:30
f2b5fa5 to
42cc95d
Compare
There was a problem hiding this comment.
ACK 42cc95d
- non-blocking,
tests/script_engine/tapscript_scaling_test.py:151,161: the ratio tests now make 15 timed pairs instead of 5, but their timeout is still_BOUND * 5(300 s). That leaves about 16 s for a linear large run, down from about 48 s. The single-run tests allow 60 s. So under load the timeout can now fail a linear implementation. Known from the decorators and the loop count in_ratio.
pmazzocchi
approved these changes
Oct 11, 2026
pmazzocchi
left a comment
Contributor
There was a problem hiding this comment.
Approves head 42cc95d.
- Relies on a fresh local reviewer's
CLEARED f2b5fa5b6. The rebase onto bbb0555 is a range-diff=, so the clearance carries over. Gates rerun on 42cc95d: pre-commit, pytest (100% coverage) and sphinx -W all exit 0. - Bot ACK: #2623 (review)
- Every check passed or was skipped on this head.
- Not checked: the Sign-off script was not run locally (CI's
lint / Sign-offpassed). The failure rate at load 20 or more is still unmeasured, as the body says.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request cites ISS 2613 and does not close it. Nobody has measured the failure rate at a load of 20 or more, where the issue saw the ratio reach 12.0. The issue stays open for that measurement.
The two ratio tests in
tests/script_engine/tapscript_scaling_test.pynow share one helper,_ratio. It times 15 interleaved small and large runs instead of 5, and takes the best of each. The limit stays at 10, against 16 for a quadratic implementation. Load only adds time, and it is likelier to hit the longer run. A ratio of 12 from a quiet value of about 4 therefore needs every large run stretched. The best of 15 is far less likely to be stretched than the best of 5.Measured at loads 2 to 11: the ratios were about 4, with extremes 2.3 and 6.0 and no failure. A reviewer planted a quadratic signature-check cost, and the test failed at 13.7. A CPU-time clock was rejected, because Windows' 15.6 ms granularity is coarser than the small run.
Gates on bd9e0ae, whose tree is this sha's (f2b5fa5 changes the message only), run by the writer under the gate lock:
uv run pytest: exit 0, coverage 100%uv run pre-commit run --all-files: exit 0sphinx-build -n -W -b html docs/source docs/build/html: exit 0The local review cleared f2b5fa5, the branch's only commit on b71525a. No rebase was needed.
🤖 Generated with Claude Code