Refactor write_benchmark_json to comply with github-action-benchmark format and ensure time series creation#948
Merged
Merged
Conversation
… with github-action-benchmark format)
JanWeldert
approved these changes
Jun 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Turns out the workflow failed because the script had created dictionary entries when only a simple list of results was allowed.
Another problem which appeared in my branch was the automatic removal of the previous benchmarking results in the gh-pages branch by the documentation workflow.
As a result, I've adapted the documentation workflow, on the one hand, to output to
docs, not the root directory, of gh-pages, and, more importantly, configured it tokeep_files. This way,dev/bench/data.jswill not get deleted and instead will be extended.It just means we will need to set the PISA (docs) URL in the right column to icecube.github.io/pisa/docs (leaving the Pages source at
/ (root)in settings.Also, ideally before this PR is merged, we should manually delete the gh-pages branch once so that the files currently in its root directory are removed.
In the future, the benchmark time series will then be extended as long as
dev/bench/data.jsin gh-pages isn't deleted. If —for some reason—it does get deleted at some point without there being a backup, the time series will just start anew.Finally, an entry point for the benchmarking script is added to setup.py.