feat: add CodSpeed benchmark tests for graph operations#922
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #922 +/- ##
==========================================
+ Coverage 78.10% 78.30% +0.19%
==========================================
Files 130 131 +1
Lines 12186 12295 +109
Branches 1481 1489 +8
==========================================
+ Hits 9518 9627 +109
Misses 2279 2279
Partials 389 389 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Add pytest-codspeed benchmarks testing core graph operations (transitive closure, topological sort, link dicts, for_each_task, JSON round-trip, and transform sequences) across 4 graph geometries with 1000 tasks each: linear chain, fan-out/fan-in, binary tree, and diamond layers.
| - name: Run benchmarks | ||
| uses: CodSpeedHQ/action@v4 | ||
| with: | ||
| run: uv run pytest test/ --codspeed |
There was a problem hiding this comment.
tbh I'm unsure if we should test everything or just test/test_graph_perf.py
There was a problem hiding this comment.
I can't imagine why we'd want to run it on the non-perf tests? (In fact, we should probably exclude the new perf test from running as a unit test as well?)
| - name: Run benchmarks | ||
| uses: CodSpeedHQ/action@v4 | ||
| with: | ||
| run: uv run pytest test/ --codspeed |
There was a problem hiding this comment.
I can't imagine why we'd want to run it on the non-perf tests? (In fact, we should probably exclude the new perf test from running as a unit test as well?)
| @@ -0,0 +1,30 @@ | |||
| --- | |||
There was a problem hiding this comment.
What's the advantage to running this as a GHA instead of in taskcluster? (I presume there is one; it should probably be documented as a top level comment.)
There was a problem hiding this comment.
Mostly for ease of use and maintainability.
If we use TC we'd also need to set it up with a "legacy Token" auth method on Codspeed.
GH has OIDC authentication built-in and works out of the box with the service.
3 participants
Add pytest-codspeed benchmarks testing core graph operations (transitive closure, topological sort, link dicts, for_each_task, JSON round-trip, and transform sequences) across 4 graph geometries with 1000 tasks each: linear chain, fan-out/fan-in, binary tree, and diamond layers.