Skip to content

[Feature] Extract reusable event-based benchmarking helper#829

Draft
jhinpan wants to merge 1 commit into
ROCm:mainfrom
jhinpan:feat/profile-tools
Draft

[Feature] Extract reusable event-based benchmarking helper#829
jhinpan wants to merge 1 commit into
ROCm:mainfrom
jhinpan:feat/profile-tools

Conversation

@jhinpan

@jhinpan jhinpan commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Extract the existing GPU event benchmark helper from flydsl.autotune into the reusable flydsl.profiling module.

This is intentionally a narrow, non-intrusive first slice of #304. It does not move the broader tests/test_common.py utilities or change any kernel, compiler, or backend implementation.

Motivation

FlyDSL already has a small event-based do_bench implementation, but it is nested inside the autotune module. Moving it to a shared package-level module lets other profiling and benchmarking callers reuse the same behavior and gives the autotune path in #770 a clearer dependency boundary.

The module does not import ROCDL, NVVM, or FlyDSL compiler bindings. Its current runtime implementation lazily uses PyTorch’s torch.cuda event interface, which PyTorch also exposes on HIP builds.

This PR does not implement or claim validation of a FlyDSL NVVM backend. Runtime/backend placement for the future dual-backend direction remains related to #813; the focused namespace question is recorded on #304.

Changes

  • Add flydsl.profiling.do_bench.
  • Preserve existing valid-call timing behavior:
    • configurable warmup and measured iteration counts;
    • CUDA/HIP event timing on PyTorch's current stream;
    • millisecond results;
    • existing upper-middle median and quantile selection semantics.
  • Support the setup hook introduced in autotune [2/5] autotune: add opt-in search and RMSNorm waves-per-EU tuning (#770) #785, running it before each warmup and timed iteration but before the start event so restore/reset work is excluded from kernel latency.
  • Keep flydsl.autotune.do_bench as a compatibility alias.
  • Load PyTorch only when profiling is invoked.
  • Raise focused errors for missing PyTorch, no visible GPU, and invalid warmup/rep/quantile arguments.
  • Document the API and its current-stream contract.
  • Add GPU-free tests for timing, quantiles, compatibility, error paths, and argument validation.

Compatibility is preserved for existing autotune callers: the import path, valid call signature, defaults, and result shape remain unchanged.

Performance

Not applicable. This is a direct extraction of the existing timer and does not change generated kernels or compiler behavior.

A live MI350X/HIP smoke test returned a valid event-based median timing.

Testing

  • GPU-free profiling and existing autotune unit tests:

    PYTHONPATH="$PWD/python${PYTHONPATH:+:$PYTHONPATH}" \
      python3 -m pytest tests/unit/test_profiling.py tests/unit/test_autotune.py -q

    Result: 31 passed.

  • Repository Python style gate (bash scripts/check_python_style.sh --include-local).

  • Targeted Black and Ruff checks for all changed Python files.

  • MI350X/HIP event-timing smoke test.

  • CUDA/NVVM execution was not tested and is not claimed by this PR.

Dependencies

  • No new third-party dependencies added.
  • PyTorch was already used by the extracted implementation and is now imported lazily.

Breaking Changes

None for valid existing calls. flydsl.autotune.do_bench remains available as an alias to the shared implementation.

Refs #304, #770, #813.

Signed-off-by: Jin Pan <jpan236@wisc.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant