Skip to content

[WIP] PyTorch ROCTx Observer Implementation#3108

Open
EvanPhoukong wants to merge 3 commits intodevelopfrom
users/ephoukong/roctx-observer
Open

[WIP] PyTorch ROCTx Observer Implementation#3108
EvanPhoukong wants to merge 3 commits intodevelopfrom
users/ephoukong/roctx-observer

Conversation

@EvanPhoukong
Copy link
Copy Markdown

@EvanPhoukong EvanPhoukong commented Mar 27, 2026

Motivation

This branch adds a custom ROCTX observer and wires it to a public API so users can annotate entire training regions with emit_roctx() without relying on Kineto for that behavior.

Technical Details

  • Standalone observer (C++): torch/csrc/profiler/standalone/roctx_observer.cpp implements ROCTXThreadLocalState and pushROCTXCallbacks(), which register enter/exit callbacks. On activation, the observer emits roctxMarkA("PyTorch_ROCTX_observer_v2_active") so traces can confirm the v2 path; each op pushes/pops a named ROCTX range (with optional input-shape reporting).
  • Profiler integration: torch/csrc/autograd/profiler_kineto.cpp routes ProfilerState::ROCTX to pushROCTXCallbacks() (same class of “non-Kineto” profilers as NVTX). ProfilerState / ActiveProfilerType expose ROCTX in the C++ profiler API and Python bindings (torch/csrc/profiler/python/init.cpp, torch/csrc/profiler/orchestration/observer.h).
  • Python API: torch.autograd.profiler.emit_roctx (torch/autograd/profiler.py) is the context manager that enables ProfilerState.ROCTX for the wrapped block.
  • Explicit ROCTX from Python: Low-level bindings live under torch._C._roctx (torch/csrc/cuda/shared/roctx.cpp) with a torch.cuda.roctx facade (torch/cuda/roctx.py) for manual range / mark style annotations (orthogonal to enabling the autograd observer).

Test Plan

Test Result

Submission Checklist

@rocm-repo-management-api
Copy link
Copy Markdown

rocm-repo-management-api bot commented Mar 27, 2026

Jenkins build for 5b031ede28630bc88ee00467913316c1782cb80a commit finished as FAILURE
Links: Pipeline Overview / Build artifacts / Test Results

@EvanPhoukong EvanPhoukong changed the title PyTorch ROCTx Observer Implementation [WIP] PyTorch ROCTx Observer Implementation Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant