feat(starry): perf call-graph capture (PERF_SAMPLE_CALLCHAIN) - #1
Open
JosephJoshua wants to merge 6 commits into
Open
feat(starry): perf call-graph capture (PERF_SAMPLE_CALLCHAIN)#1JosephJoshua wants to merge 6 commits into
JosephJoshua wants to merge 6 commits into
Conversation
The PMU overflow handler runs inside dispatch_irq and can read the interrupted PC/EL live (ELR_EL1/SPSR_EL1), but the interrupted frame pointer (x29) survives only in the saved TrapFrame — the handler's own frames have already clobbered the GPR. Call-graph sampling (PERF_SAMPLE_CALLCHAIN) needs that x29. Publish a per-CPU *const TrapFrame at the two IRQ-entry sites (EL1 kernel interrupt in trap.rs, EL0 user interrupt in UserContext::run) immediately before dispatch_irq and clear it immediately after, both inside the IRQ-masked window so no nested IRQ observes a stale frame. Add interrupted_fp()/interrupted_sp() accessors alongside the existing interrupted_pc()/interrupted_is_user(). The publish/clear is a single per-CPU store each (TPIDR-relative, cannot fault) and does not reorder the register save/restore path. Pulls ax-percpu into the aarch64 dependency set (already present for x86_64) for the per-CPU cell.
…e getters Add walk_fp(pc, fp, ip_range, fp_range, read, out) -> usize: an allocation-free frame-pointer walk into a caller-supplied buffer that reads memory through an injected closure instead of dereferencing directly, so a hard-IRQ caller (PMU-sampling call-graph capture) can supply a fault-safe reader. Mirrors unwind_core's guards (monotonic fp, 8 MiB jump cap, word alignment, depth cap via out.len()) but never allocates and is not behind the alloc feature, so it is available in the default (no-alloc) kernel build. Add ip_range()/fp_range() getters over the ranges init() records, each a lock-free spin::Once load safe to call from IRQ context, so the sampling unwinder can reuse the kernel text / address-space bounds. Serialize the depth-sensitive tests: set_max_depth mutates a process global, so init_for_tests now returns a guard that holds a shared lock for the test body, removing a latent flake under cargo test's parallel harness (surfaced by the new walk_fp tests raising thread contention).
Emit PERF_SAMPLE_CALLCHAIN so perf record -g captures per-sample kernel stack backtraces (frame-pointer), rendering as flamegraphs off-target. - perf/unwind.rs: kernel-side integration over axbacktrace::walk_fp — kernel_ranges() reuses the boot-time backtrace ranges, a direct-deref reader for always-mapped kernel frames, and kernel_callchain() that walks the interrupted x29 chain into a fixed buffer. - sampling.rs: define PERF_SAMPLE_CALLCHAIN / PERF_CONTEXT_KERNEL / PERF_CONTEXT_USER, add CALLCHAIN to SUPPORTED_SAMPLE_TYPE (both hw.rs open sites accept it via the mask), grow SAMPLE_RECORD_MAX_LEN to bound the callchain block, emit 'u64 nr + entries' in build_sample in Linux field order, and build the chain in the overflow handler from the interrupted frame pointer (leaf-only fallback when none is published, so a sample is never dropped). Kernel frames only in this pass; the user region is a leaf IP with a PERF_CONTEXT_USER marker (the no-fault user FP unwind is M4b). Sampling stays aarch64-only and allocation-free in IRQ context.
…d bound Follow-up hardening from adversarial review of the PMU-sampling callers: - Bound TOTAL iterations (a small multiple of out.len()), not just recorded frames. A return address outside ip_range is skipped without consuming a recorded slot, so the depth cap alone did not bound the loop; a crafted all-skip user chain (every [fp+8] out of range, monotonic small-gap [fp]) could otherwise spin the walk — each step doing real work (two no-fault page-table walks) — into an IRQ-latency DoS. Apply the 8 MiB-gap check on every fp advance (both the skip and record paths), not only after recording. - Bound the whole frame record [fp, fp+16) inside fp_range (not just fp), so the saved-LR read at fp+8 can never touch one word past fp_range.end. Adds a test that would hang without the step cap (an ever-advancing all-skip chain) and confirms it terminates recording only the leaf.
Add user-space callchains (PERF_SAMPLE_CALLCHAIN user region) and make the whole FP unwind fault-proof from hard-IRQ context. - perf/nofault.rs: read_user_word_nofault / read_kernel_word_nofault walk TTBR0 / TTBR1 by hand (4-level, 48-bit VA) against the always-mapped direct map, never dereferencing the input VA. Every table frame and the resolved leaf is bound-checked against phys_ram_ranges() before deref, so a valid PTE whose output is device MMIO (a user mmap of an RGA/NPU/JPU register window, or a concurrently-torn table on another CPU) can never trigger a live MMIO read (device side effect) or a data abort (panic) in the overflow handler. - perf/unwind.rs: route the kernel reader through the no-fault path too, so a corrupt/stale in-kernel frame pointer that slips past the range guards yields None instead of faulting; add user_callchain, bounding the walk to a window above the interrupted SP_EL0. - perf/sampling.rs: build_callchain now unwinds the user region for user samples (deep [PERF_CONTEXT_USER, ip, ra…]) via the no-fault reader, seeded from the interrupted x29 + SP. Kernel deep frames still require an FP-enabled kernel (-Cforce-frame-pointers); user frames unwind whenever the sampled binary keeps frame pointers.
Two grouped-C qemu/system cases exercising PERF_SAMPLE_CALLCHAIN end to end (open sampling event with IP|TID|TIME|CALLCHAIN, mmap ring, parse the callchain block, classify by PERF_CONTEXT_* markers): - perf-hw-callchain-kernel: asserts a well-formed callchain record with a PERF_CONTEXT_KERNEL region + leaf IP. CI-safe: does NOT require a deep kernel chain, since the default kernel is built without frame pointers (like Linux without CONFIG_FRAME_POINTER), so the kernel region is leaf-only. - perf-hw-callchain-user: builds a known nested chain outer->mid->inner->busy with -fno-omit-frame-pointer and asserts >= 4 user IPs after PERF_CONTEXT_USER — proving the kernel unwinds several USER frames via the no-fault TTBR0 reader. Uses a read(/dev/zero) workload: QEMU-TCG's cycle counter barely advances on a pure-ALU loop, so the sampling counter would seldom overflow. Both skip-as-pass off aarch64. Validated in QEMU (smp4): kernel record well-formed; user unwinds 8 frames deep across repeated runs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
在多核硬件 PMU perf（rcore-os#1577）之上，为 PMU 采样补齐调用图捕获（
PERF_SAMPLE_CALLCHAIN），使perf record -g能采集每个样本的栈回溯（内核 + 用户，基于帧指针），从而在主机侧用perf report/stackcollapse生成火焰图——这是当前 perf 功能集中最大的缺口。此前采样只有一个叶子 IP（hw.rs在 open 时显式拒绝PERF_SAMPLE_CALLCHAIN）。主要内容
dispatch_irq内，能实时读中断 PC/EL（ELR_EL1/SPSR_EL1），但被中断的x29（帧指针）已被处理程序自身的栈帧覆盖。在两个 IRQ 入口（EL1 内核中断 / EL0 用户中断）以 per-CPU 指针发布被中断的TrapFrame，dispatch_irq前置、返回后即清；新增interrupted_fp()/interrupted_sp()（axcpu）。axbacktrace::walk_fp）：读取器注入、固定缓冲、镜像unwind_core的护栏（单调递增 fp、8 MiB 跳变、字对齐、深度/总步数上限），可在硬中断上下文使用。sampling.rs）：新增PERF_SAMPLE_CALLCHAIN/PERF_CONTEXT_KERNEL(-128) /PERF_CONTEXT_USER(-512)，加入SUPPORTED_SAMPLE_TYPE（两处 hw open 校验自动放行），按 Linux 字段序在build_sample末尾发出u64 nr+ 各条目，固定SAMPLE_RECORD_MAX_LEN覆盖最坏记录。TTBR0/TTBR1，只经直映射访问页表帧与解析出的叶子物理页，从不解引用输入 VA）。对每个将要解引用的物理地址都用phys_ram_ranges()做 RAM 边界检查，因此一个合法但指向设备 MMIO（用户 mmap 的 RGA/NPU/JPU 寄存器窗口）或另一核并发改表得到的撕裂表项，都不会在硬中断里触发 MMIO 副作用读或缺页 panic。验证
perf-hw-callchain-kernel校验内核区记录格式（含PERF_CONTEXT_KERNEL标记 + 叶子 IP，CI 安全：默认内核未开帧指针，故内核区只有叶子——与 Linux 未开CONFIG_FRAME_POINTER一致）。perf-hw-callchain-user以已知嵌套链outer→mid→inner→busy（-fno-omit-frame-pointer）断言PERF_CONTEXT_USER后 ≥4 个用户 IP，多次运行稳定回溯出 8 层，证明无缺页读取器确实走通了用户栈。walk_fp增加总步数上限（避免构造的全跳过链在硬中断里空转成 DoS）。说明
#[cfg(target_arch = "aarch64")]门控；axbacktrace::walk_fp/范围 getter 为纯core，不影响其它架构。-Cforce-frame-pointers（BACKTRACE=y）构建内核（同 LinuxCONFIG_FRAME_POINTER）；用户回溯只要被采样二进制保留帧指针即可。cargo xtask clippy与cargo fmt --all -- --check。