Merge pull request #8 from Fidget-Spinner/frame-pointers-kj2

Add perf figures, tighten JIT claims, temporarily remove Diego

Author: pablogsal

peps/pep-0830.rst

@@ -3,7 +3,6 @@ Title: Frame Pointers Everywhere: Enabling System-Level Observability for Python
 Author: Pablo Galindo Salgado <[email protected]>,
         Ken Jin <[email protected]>,
         Savannah Ostrowski <[email protected]>,
-        Diego Russo <[email protected]>
 Discussions-To:
 Status: Draft
 Type: Standards Track
@@ -405,18 +404,18 @@ The JIT Compiler Needs Frame Pointers to Be Debuggable
 ------------------------------------------------------
 
 CPython's copy-and-patch JIT (:pep:`744`) generates native machine code at
-runtime.  Without frame pointers in the interpreter, stack unwinding through
+runtime.  Without reserved frame pointers in the JIT code, stack unwinding through
 JIT frames is broken for virtually every tool in the ecosystem: GDB, LLDB,
 libunwind, libdw (elfutils), py-spy, Austin, pystack, memray, ``perf``, and
 all eBPF-based profilers.  Ensuring full-stack observability for JIT-compiled
 code is a prerequisite for the JIT to be considered production-ready.
 
 Individual JIT stencils do not need frame-pointer prologues; the entire JIT
 region can be treated as a single frameless region for unwinding purposes.
-What matters is that the interpreter itself is built with frame pointers, so
+What matters is that the JIT itself is must reserve frame pointers, so
 that the frame-pointer register (``%rbp`` on x86-64, ``x29`` on AArch64) is
 reserved and not clobbered by stencil code.  With frame pointers in the
-interpreter, unwinders can walk through JIT regions without needing to inspect
+JIT, most unwinders can walk through JIT regions without needing to inspect
 individual stencils.  This is a remarkably good outcome compared to other
 JIT compilers (V8, LuaJIT, .NET CoreCLR, Julia, LLVM's ORC JIT), which
 typically require hundreds to thousands of lines of code to implement custom
@@ -840,14 +839,14 @@ pyperformance JSON files can be found in
 =====================================  =======================
 Machine                                Geometric mean overhead
 =====================================  =======================
-Apple M2 Mac Mini (arm64)              1.01x slower
-Intel Xeon Platinum 8480 (x86-64)      1.01x slower
-AMD EPYC 9654 (x86-64)                 1.01x slower
-AWS Graviton c7g.16xlarge (aarch64)    1.02x slower
-Ampere Altra Max (aarch64)             1.01x slower
-Raspberry Pi (aarch64).                1.00x slower
-macOS M3 Pro (arm64)                   1.00x slower
-Intel i7 12700H (x86-64)               1.02x slower
+Apple M2 Mac Mini (arm64)              1.006x slower
+macOS M3 Pro (arm64)                   1.001x slower
+Raspberry Pi (aarch64).                1.002x slower
+Ampere Altra Max (aarch64)             1.020x slower
+AWS Graviton c7g.16xlarge (aarch64)    1.027x slower
+Intel i7 12700H (x86-64)               1.019x slower
+AMD EPYC 9654 (x86-64)                 1.008x slower
+Intel Xeon Platinum 8480 (x86-64)      1.006x slower
 =====================================  =======================
 
 This overhead applies to both the interpreter and to C extensions that inherit
@@ -1048,7 +1047,20 @@ Footnotes
 Appendix
 ========
 
-# TODO: KJ, once we have Diego's results.
+For all graphs below, the green dots are geometric means of the
+individual benchmark's median, while orange lines are the median of our data points.
+Hollow circles reperesent outliers.
+
+The first graph is the overall effect on pyperformance seen on each system.
+Apart from the Ubuntu AWS Graviton System, all system configurations have below 2%
+geometric mean and median slowdown:
+
+.. image:: pep-0830_perf_over_baseline.svg
+
+For individual benchmark results, see the following:
+
+.. image:: pep-0830_perf_over_baseline_indiv.svg
+
 
 Copyright
 =========