Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion windows/bench/Emit-PairedSession.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -447,7 +447,7 @@ $doc = [ordered]@{
requested_samples = [int]$sess.rounds
interleaving = 'round-robin-randomized'
label = 'kel25-windows-keld-vs-tauri-paired-30'
notes = ('First PAIRED Windows measurement in this repo. Round-major randomized interleaving, seed ' + $sess.seed + ': every round runs each arm exactly once and the order within the round is shuffled, so a session-scale trend is sampled by both arms alike instead of accumulating on one. This does NOT eliminate drift: within a round the second arm is still measured later than the first by about one launch, and shuffling makes which arm that is random rather than removing the gap. The claim is reduced systematic order bias, not an absence of within-session drift. Both arms drive the SAME system engine build (WebView2 Evergreen ' + $sess.webview2_start + '), verified identical at both session boundaries. SCORED VALUE is the native host process working set for both arms: RSS is RESIDENT set size and metrics.v1.json MEM-IDLE directs scoring "the Keld-owned (main) RSS". THE COMPARISON IS HOST-TO-HOST ONLY. The Keld arm additionally runs application JavaScript in a supervised Bun child over authenticated kipc; the Tauri fixture backend is tauri::Builder::default().run() and runs no application JavaScript at all. framework_ws_kib (host+runtime) is recorded per run as a diagnostic and is deliberately excluded from the comparison block, because comparing a host-plus-JS-runtime against a backend that runs no JS is not a peer result. The first launch of each arm is an explicit unscored WARMUP sample, recorded with reason WARMUP_UNSCORED rather than silently dropped, because Defender first-touch of a fresh binary is a large outlier and interleaving relocates that bias onto whichever arm draws round 1 rather than removing it. Paired ratio CI resamples whole ROUNDS with replacement so the pairing is preserved. DISCLOSURES, none of which block publication under HARNESS-CONTRACT.md but all of which bound how this number may be read. (1) CAPABILITY ASYMMETRY: the Keld arm runs application JavaScript in a supervised Bun child over authenticated kipc; the Tauri fixture backend is a bare tauri::Builder that runs none. The comparison is native-host-to-native-host only. framework_ws_kib (host+runtime) is recorded per run and deliberately excluded from the comparison block; do not read this as total application footprint. (2) PAYLOAD PARITY IS SOURCE-LEVEL, NOT ENVIRONMENT-LEVEL: both arms are handed byte-identical document bytes, verified by extracting the page from the Tauri artifact this document cites, but the rendered environments are not identical. Tauri exposes __TAURI_INTERNALS__, __TAURI_EVENT_PLUGIN_INTERNALS__, ipc and isTauri before the document runs and loads via a custom asset protocol at http://tauri.localhost; Keld exposes none of those and loads via NavigateToString at origin null. (3) REGISTRY SCOPE: architecture 01 section 5 budgets the sum of keld processes while the metrics.v1.json note directs scoring the Keld-owned (main) RSS. This document scores the main process, which is also the only like-for-like quantity across the two arms. Both readings pass the budget, so it changes no verdict, but the registry should still state which is scored. (4) TAURI NPM LAYER UNPINNED: windows/tauri/hello/package-lock.json is gitignored, so that fixture npm layer is not reproducible from this repo. Its Cargo.lock is committed and the CLI is not on the measured build path. (5) SCOPE: the Keld arm measures the full keld dev developer flow (doctor checks, echo server, supervised Bun spawn, authenticated kipc echo, then window) against a packaged Tauri Release exe. Payload parity does not make these scope-matched.')
notes = ('First PAIRED Windows measurement in this repo. Round-major randomized interleaving, seed ' + $sess.seed + ': every round runs each arm exactly once and the order within the round is shuffled, so a session-scale trend is sampled by both arms alike instead of accumulating on one. This does NOT eliminate drift: within a round the second arm is still measured later than the first by about one launch, and shuffling makes which arm that is random rather than removing the gap. The claim is reduced systematic order bias, not an absence of within-session drift. The session-boundary integrity check compares the installed WebView2 Evergreen registry version (start: ' + $sess.webview2_start + '); equality of those registry observations does not verify the runtime build loaded by either arm. SCORED VALUE is the native host process working set for both arms: RSS is RESIDENT set size and metrics.v1.json MEM-IDLE directs scoring "the Keld-owned (main) RSS". THE COMPARISON IS HOST-TO-HOST ONLY. The Keld arm additionally runs application JavaScript in a supervised Bun child over authenticated kipc; the Tauri fixture backend is tauri::Builder::default().run() and runs no application JavaScript at all. framework_ws_kib (host+runtime) is recorded per run as a diagnostic and is deliberately excluded from the comparison block, because comparing a host-plus-JS-runtime against a backend that runs no JS is not a peer result. The first launch of each arm is an explicit unscored WARMUP sample, recorded with reason WARMUP_UNSCORED rather than silently dropped, because Defender first-touch of a fresh binary is a large outlier and interleaving relocates that bias onto whichever arm draws round 1 rather than removing it. Paired ratio CI resamples whole ROUNDS with replacement so the pairing is preserved. DISCLOSURES, none of which block publication under HARNESS-CONTRACT.md but all of which bound how this number may be read. (1) CAPABILITY ASYMMETRY: the Keld arm runs application JavaScript in a supervised Bun child over authenticated kipc; the Tauri fixture backend is a bare tauri::Builder that runs none. The comparison is native-host-to-native-host only. framework_ws_kib (host+runtime) is recorded per run and deliberately excluded from the comparison block; do not read this as total application footprint. (2) PAYLOAD PARITY IS SOURCE-LEVEL, NOT ENVIRONMENT-LEVEL: both arms are handed byte-identical document bytes, verified by extracting the page from the Tauri artifact this document cites, but the rendered environments are not identical. Tauri exposes __TAURI_INTERNALS__, __TAURI_EVENT_PLUGIN_INTERNALS__, ipc and isTauri before the document runs and loads via a custom asset protocol at http://tauri.localhost; Keld exposes none of those and loads via NavigateToString at origin null. (3) REGISTRY SCOPE: architecture 01 section 5 budgets the sum of keld processes while the metrics.v1.json note directs scoring the Keld-owned (main) RSS. This document scores the main process, which is also the only like-for-like quantity across the two arms. Both readings pass the budget, so it changes no verdict, but the registry should still state which is scored. (4) TAURI NPM LAYER UNPINNED: windows/tauri/hello/package-lock.json is gitignored, so that fixture npm layer is not reproducible from this repo. Its Cargo.lock is committed and the CLI is not on the measured build path. (5) SCOPE: the Keld arm measures the full keld dev developer flow (doctor checks, echo server, supervised Bun spawn, authenticated kipc echo, then window) against a packaged Tauri Release exe. Payload parity does not make these scope-matched.')
}
environment = [ordered]@{
os = [ordered]@{ name = 'windows'; version = $os.Caption; build = "$($os.Version)" }
Expand Down