Skip to content

perf(quality-gates): Add tbt metrics to global startup and flow timer gates #41963

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
MajorLift wants to merge 10 commits into
base: main
Choose a base branch
from
+54 −0
Draft

perf(quality-gates): Add tbt metrics to global startup and flow timer gates #41963

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
49d150a
perf(quality-gates): add TBT thresholds to THRESHOLD_REGISTRY
MajorLift Apr 20, 2026
b1eda1e
Merge `CLS_THRESHOLDS` from main into `tbt` thresholds branch
MajorLift Apr 21, 2026
88d36eb
Merge main into branch, resolving \`CLS_THRESHOLDS\` / \`tbt\` conflict
MajorLift Apr 21, 2026
cd85d26
Merge branch 'main' into jongsun/perf/quality-gates/260420-tbt-thresh…
MajorLift Apr 22, 2026
b1f6aee
[skip-e2e] [force-builds]
MajorLift Apr 22, 2026
9db010e
Recalibrate `STANDARD_HOME` TBT thresholds from CI-observed values
MajorLift Apr 22, 2026
f1965d7
[skip-e2e] [force-builds]
MajorLift Apr 22, 2026
c65fb31
Fix \`STANDARD_HOME\` TBT calibration to match other presets
MajorLift Apr 23, 2026
36d90a6
[force-builds] [skip-e2e]
MajorLift Apr 27, 2026
707f906
Merge branch 'main' into jongsun/perf/quality-gates/260420-tbt-thresh…
MajorLift May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions test/e2e/benchmarks/utils/thresholds.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,12 @@
ciMultiplier: CI_MULTIPLIER.TIER_1,
},
...CLS_THRESHOLDS,
// Baseline: p75=485ms, p95=559ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 575, fail: 625 },
p95: { warn: 675, fail: 725 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 96 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const ONBOARDING_NEW_WALLET = {
Expand Down Expand Up @@ -128,6 +134,12 @@
ciMultiplier: CI_MULTIPLIER.TIER_1,
},
...CLS_THRESHOLDS,
// Baseline: p75=326ms, p95=377ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 400, fail: 425 },
p95: { warn: 450, fail: 500 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 141 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const IMPORT_SRP_HOME = {
Expand All @@ -152,6 +164,12 @@
ciMultiplier: CI_MULTIPLIER.TIER_2,
},
...CLS_THRESHOLDS,
// Baseline: p75=3354ms, p95=3552ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 4000, fail: 4350 },
p95: { warn: 4250, fail: 4625 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 171 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const SWAP = {
Expand All @@ -171,6 +189,12 @@
ciMultiplier: CI_MULTIPLIER.TIER_2,
},
...CLS_THRESHOLDS,
// Baseline: p75=438ms, p95=483ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 525, fail: 575 },
p95: { warn: 575, fail: 625 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 196 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const SEND_TRANSACTIONS = {
Expand All @@ -190,6 +214,12 @@
ciMultiplier: CI_MULTIPLIER.DEFAULT,
},
...CLS_THRESHOLDS,
// Baseline: p75=151ms, p95=151ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 200, fail: 225 },
p95: { warn: 200, fail: 225 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 221 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const ASSET_DETAILS = {
Expand All @@ -199,6 +229,12 @@
ciMultiplier: CI_MULTIPLIER.DEFAULT,
},
...CLS_THRESHOLDS,
// Baseline: p75=95ms, p95=127ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 125, fail: 150 },
p95: { warn: 150, fail: 175 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 236 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const SOLANA_ASSET_DETAILS = {
Expand All @@ -208,6 +244,12 @@
ciMultiplier: CI_MULTIPLIER.DEFAULT,
},
...CLS_THRESHOLDS,
// Baseline: p75≈1ms, p95≈1ms — floor applied (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 50, fail: 75 },
p95: { warn: 75, fail: 100 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 251 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

const STANDARD_HOME = {
Expand All @@ -227,6 +269,12 @@
ciMultiplier: CI_MULTIPLIER.TIER_1,
},
...CLS_THRESHOLDS,
// Baseline from CI: p75≈850ms, p95≈1021ms (2026-04-22, chrome-browserify).
tbt: {
p75: { warn: 1020, fail: 1100 },
p95: { warn: 1225, fail: 1325 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 276 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
Comment thread
cursor[bot] marked this conversation as resolved.
Comment thread
cursor[bot] marked this conversation as resolved.
Copy link
Copy Markdown

@cursor cursor Bot May 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STANDARD_HOME TBT thresholds ~2.7x higher than documented

Medium Severity

The STANDARD_HOME tbt baseline comment says p75≈850ms / p95≈1021ms (from 2026-04-22, chrome-browserify), producing thresholds of warn=1020/fail=1100. But the PR description table documents a p75=313ms baseline with thresholds of warn=375/fail=425 — about 2.7x tighter. All other presets match their documented baselines. This discrepancy means the quality gate for startupStandardHome TBT is far more permissive than intended, potentially masking regressions.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 707f906. Configure here.

} satisfies ThresholdConfig;

const POWER_USER_HOME = {
Expand All @@ -246,6 +294,12 @@
ciMultiplier: CI_MULTIPLIER.STARTUP_POWER_USER,
},
...CLS_THRESHOLDS,
// Baseline: p75=1189ms, p95=1367ms (2026-04-07 – 2026-04-20, ci.branch:main)
tbt: {
p75: { warn: 1425, fail: 1550 },
p95: { warn: 1650, fail: 1775 },
ciMultiplier: DEFAULT_CI_MULTIPLIER,

Check failure on line 301 in test/e2e/benchmarks/utils/thresholds.ts

View workflow job for this annotation

GitHub Actions / Test lint 

Cannot find name 'DEFAULT_CI_MULTIPLIER'.
},
} satisfies ThresholdConfig;

// Threshold keys must match timer IDs emitted by the benchmark flows (snake_case).
Expand Down
Loading