Add full-stack E2E test infrastructure for Boost Cloud pipeline by LiamSarsfield · Pull Request #47961 · Automattic/jetpack

LiamSarsfield · 2026-04-06T19:50:48Z

Fixes AIE-95

Proposed changes

Add Playwright full-stack test infrastructure that exercises the complete Boost Cloud pipeline (WordPress → Shield → Redis → Hydra → callback) against the local dev Docker environment
Create FullStackUtils class with Docker-aware helpers: executeDevWpCommand (targets dev container with --url flag), flushRedis, waitForCssGeneration (WP-CLI polling), captureDockerLogs (attached to Playwright report on failure)
Create full-stack-test.ts fixture extending base-test from e2e-commons
Create full-stack-global-setup.ts setup project with 9 gates: config validation, WordPress/Shield/Redis/Hydra health checks, boost-developer verification, debug mu-plugin guard, Redis flush, and dev WordPress authentication
Modify playwright.config.ts: add testIgnore for full-stack specs on existing project, conditionally register full-stack setup and full-stack projects when BOOST_CLOUD_DIR is set, per-project baseURL read from boost-cloud .env
Add Critical CSS smoke test (specs/full-stack/critical-css.test.ts) that validates CSS generation through Shield+Hydra and checks frontend output
Add cloud_css to CLI module allowlist in class-cli.php
Fire jetpack_boost_module_status_updated action in CLI set_module_status() so CLI module activation triggers the same hooks as the DataSync REST path (e.g., Cloud_CSS::activate() → Regenerate::start())

Other information

Full-stack tests target the dev WordPress container (not e2e) because boost-cloud services run on jetpack_dev_default Docker network — the e2e container is on a separate network where boost-shield:1982 DNS does not resolve
All WP-CLI commands pass --url=http://${DEV_DOMAIN} because inside Docker $_SERVER["HTTP_HOST"] is empty and wp-config.php defaults WP_HOME to http://localhost, which Hydra containers cannot reach
The do_action addition in class-cli.php is a behavioral change: previously CLI module activate only wrote the option, now it also fires the module status hook. This aligns CLI behavior with the DataSync REST path and is required for Cloud CSS generation to trigger via CLI

Related product discussion/links

Does this pull request change what data or activity we track or use?

No.

Testing instructions

Prerequisites

All three local environments must be running:

Jetpack dev Docker: pnpm jetpack docker up -d (from monorepo root)
boost-cloud Docker: docker compose up -d (from boost-cloud repo)
boost-developer plugin installed and active in dev WordPress (via setup-boost-local)

Run full-stack tests

cd projects/plugins/boost/tests/e2e
BOOST_CLOUD_DIR=/path/to/boost-cloud npx playwright test --project=full-stack
Verify the Playwright HTML report shows:
- full-stack setup project: all 9 gates pass (health checks + auth)
- full-stack project: both Critical CSS tests pass (generation + frontend check)

Verify existing tests are unaffected

npx playwright test --project="jetpack boost e2e" (without BOOST_CLOUD_DIR)
Existing mock-based tests should pass normally — testIgnore prevents them from picking up specs/full-stack/

Verify CLI `do_action` change

pnpm jetpack docker wp -- jetpack-boost module activate cloud_css --url=http://jetpack-boost.test (with boost-developer css_mode: cloud)
Verify CSS generation starts (check wp option get jetpack_boost_ds_critical_css_state --format=json shows pending → generated)

Create Playwright fixtures, setup project, and smoke test that exercise the complete cloud pipeline (WordPress → Shield → Redis → Hydra → callback) against the local dev Docker environment. - Add FullStackUtils class with Docker-aware helpers (executeDevWpCommand, flushRedis, waitForCssGeneration, captureDockerLogs) - Create full-stack-test.ts fixture extending base-test from e2e-commons - Create full-stack-global-setup.ts with 9 health-check/auth gates - Add Critical CSS smoke test validating generation through Shield+Hydra - Add cloud_css to CLI module allowlist in class-cli.php - Fire jetpack_boost_module_status_updated action in CLI set_module_status() so CLI activation triggers the same hooks as the DataSync REST path

github-actions · 2026-04-06T19:51:52Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
✅ Add a "[Status]" label (In Progress, Needs Review, ...).
✅ Add testing instructions.
✅ Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

Follow this PR Review Process:

Ensure all required checks appearing at the bottom of this PR are passing.
Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
You can use GitHub's Reviewers functionality to request a review.
When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

Boost plugin:

No scheduled milestone found for this plugin.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

jp-launch-control · 2026-04-06T19:55:22Z

Code Coverage Summary

Coverage changed in 1 file.

File	Coverage	Δ%	Δ Uncovered
projects/plugins/boost/app/lib/class-cli.php	0/75 (0.00%)	0.00%	6 💔

Full summary · PHP report · JS report

_{Coverage check overridden by

Coverage tests to be added later
Use to ignore the Code coverage requirement check when tests will be added in a follow-up PR
.}

- Add missing changelog entry for plugins/boost - Fix wp eval command in Gate 7: use array form to prevent whitespace splitting of PHP code argument - Deduplicate utility functions: import getDevDomain and executeDevWpCommand from full-stack-utils.ts in global setup - Guard do_action in CLI set_module_status() on actual status change, matching the DataSync path behavior - Move activateBoostModuleDev/deactivateBoostModuleDev onto FullStackUtils class as activateModule/deactivateModule methods

@param

- Fix missing `join` import in full-stack-global-setup.ts (blocker: ReferenceError at module load prevented all full-stack tests) - Add .state/ to Boost e2e .gitignore (auth storage state files) - Remove unused STORAGE_STATE_PATH export, add cross-reference comments between global setup and playwright.config.ts - Trim getDevDomain() regex result to handle \r\n line endings - Remove unused getShieldHealth() method (YAGNI, no retry logic) - Fix @param type: string → bool for $status in class-cli.php docblock - Add cross-reference comment for intentionally duplicated getDevDomain()

- Strip quotes from DEV_DOMAIN .env value in both getDevDomain() copies - Extract execDocker() and flushRedis() as standalone exported functions from full-stack-utils.ts; import in global setup (removes duplication) - Remove private execDocker class method (class delegates to standalone)

- Replace non-null assertion on BOOST_CLOUD_DIR with explicit guard and descriptive error message (blocker: opaque TypeError otherwise) - Parse last line of execDocker output in Gate 5 to handle Docker Compose stderr warnings before stdout - Log swallowed mock plugin deactivation errors for debugging - Update changelog to mention CLI do_action bug fix - Add comment explaining pageerror listener in full-stack fixture

- Fix Gate 5 HTTP status parsing: use regex to extract 3-digit code from mixed stdout+stderr (important: Docker Compose warnings broke pop()) - Fix changelog: use proper entry (Type: fixed) instead of Comment field so the CLI behavioral fix appears in release notes - Add warning message when CLI module activation is a no-op - Add comment explaining localhost:1982 vs boost-shield:1982 in Gate 3

Copilot

Pull request overview

This PR adds Playwright-based full-stack E2E coverage for Jetpack Boost’s “Boost Cloud” Critical CSS pipeline (dev WordPress → boost-cloud services → callback), and aligns WP-CLI module activation with the module-status update hooks used by the DataSync REST path.

Changes:

Added full-stack Playwright projects (conditional on BOOST_CLOUD_DIR), plus fixtures/utilities for Docker/WP-CLI/Redis operations and environment gating.
Added a Critical CSS smoke test that validates cloud generation end-to-end and checks frontend output.
Updated Boost WP-CLI module activation to trigger jetpack_boost_module_status_updated and allowlisted the cloud_css module for the CLI command.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
projects/plugins/boost/tests/e2e/specs/full-stack/critical-css.test.ts	Adds a full-stack Critical CSS generation + frontend presence smoke test.
projects/plugins/boost/tests/e2e/playwright.config.ts	Conditionally registers full-stack Playwright projects and ignores full-stack specs in existing project.
projects/plugins/boost/tests/e2e/lib/utils/full-stack-utils.ts	Introduces Docker-aware helpers (dev WP-CLI exec, Redis flush, CSS polling, log capture).
projects/plugins/boost/tests/e2e/lib/full-stack-global-setup.ts	Adds a setup project that gates environment readiness and saves auth storage state for dev WP.
projects/plugins/boost/tests/e2e/lib/fixtures/full-stack-test.ts	Adds a dedicated fixture for full-stack tests and attaches Docker logs on failure.
projects/plugins/boost/tests/e2e/.gitignore	Ignores `.state/` used for full-stack storage state.
projects/plugins/boost/changelog/add-boost-full-stack-e2e-infrastructure	Adds a changelog fragment describing the CLI hook fix + E2E infra.
projects/plugins/boost/app/lib/class-cli.php	Allowlists `cloud_css` and fires `jetpack_boost_module_status_updated` when CLI changes module state.

projects/plugins/boost/tests/e2e/playwright.config.ts

projects/plugins/boost/tests/e2e/lib/full-stack-global-setup.ts

projects/plugins/boost/tests/e2e/lib/fixtures/full-stack-test.ts

projects/plugins/boost/tests/e2e/playwright.config.ts

- Force workers: 1 on full-stack project to prevent race conditions on shared dev WordPress and Redis state - Use fileURLToPath for STORAGE_STATE_PATH (cross-platform correctness) - Verify authentication after login POST by requesting admin profile page and asserting no redirect to wp-login.php - Remove duplicate pageerror handler (already inherited from base-test) - Use path.join for .env path in config getDevDomain

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

projects/plugins/boost/tests/e2e/lib/fixtures/full-stack-test.ts

projects/plugins/boost/tests/e2e/lib/full-stack-global-setup.ts

projects/plugins/boost/tests/e2e/playwright.config.ts

fetch() had no timeout — if Shield accepts the connection but stalls, the gate hangs indefinitely. Use AbortSignal.timeout(5s) so each attempt fails fast and the retry loop behaves predictably.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

projects/plugins/boost/tests/e2e/lib/utils/full-stack-utils.ts

…vWpCommand

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

projects/plugins/boost/app/lib/class-cli.php

- Add explicit assertion when HTTP status regex fails to match, showing raw Docker output instead of opaque "HTTP NaN" message - Document that full-stack specs must use test.describe.serial due to shared Docker environment

github-actions bot added [Plugin] Boost A feature to speed up the site and improve performance. [Status] In Progress [Tests] Includes Tests E2E Tests labels Apr 6, 2026

github-actions bot added the [Status] Needs Author Reply We need more details from you. This label will be auto-added until the PR meets all requirements. label Apr 6, 2026

LiamSarsfield added 5 commits April 6, 2026 21:40

LiamSarsfield requested a review from Copilot April 7, 2026 08:50

Copilot started reviewing on behalf of LiamSarsfield April 7, 2026 08:51 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

LiamSarsfield requested a review from Copilot April 7, 2026 09:09

Copilot started reviewing on behalf of LiamSarsfield April 7, 2026 09:10 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

projects/plugins/boost/tests/e2e/lib/fixtures/full-stack-test.ts Show resolved Hide resolved

projects/plugins/boost/tests/e2e/lib/full-stack-global-setup.ts Show resolved Hide resolved

projects/plugins/boost/tests/e2e/playwright.config.ts Show resolved Hide resolved

Add timeout to Shield health check fetch in global setup

3aafdca

fetch() had no timeout — if Shield accepts the connection but stalls, the gate hangs indefinitely. Use AbortSignal.timeout(5s) so each attempt fails fast and the retry loop behaves predictably.

LiamSarsfield requested a review from Copilot April 7, 2026 09:25

Copilot started reviewing on behalf of LiamSarsfield April 7, 2026 09:26 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

projects/plugins/boost/tests/e2e/lib/utils/full-stack-utils.ts Show resolved Hide resolved

Add JSDoc warning about string form whitespace splitting in executeDe…

a989494

…vWpCommand

LiamSarsfield requested a review from Copilot April 7, 2026 09:35

Copilot started reviewing on behalf of LiamSarsfield April 7, 2026 09:36 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

projects/plugins/boost/app/lib/class-cli.php Show resolved Hide resolved

LiamSarsfield added 2 commits April 7, 2026 10:48

Update phan baseline for Boost CLI changes

49e2584

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add full-stack E2E test infrastructure for Boost Cloud pipeline#47961

Add full-stack E2E test infrastructure for Boost Cloud pipeline#47961
LiamSarsfield wants to merge 11 commits intotrunkfrom
add/boost-full-stack-e2e-infrastructure

LiamSarsfield commented Apr 6, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 6, 2026 •

edited

Loading

Uh oh!

jp-launch-control bot commented Apr 6, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

LiamSarsfield commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Other information

Related product discussion/links

Does this pull request change what data or activity we track or use?

Testing instructions

Prerequisites

Run full-stack tests

Verify existing tests are unaffected

Verify CLI do_action change

Uh oh!

github-actions bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jp-launch-control bot commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

LiamSarsfield commented Apr 6, 2026 •

edited

Loading

Verify CLI `do_action` change

github-actions bot commented Apr 6, 2026 •

edited

Loading

jp-launch-control bot commented Apr 6, 2026 •

edited

Loading