Skip to content

chore: migrate to utoo CI#5830

Draft
elrrrrrrr wants to merge 75 commits intonextfrom
chore-ut-ci
Draft

chore: migrate to utoo CI#5830
elrrrrrrr wants to merge 75 commits intonextfrom
chore-ut-ci

Conversation

@elrrrrrrr
Copy link
Copy Markdown
Contributor

@elrrrrrrr elrrrrrrr commented Mar 12, 2026

Summary

Migrate the main CI workflow (.github/workflows/ci.yml) from pnpm to utoo (ut). The E2E workflow (.github/workflows/e2e-test.yml) keeps pnpm for now — it requires catalog: / workspace: protocol resolution and publishConfig overrides during pack which ut pm-pack does not yet provide.

image

CI changes

  • .github/workflows/ci.yml: replace pnpm/action-setup + pnpm install + pnpm run with utooland/setup-utoo + ut install --from pnpm + ut run
  • package.json: convert pnpm-specific scripts (pnpm -r, pnpm --filter, pnpm exec) to ut equivalents (ut run --workspaces, ut run --workspace <name>)
  • tools/scripts/package.json: same conversion for cov / ci scripts
  • .gitignore: add .utoo.toml (ut local cache) and .claude/ (Claude Code session state)
  • .oxfmtrc.json: ignore root package.json because ut install --from pnpm rewrites it (adds an npm-style workspaces field) and the new field position trips oxfmt's expected ordering — this is a no-op for the source package.json since it's never directly hand-formatted

Source code adaptations

These are the only source/test changes required to make the migrated CI work. Each is justified below.

File What Why ut requires it
packages/utils/src/import.ts Replace the ESM/CJS branching in importResolve with a single getRequire().resolve(filepath, { paths }) call (-34 net lines) The original ESM branch called import.meta.resolve(filepath) without a parent URL, which silently ignored the caller's paths and resolved from @eggjs/utils's own context. That worked under pnpm's nested node_modules (each consumer had its own copies) but breaks under ut's flat hoisting. require.resolve correctly honors paths, walks up node_modules, reads exports, and auto-appends extensions for legacy CJS subpaths (e.g. tsconfig-paths/register). It fixes a latent bug at the same time.
packages/cluster/test/options.test.ts Add the flat-hoisted node_modules/egg path to the accepted framework paths (+2 lines) Under pnpm, egg is found at packages/cluster/node_modules/egg. Under ut flat hoisting, it's at the workspace root node_modules/egg. The test now accepts both.
packages/tsconfig/test/index.test.ts Use require.resolve('typescript/bin/tsc') instead of hardcoded path.join(__dirname, '..', 'node_modules', ...) The hardcoded path assumes typescript is nested under packages/tsconfig/node_modules/. Under flat hoisting it lives at the root. require.resolve works under both layouts and is strictly a portability improvement.
tegg/plugin/controller/src/lib/impl/mcp/MCPControllerRegister.ts Remove // @ts-expect-error content-type is not typed (1 line) Under ut's flat hoisting, @types/content-type is discoverable from this file's location, so the underlying type error no longer exists and @ts-expect-error itself becomes an unused-directive lint failure.
tegg/plugin/mcp-proxy/src/index.ts Remove two // @ts-expect-error directives for content-type and koa-compose (2 lines) Same as above.
tsdown.config.ts Add publint: { pack: 'npm' } (1 line) publint auto-detects the package manager and runs <pm> pack to validate. The ut CI environment has no pnpm binary in PATH, so without this flag publint runs pnpm pack and fails. Pinning to npm makes it portable.

Why E2E stays on pnpm

The E2E job packs every workspace package into a tgz so downstream projects (cnpmcore, examples) can install them as overrides. This requires three things during pack that ut pm-pack does not currently do:

  1. Apply publishConfig overrides (so exports points at dist/ instead of src/.ts)

Without those, the resulting tgz contains literal catalog: / workspace:* strings in dependencies and source-tree paths in exports, which break downstream npm install. pnpm -r pack handles all three natively. We will revisit once ut adds them.

Test plan

  • typecheck
  • Test scripts (ubuntu 22 + 24) ✅
  • Test bin (ubuntu + windows) ✅
  • Test matrix (ubuntu × 22/24, macos × 22/24, windows × 22/24) ✅
  • E2E Test (examples + cnpmcore) ✅

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 12, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: eb329485-cbfc-4ecd-b8be-45d5dee38115

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore-ut-ci

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the project's dependency and workspace management. It transitions from using pnpm-workspace.yaml to a combination of a new .utoo.toml file for dependency catalogs and direct configuration within package.json for workspaces and overrides. This change streamlines how dependencies are declared and managed across the monorepo, likely in preparation for or as part of a new CI setup.

Highlights

  • Configuration Migration: Migrated dependency catalog and workspace definitions from pnpm-workspace.yaml to .utoo.toml and package.json respectively.
  • New Configuration File: Introduced a new .utoo.toml file to centralize dependency version management.
  • Package.json Updates: Updated package.json to include workspace paths and define package overrides, specifically for vite.
  • Removed PNPM Workspace File: Removed the pnpm-workspace.yaml file as its configurations were moved to other files.
Changelog
  • .utoo.toml
    • Added a new file to define a comprehensive catalog of project dependencies and their versions.
  • package.json
    • Added a workspaces array to explicitly define the project's monorepo structure.
    • Introduced an overrides section to specify a custom version for the vite package.
    • Removed the packageManager field, indicating a shift in how the package manager is specified or managed.
  • pnpm-workspace.yaml
    • Removed the file, as its contents for workspace definition and dependency catalog have been migrated to package.json and .utoo.toml.
Ignored Files
  • Ignored by pattern: .github/workflows/** (2)
    • .github/workflows/ci.yml
    • .github/workflows/e2e-test.yml
Activity
  • No human activity has been recorded on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@socket-security
Copy link
Copy Markdown

socket-security bot commented Mar 12, 2026

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the project's dependency and workspace management by migrating configuration from pnpm-workspace.yaml to a new .utoo.toml file and package.json. My review focuses on the maintainability of the new configuration and the potential impact on the developer workflow. I've suggested sorting the new dependency catalog for better readability and pointed out a potential issue with the removal of the packageManager enforcement that could be addressed with documentation.

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Mar 12, 2026

Deploying egg-v3 with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1b13d61
Status:🚫  Build failed.

View logs

@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Mar 12, 2026

Deploying egg with  Cloudflare Pages  Cloudflare Pages

Latest commit: 1b13d61
Status:🚫  Build failed.

View logs

@elrrrrrrr elrrrrrrr changed the title chore(ci): utoo ci chore: migrate to utoo CI and fix circular dependencies Mar 16, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.39%. Comparing base (25d86b1) to head (1b13d61).

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #5830      +/-   ##
==========================================
- Coverage   85.88%   85.39%   -0.50%     
==========================================
  Files         666      651      -15     
  Lines       18841    18693     -148     
  Branches     3636     3627       -9     
==========================================
- Hits        16182    15962     -220     
- Misses       2296     2355      +59     
- Partials      363      376      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

elrrrrrrr and others added 10 commits March 30, 2026 17:13
- Replace pnpm/action-setup with setup-utoo in all CI jobs
- Use ut install --from pnpm for dependency installation
- Replace pnpm run/filter commands with ut run equivalents
- Use --workspaces --if-present for topological workspace execution
- Use --workspace <pkg> for targeted package execution
- Use -- passthrough for tsdown args (ut run build -- --workspace)
- Remove pnpm dedupe --check step (no longer needed)
- Fix tools/scripts ci script to use ut run cov
ut does not install optional peer deps automatically, so
unplugin-unused (required by tsdown's unused.level feature) must
be declared explicitly. Also includes workspaces/overrides fields
auto-resolved from pnpm config by ut install.
publint auto-detects pnpm via pnpm-lock.yaml and calls pnpm pack,
but pnpm is not on PATH when using setup-utoo. Set pack: 'npm'
explicitly so pnpm binary is not required for publint checks.
- Replace pnpm/action-setup with setup-utoo
- Use ut install --from pnpm for dependency installation
- Use ut run build for building all packages
- Replace pnpm -r pack with npm pack --workspaces
- Sync pnpm-lock.yaml to include unplugin-unused
npm pack --workspaces fails on packages without version (e.g. site).
Also, pnpm -r pack places tarballs in workspace root, while npm pack
places them in each package's own directory.

pack-all.mjs replicates pnpm -r pack behavior:
- reads workspace patterns from pnpm-workspace.yaml
- skips private/unnamed/unversioned packages
- packs each with --pack-destination to workspace root
oxfmt reordered imports and placed execSync import inside the JSDoc
comment block, making it unavailable at runtime.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
npm does not understand catalog: protocol; ut install handles it correctly.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…pack

npm pack does not resolve pnpm catalog: or workspace: protocol entries,
leaving them raw in the tgz package.json. Downstream npm install then
fails with EUNSUPPORTEDPROTOCOL. Pre-resolve these to actual semver
versions before packing, then restore the originals.

Also revert downstream test commands back to npm install/run since
cnpmcore/examples use plain semver and npm can install the cleaned tgzs.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
npm pack does not apply publishConfig.exports automatically. Packages
use devExports (src/) in exports and dist/ in publishConfig.exports.
Without merging publishConfig first, the tgz contains src/ exports and
downstream npm install fails to find the source files.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Use import.meta.resolve(specifier, parentUrl) with each caller-provided
path to avoid relying on package manager hoisting behavior. Falls back
to resolving from the current module context only after all provided
paths have been exhausted.

Also remove stale @ts-expect-error comments in tegg mcp-proxy and
controller plugins where content-type and koa-compose now ship types.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…in pack-all.mjs

Named catalogs (pnpm catalogs.<name>) were being looked up in the
default catalog instead of the named catalog section. This caused
@eggjs/router to be packed with path-to-regexp ^6.3.0 instead of
^1.9.0, breaking the Layer.js constructor which uses the old default
export API.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The utoo CI environment does not have pnpm in PATH, causing publint's
auto-detected `pnpm pack` command to fail during build.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
elrrrrrrr and others added 5 commits April 7, 2026 19:42
Under ut install the .oxfmtrc.json walk-up discovery seems to miss
the printWidth: 120 setting, causing root package.json line 34 (115
chars) to exceed the implicit default and fail formatting.  Pass
the config path explicitly.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
ut install --from pnpm rewrites package.json by appending a workspaces
field at the bottom (after packageManager).  This trips oxfmt because
oxfmt expects workspaces to come immediately after repository per npm
field-order convention.

Add a 'git checkout -- package.json' step right after every 'ut install'
to discard ut's rewrite and keep oxfmt happy.

Also revert the explicit -c .oxfmtrc.json on fmtcheck (no longer needed,
since the real cause was ut's rewrite, not config discovery).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The real issue was ut install rewriting package.json (fixed in previous
commit), not config discovery.  Restore the simpler fmtcheck command.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…install

ut install --from pnpm intentionally rewrites package.json to add the
npm-style workspaces field (that's its whole purpose).  Reverting it
defeats the migration tool.  Instead, ignore the root package.json in
.oxfmtrc.json so oxfmt does not enforce field ordering on it.

Other package.json files in workspaces are still checked.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@elrrrrrrr elrrrrrrr changed the title chore: migrate to utoo CI and fix circular dependencies chore: migrate to utoo CI Apr 7, 2026
elrrrrrrr and others added 17 commits April 7, 2026 22:12
ut now handles workspace package linking correctly, so the manual
symlink-creation step (added when ut copied workspace packages instead
of symlinking them) is no longer needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Try running CI without the require.resolve fallback.  If something
breaks (likely tsconfig-paths/register or similar CJS subpath imports),
this commit will be reverted.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
This PR focuses on migrating the main CI workflow to ut.  The E2E
workflow needs catalog: and workspace: protocol resolution which ut
does not provide yet (ut pm-pack ships unresolved protocols), and
pnpm -r pack handles this natively.  Keep pnpm for E2E and revisit
once ut adds publishConfig + protocol resolution.

Also remove ecosystem-ci/pack-all.mjs which was added solely as a
workaround for the pnpm-less E2E.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Node.js 24's ESM resolver returns extensionless paths for CJS packages
without an `exports` field, e.g. import.meta.resolve('tsconfig-paths/register')
returns '.../register' instead of '.../register.js'.  fs.statSync then
fails and importResolve throws.

Fall back to tryToResolveFromFile to probe common extensions
(.mjs/.js/.cjs/.ts) when stat says the resolved path is not a file.

This is a much smaller fix than the previous 70-line rewrite — it only
adds the extension-probe in the failing branch, leaving the rest of
importResolve unchanged.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
ut install --from pnpm reads pnpm-workspace.yaml which already declares
the vite → rolldown-vite override.  The duplicate npm-style overrides
in package.json is unnecessary.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ehavior

The previous fix only handled Node 25's behavior of returning an
extensionless path. Node 24 instead throws ERR_MODULE_NOT_FOUND from
import.meta.resolve for CJS subpaths without an exports field
(e.g. tsconfig-paths/register).

Add a require.resolve fallback in the catch branch as well, so both
Node 24 (throw) and Node 25 (extensionless return) cases are handled.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…solve

The previous ESM branch called import.meta.resolve(filepath) without
a parentURL, which has two problems:

1. It silently ignores the caller-supplied paths option, always
   resolving from this module's own context.
2. Node 24 throws ERR_MODULE_NOT_FOUND for CJS subpaths without an
   exports field (e.g. tsconfig-paths/register), and Node 25 returns
   the extensionless path which then fails fs.statSync.

require.resolve handles both cases correctly:
- honors caller-supplied paths
- walks up the node_modules chain via Node.js standard module
  resolution algorithm
- reads the exports field on modern Node.js
- auto-appends extensions for legacy CJS subpaths
- resolves .ts/.mjs files alike

Verified locally with all egg-bin importResolve targets.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
I removed nodeMajorVersion when removing supportImportMetaResolve,
but it's also used at line 66 for the ts enum support detection.
Restore the const.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ssage

require.resolve throws 'Cannot find module' (CJS resolver format)
while import.meta.resolve throws 'Cannot find package' (ESM resolver
format). After switching importResolve to use require.resolve, update
the test regex to accept either wording.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…r both Node 24/25

The previous refactor switched to require.resolve completely, which
broke cnpmcore E2E because import.meta.resolve goes through the
ts-node ESM loader hook (which maps .js → .ts), while require.resolve
goes through the CJS resolver chain (where ts-node/register doesn't
do the .js → .ts substitution).

Restore the import.meta.resolve as the primary path so the ts-node
loader hook still applies, and add require.resolve fallback only when
import.meta.resolve fails (Node 24 throws for CJS subpaths without
exports) or returns an extensionless path (Node 25 case for the same
type of CJS subpath).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The require.resolve fallback (Node 24 throw case) and extensionless
probing fallback (Node 25 case) were added as defensive code but are
not exercised by CI. cnpmcore E2E (Node 24 + ts-node ESM loader)
verified that import.meta.resolve alone is sufficient — failure
count remained 1 (unrelated EdgedriverBinary network flaky test)
with or without the fallbacks.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
elrrrrrrr and others added 4 commits April 8, 2026 18:27
The previous refactor (f170210) dropped this fallback assuming it
was unused, but egg-bin tests on Node 24 (both ubuntu and windows)
regressed because import.meta.resolve throws ERR_MODULE_NOT_FOUND
for `tsconfig-paths/register` (a CJS package without `exports`).

Restore the require.resolve fallback inside the catch block so
caller-supplied `paths` are honored and legacy CJS subpaths resolve
via Node's standard CJS resolver. The Node 25 extensionless probing
fallback (Fallback 2) remains removed since no test exercised it.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Expand the inline comment for the require.resolve fallback inside
import.meta.resolve's catch block. Mark it explicitly as "Fallback 1"
and explain the two combined reasons it is reached:

  1. Node 22+ ESM resolver requires explicit file extensions for
     subpaths of packages without an `exports` field.
  2. The manual node_modules walk only checks immediate children
     plus two pnpm sibling levels, not the full directory tree, so
     workspace-hoisted dependencies fall through to import.meta.resolve.

Also note the public-API constraint: removing this fallback would be
a breaking change for any `@eggjs/utils` consumer (internal or
downstream) that resolves a bare CJS subpath like
`tsconfig-paths/register`.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…ary fix

Picks up cnpm/cnpmcore#1026, which replaces the dead Azure Blob XML
listing API with directly-generated per-platform download URLs on
msedgedriver.microsoft.com. This unblocks the chore-ut-ci E2E run,
which has been failing on `EdgedriverBinary.test.ts > should work`
with `items.length >= 3` since 2026-04-08 04:19 UTC.

Old hash:  e82df3f4 (pre-fix)
New hash:  98463c33 (cnpmcore release v4.32.1, contains the merged PR)

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The Test bin job's "Run tests" step ran `ut run build` (no
`--workspace`), which built the entire monorepo (all packages, plugins,
tegg, tools — 30+ packages) before invoking egg-bin's tests. The
equivalent step on next runs `pnpm build --workspace ./tools/egg-bin`,
building only the package under test.

The full-monorepo build was the cause of:
  - Test bin (ubuntu-latest, 24): 134s -> 234s (+100s, +75%)
  - Test bin (windows-latest, 24): 189s -> 345s (+156s, +82%)

`ut run ci --workspace @eggjs/bin` already correctly limits the test
phase; the build phase was the only thing leaking outside the
@eggjs/bin scope.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants