π Add clay vite β Vite ESM bundling pipeline#239
Merged
Conversation
## Why this change exists
The legacy `clay compile` command uses Browserify (via megabundler) to produce
browser bundles. Browserify generates a runtime `window.modules` registry and
synchronous `window.require()` function. This architecture has two problems:
1. The output is CommonJS/IIFE β not native ES Modules β so browsers cannot use
<script type="module"> loading, import() splitting, or tree-shaking.
2. Several dependencies (`event-stream`, `glob@7`, `kew`) are unmaintained and
incompatible with Node 20.
This PR adds a new `clay pack-next` command backed by esbuild and updates the
`clay compile` sub-commands to be Node 20-compatible.
---
## New: `cli/pack-next.js` + `lib/cmd/pack-next/`
### `build.js` β esbuild configuration and entry-point discovery
- Discovers all `components/*/client.js`, `components/*/model.js`,
`layouts/*/client.js`, and any path returned by `packNextConfig.extraEntries`.
- Sets `format: 'esm'`, `splitting: true`, `chunkNames: 'chunks/chunk-[hash]'`
so every entry gets a hashed ES module with shared code-split into chunks.
- Writes a `public/js/_manifest.json` that maps logical entry keys
(e.g. `components/article/client`) to their hashed output path plus the
list of transitively-imported chunks. `resolveMedia.js` consumes this to
inject exactly the right `<script type="module">` tags per page.
- **Kiln edit-mode aggregator**: before building, `generateKilnEditEntry()`
scans the repo for every `model.js` and `kiln.js` file and writes
`.clay/_kiln-edit-init.js`. This generated file imports all of them and
explicitly assigns the exports to `window.kiln.componentModels`,
`window.kiln.componentKilnjs`, and calls the site's `services/kiln/index.js`
plugin initialiser. This completely replaces the Browserify `window.modules`
registry for edit-mode without changing any component source files.
- Reads a `packNextConfig(config)` hook from `claycli.config.js` in the
project root, allowing per-project esbuild overrides (aliases, defines,
plugins, inject files, etc.).
### `get-script-dependencies.js` β manifest-based script lookup
- `getDependenciesNextForComponents(components, assetPath, globalKeys)`:
given a list of Clay component instance names (from `locals._components`)
returns the deduplicated set of hashed JS URLs (entry + chunks) needed to
run those components on a page β plus any always-loaded global entries.
- `getEditScripts(assetPath)`: returns the hashed URLs for the kiln edit-mode
aggregator bundle (`_kiln-edit-init`) and all its shared chunks.
### `plugins/service-rewrite.js` β serverβclient service redirect
An esbuild `onResolve` plugin that mirrors the Browserify
`rewriteServiceRequire` transform: any import whose path contains
`services/server` is redirected to the matching `services/client` path.
This prevents server-only code (database drivers, Redis clients, etc.)
from ever being bundled into browser output.
### `plugins/vue2.js` β Vue 2 SFC compiler plugin
Compiles `.vue` files (Vue 2 Single-File Components) at bundle time:
- Parses the SFC with `@vue/component-compiler-utils` +
`vue-template-compiler`.
- Extracts the `<script>` block; strips `export default` / `module.exports =`
and reassigns to a local `__sfc__` variable so render functions can be
injected.
- Compiles the `<template>` block to `{ render, staticRenderFns }` and
attaches them to `__sfc__`.
- **CJS/ESM interop fix**: if the original `<script>` block used
`module.exports = {}` (CommonJS style), the plugin emits
`module.exports = __sfc__; module.exports.default = __sfc__`
so that `require('./foo.vue')` returns the component object directly.
If the original used `export default {}` (ESM style), the plugin emits
`export default __sfc__`. Without this distinction esbuild's interop
wraps the component in `{ default: component }` and Vue's runtime-only
build reports "template or render function not defined" because `render`
is not at the top level of the object.
- Injects scoped-style `data-v-XXXX` attributes when `<style scoped>` is
present.
- Inlines `<style>` blocks as `document.createElement('style')` side-effects.
---
## `clay compile` Node 20 compatibility fixes
All five compile sub-commands (`fonts`, `media`, `scripts`, `styles`,
`templates`) used `event-stream` for stream manipulation. `event-stream` is
abandoned, has a known supply-chain vulnerability history, and emits Node 20
deprecation warnings. It has been replaced with `through2` (object-mode
transform streams) and `merge-stream` (stream merging), both actively
maintained.
Additional Node 20 fixes:
- `new Buffer(...)` β `Buffer.from(..., 'utf8')` throughout (the `Buffer()`
constructor has been deprecated since Node 10 and removed in Node 22).
- `glob` upgraded from v7 to v10; synchronous calls updated to use
the named `globSync` export instead of `glob.sync`.
- `kew` (unmaintained Promise library) removed; usages replaced with native
Promises.
### `gulp-plugins/gulp-newer/index.js`
The bundled `gulp-newer` plugin was rewritten from a prototype-chain class to
an ES6 `class extends Transform`. The key behaviour fix: the original code
called `fs.promises.stat()` on the destination file in the constructor and
let the resulting Promise float. On Node 20, unhandled promise rejections from
`ENOENT` (destination file does not exist yet on a clean build) become fatal
errors. The rewrite catches `ENOENT` in the stat call and resolves to `null`
so the plugin correctly falls through to copying the file.
---
## Dependency changes (`package.json`)
Added:
- `esbuild` ^0.27 β the bundler
- `@vue/component-compiler-utils` ^3.3 β Vue 2 SFC parsing/compilation
- `through2` ^4 β replaces event-stream for object-mode stream transforms
- `merge-stream` β replaces es.merge() for fanning-in multiple gulp streams
- `glob` upgraded to ^10 (breaking: use globSync named export)
- `autoprefixer` upgraded from ^9 to ^10 (PostCSS 8 compatible)
Removed:
- `event-stream` β abandoned, replaced by through2/merge-stream
- `kew` β abandoned Promise library
- `isomorphic-fetch` β replaced by native fetch (Node 18+)
Dev dependencies:
- `jest` upgraded from ^24 to ^29 (Node 20 compatible)
- `jest-fetch-mock`, `jest-mock-console` updated to match
Made-with: Cursor
β¦ steps Introduces five new build steps that bring the full legacy `clay compile` asset pipeline into `clay pack-next`: - **styles.js** β PostCSS 8 pipeline: `postcss-import` resolves `@import` chains across styleguide directories, `autoprefixer` adds vendor prefixes, and `cssnano` minifies in production. Writes per-styleguide CSS files to `public/css/`. - **fonts.js** β copies `global/fonts/**` and styleguide font files to `public/fonts/`, and writes `_linked-fonts.<styleguide>.css` files. - **templates.js** β compiles every `*.template.handlebars` file to a JS module that writes `window.kiln.componentTemplates[name]`, bucketed into `_templates-a-d.js` etc. in production. - **vendor.js** β copies `global/js/vendor/**` to `public/js/vendor/`. - **media.js** β copies `global/media/**` to `public/media/`. Adds `cssnano` and `p-limit` to dependencies; `p-limit` bounds the PostCSS concurrency to avoid memory pressure when processing large styleguide sets. Made-with: Cursor
The vue2 esbuild plugin now accumulates raw CSS from every `<style>` block it
processes across all `.vue` SFCs and writes them into
`public/css/_kiln-plugins.css` via an `onEnd` hook.
Previously the plugin injected styles as `document.createElement('style')`
runtime side-effects, which caused a Flash Of Unstyled Content in Kiln edit
mode and made the CSS invisible to server-side rendering. Writing a single
concatenated CSS file lets the page load it as a normal `<link>` tag.
Made-with: Cursor
β¦ noise esbuild reports warnings for every file it touches on each incremental rebuild, not just the file that changed. In `make watch` this floods the terminal with hundreds of irrelevant lint-style messages on every keystroke. Watch mode now surfaces only errors. Full warning output remains visible during `clay pack-next` (one-shot build / `make compile`). Made-with: Cursor
β¦ate _view-init.js
build.js β orchestration and watch mode
- `buildAll()` now calls all asset steps (styles, fonts, templates, vendor,
media) in parallel alongside the esbuild JS step.
- `watch()` uses chokidar (polling, for Docker + macOS host volumes) to watch
every file type individually. Each watcher debounces rebuilds and logs which
file changed (`Changed:`) and what was rebuilt (`Rebuilt:`) in distinct
colours.
- CSS watch rebuilds all variation files sharing a component basename
(e.g. editing `text-list.css` also rebuilds `text-list_amp.css`).
- `global/js/*.js` added to ENTRY_GLOBS so global scripts (ads, cid, facebook,
aaa-module-mounting) compile as independent manifest entries β no hand-rolled
init file required to pull them in.
- Non-existent `extraEntries` (e.g. the webpack-era `components/init.js` still
referenced in older `claycli.config.js` files) are silently skipped.
generateViewInitEntry() β _view-init.js generator
- Before every build and on every JS file add/remove in watch mode, claycli
writes `.clay/_view-init.js` containing:
1. A sticky custom-event shim: if `auth:init` (or any listed event) has
already fired when a late `window.addEventListener` subscriber registers,
the handler receives an immediate replay in the next microtask. This
restores the Browserify synchronous-bundle guarantee without touching
`auth.js` or any `client.js` file.
2. An explicit component module map (all `components/**/client.js` entries)
that esbuild resolves to content-hashed paths at build time.
3. Per-element component mounting that supports both patterns:
- Function-export: `mod(element)` is called directly.
- Dollar-Slice controller: `DS.get(name, element)` instantiates the
controller after `aaa-module-mounting.js` has registered globals.
- The consuming repo no longer needs to maintain `components/init.js`.
get-script-dependencies.js β auto-inject _view-init
- `getDependenciesNextForComponents()` now automatically prepends the generated
`_view-init` scripts before any caller-supplied GLOBAL_KEYS. The consuming
`resolveMedia.js` does not need to know the generated key.
Made-with: Cursor
Removes the need for pack-next-inject.js (a webpack ProvidePlugin clone)
from consuming repos entirely.
Previously, consuming repos had to ship a pack-next-inject.js file and wire
it up via config.inject in packNextConfig. This was a webpack-era pattern
that has no place in the esbuild pipeline.
Two cleaner alternatives used instead:
**DS / Eventify / Fingerprint2**
aaa-module-mounting.js already sets window.DS, window.Eventify, and
window.Fingerprint2 synchronously before any component module code runs.
esbuild define entries now map the free variable references (DS, Eventify,
Fingerprint2) to their window-registered equivalents at compile time.
No runtime behaviour change; the inject file is no longer needed.
**process.*
esbuild define entries cover every common pattern:
process.env.NODE_ENV β respects NODE_ENV env var (defaults to 'development')
process.env β { NODE_ENV: ... }
process.browser β true
process.version β ""
process.versions β {}
global, __filename, __dirname, and mainFields are also promoted to claycli
defaults so consuming repos don't need to repeat them in packNextConfig.
Made-with: Cursor
β¦cosystem browser compat
Moves all Clay/Node.js compatibility stubs out of sites' claycli.config.js
and into claycli itself so no consuming repo needs to maintain this boilerplate:
browserCompatPlugin (new):
- Stubs clay-log and services/universal/log.js (Node-only transports)
- Stubs Clay server-only packages (amphora-search, amphora-storage-postgres, etc.)
- Stubs all Node.js built-in modules (fs, crypto, net, path, etc.)
- Provides rich stubs for events, stream, util, buffer (needed for util.inherits chains)
- Provides http/https stubs with agent-base patch flag pre-set
serviceRewritePlugin (enhanced):
- Case 1 (existing): rewrites imports whose raw string contains 'services/server'
- Case 2 (new): catches relative '../server/' imports from services/universal/
whose raw path doesn't contain 'services/server' but whose resolved absolute
path lands inside services/server/ β e.g. require('../server/db') in utils.js
Made-with: Cursor
β¦m pack-next to build
File/directory renames:
- lib/cmd/pack-next/ β lib/cmd/build/
- cli/pack-next.js β cli/build.js
CLI changes:
- Command is now 'clay build' (exports.command = 'build')
- 'clay pack-next' and 'pn' kept as backward-compat aliases so existing
Makefiles don't break before they are updated
- New short alias 'b' added (clay b)
- cli/index.js routing updated: b/pn/pack-next all resolve to cli/build.js
Internal changes:
- browserCompatPlugin and serviceRewritePlugin registered in getEsbuildConfig()
- Config hook key renamed from packNextConfig β esbuildConfig in claycli.config.js
- All user-facing error messages and comments updated from 'clay pack-next' to 'clay build'
- Auto-generated _view-init.js header updated accordingly
Made-with: Cursor
The >=22 constraint was unnecessarily strict. The only API that could have justified it is fetch(), which has been stable and unflagged since Node 18. No Node 22-specific APIs are used anywhere in claycli. Sites running Node 20 LTS do not need to upgrade their runtime just to use clay build. Made-with: Cursor
- buildAll() now prints a permanent done line for each step as it finishes, with an animated single-line spinner showing in-progress steps and their live completion percentage (e.g. β Έ [styles 45%] [templates 67%] (18s)) - buildStyles and buildTemplates accept an onProgress callback so the progress bar shows real done/total counts rather than a spinner alone - buildStyles accepts an onError callback so CSS compile errors are routed through the display manager instead of stderr, preventing them from corrupting the progress line - cli/build.js suppresses verbose per-warning logs in build mode; shows a single summary count instead (e.g. "21 esbuild warning(s)") - Add *.tgz to .gitignore Made-with: Cursor
Tests (39 passing): - manifest.test.js: writeManifest β entry keys, chunk skipping, public URL mapping - styles.test.js: buildStyles β changedFiles, onProgress, onError routing - templates.test.js: buildTemplates β HBS precompile, progress, watch-mode error resilience, minified buckets - media.test.js: copyMedia β component + layout media copy - get-script-dependencies.test.js: hasManifest, getDependenciesNextForComponents β dedup, _view-init ordering Documentation (CLAY-BUILD.md): - Full pipeline comparison: Browserify+Gulp vs esbuild+PostCSS 8 - ASCII architecture diagram and step-by-step pipeline comparison table - Per-feature comparison (JS, CSS, templates, fonts, script resolution) - Performance numbers: full build 2-4x faster, watch JS rebuild 60-200x faster - Audience-specific sections: developer, SRE, product manager - Configuration guide, migration guide, code references with file links Also: update jest collectCoverageFrom to include lib/cmd/build/ modules Made-with: Cursor
Two color-coded flowcharts (one per pipeline) replacing the hard-to-read side-by-side ASCII box. Nodes are color-coded by speed (red=slow, green=fast, amber=medium). Followed by clean tables for shared outputs and key differences. Made-with: Cursor
Replace two separate top-down flowcharts with one left-to-right diagram containing both pipelines as labeled subgraphs. Source and output nodes flank both lanes, making the parallel-vs-sequential contrast immediately visible. Adds a summary table directly below the diagram. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
β¦ll glob + ctime filter, not single-file rebuild Made-with: Cursor
β¦ion table Made-with: Cursor
β¦b vitals/SEO/analytics - Correct ~0.5s CSS watch claim to a realistic ~1β3s in both the performance table and the PM section - Expand PM section: code splitting payload reduction, Core Web Vitals (LCP, INP) explanation, SEO ranking signal context, and analytics impact (bounce rate, session depth, conversion) Made-with: Cursor
β¦mponents on every page clay compile getDependencies() in view mode was page-specific β it walked _registry.json for only the components amphora placed on the page. Correct all instances that claimed otherwise and accurately describe the real differences: no shared chunk deduplication, and _client-init.js mounting loaded modules without DOM-presence checks. Made-with: Cursor
β¦tions Errors fixed: - _modules-a-d.js β actual output is per-component files + _deps-a-d.js buckets - @nymag/vueify esbuild plugin β custom plugin using vue-template-compiler directly - Vue top-level side-effects claim scoped correctly - Globals section: standard Clay globals already handled in claycli defaults New content added: - Source maps (new generates them, old did not) in Why, JS table, SRE, Dev sections - Content-hashed filenames and CDN cache efficiency throughout - Tree shaking (esbuild) vs no tree shaking (Browserify) - Build-time process.env.NODE_ENV β dead code elimination in minified builds - _prelude.js/_postlude.js custom runtime overhead called out explicitly - browserify-cache.json stale-build risk and rollback safety - Template process.exit(1) vs graceful error continuation - npm dependency reduction table (~20 removed, ~5 added) - CI compute cost reduction (~63%) in PM and SRE sections - CDN cache efficiency and operational confidence in PM section - Rollback safety (manifest atomicity) in SRE section Made-with: Cursor
jjpaulino
changed the title
jjpaulino
changed the title
jjpaulino
changed the title
β¦ jordan/yolo-update
After rebuilding templates in watch mode, write a timestamp to .clay/reload-signal. The app server (start.js) watches this single file with fs.watchFile so nodemon restarts immediately instead of polling thousands of bind-mounted files via legacyWatch. Made-with: Cursor
After each CSS rebuild in watch mode, write a timestamp to .clay/css-build-id. resolve-media.js reads this per-request and appends ?v=<id> to all CSS URLs so the browser fetches fresh styles on a normal page refresh β no server restart required. Made-with: Cursor
Write .clay/reload-signal after CSS rebuilds, same as templates. Reverts the css-build-id cache-busting approach β consistent behavior: any rebuild (CSS or template) triggers a server restart via start.js. Made-with: Cursor
Replace the full vite.build() on every kiln.js/model.js change with a parallel Rollup incremental watcher (same pattern as the view watcher). - captureKilnOutputPlugin logs which file changed and captures output - kiln BUNDLE_END updates kilnOutput and rewrites the manifest - add/unlink events still regenerate the entry file; Rollup picks up the change automatically from there - lastViewOutput tracked so kiln-triggered manifest writes have data Made-with: Cursor
- transform(): drop unused _code/_id params (no-unused-vars) - Extract handleKilnBundleEnd() to reduce kilnWatcher handler complexity from 9 to β€8 (complexity rule) - Extract regenerateEntryFiles() to reduce rebuildBootstrap complexity from 11 to β€8 - Auto-fix newline-after-var spacing (eslint --fix) All 402 tests pass, 0 lint errors. Made-with: Cursor
Made-with: Cursor
[js] and [kiln] use event.duration from Rollup's BUNDLE_END event. [styles], [fonts], and [templates] capture Date.now() before the build and diff it on completion. Output now reads e.g.: [js] Rebuilt successfully (3 modules transformed in 412ms) [styles] Rebuilt (287ms) [templates] Rebuilt (94ms) Made-with: Cursor
Prevents the terminal from looking frozen on slow rebuilds (kiln, full CSS, etc). A no-op stopTick replaces null guards to keep complexity within the eslint limit. The tick is skipped for the very first cold build. Made-with: Cursor
[js] β bright cyan [kiln] β magenta [styles] β blue [fonts] β bright yellow [templates] β bright green The prefix tag always uses the asset color; "Rebuilt" stays green and errors stay red so state is still conveyed independently of identity. Made-with: Cursor
CSS changes still write .clay/reload-signal β full nodemon restart, which gives the browser a fresh ETag for the stable CSS URL. Template changes now write .clay/template-signal β html.init() in-process (~100ms), no restart needed. Previously both used reload-signal which broke CSS visibility when we replaced nodemon.restart() with html.init(). Made-with: Cursor
β¦gration CSS uses a full nodemon restart (reload-signal) because output filenames are stable and the browser caches them. Once we move to Lightning CSS with content-hashed filenames the signal and restart can be removed entirely, matching how JS chunks already work. Made-with: Cursor
Made-with: Cursor # Conflicts: # lib/prefixes.js # lib/prefixes.test.js
Resolve remaining ESLint warnings and one test style error in Vite command files so claycli test runs cleanly after merging master. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Create public/css before vueify extract-css writes _kiln-plugins.css so clean Docker builds no longer fail with ENOENT during clay compile scripts. Made-with: Cursor
When CLAYCLI_COMPILE_SCRIPTS_SKIP_STYLE_POSTCSS=true, compile vueify sass/scss blocks with the sass package so Browserify builds no longer fail on missing node-sass in staged rollout environments. Made-with: Cursor
Vite externalizes any unresolved Node-only import with a Proxy that throws on the first property access, breaking module init for isomorphic code that Browserify (and raw Rollup before Vite) compiled fine because both left the same imports as silent undefined. - browser-compat plugin now reads the importer's enclosing package.json "browser" field and redirects `false` mappings (e.g. postcss's `./lib/terminal-highlight: false`) to an empty ESM stub, matching Browserify's `browser-resolve` behaviour. Results cached per package.json. - New opt-in `config.lenientBrowserExternalize` flag intercepts Vite's own `__vite-browser-external` virtual at load time and serves an empty ESM module instead of the throwing proxy. Use as a migration safety net while tracking down latent server-only imports in isomorphic code. Default stays off so real problems surface on new sites. Made-with: Cursor
Extract readBrowserEntry, matchesRelativeFalseEntry, isLenientExternalLoad, and loadVirtualStub as focused helpers. Hoist loadRichStub to module scope (it's pure). Fix the one JSDoc-union warning. No behaviour change; all 403 claycli tests pass. Made-with: Cursor
The lockfile was left pointing at the old clay/vueify git url (resolving to 9.4.5) while package.json moved to ^10.0.0-0 in commit fd75017, which blew up `npm ci` in CI with "lock file's @nymag/vueify@9.4.5 does not satisfy @nymag/vueify@10.0.0-0". Regenerated via `npm install --package-lock-only`. `npm ci`, eslint, and the full jest suite all pass. Made-with: Cursor
β¦crashes
Clay-kiln's preloader (lib/preloader/actions.js) calls
`Object.keys(window.modules)` to enumerate Browserify-registered model.js
and kiln.js modules. Under the Vite pipeline there is no Browserify runtime
and nothing populates window.modules before clay-kiln's DOMContentLoaded
handler fires, so editors were seeing:
TypeError: Cannot convert undefined or null to object
at Object.keys (<anonymous>)
at getComponentModels (clay-kiln-edit.js:...)
Previously this was patched in clay-kiln itself (guarded `window.modules || {}`).
That forced sites to pin clay-kiln to a forked branch just to run on Vite.
Putting the stub in the Vite bootstrap β which runs synchronously on every
Vite page before any dynamic import β makes the behaviour transparent to
clay-kiln: its existing `Object.keys(window.modules)` short-circuits into a
harmless `Object.keys({})` and it then falls through to the already-populated
`window.kiln.componentModels` / `componentKilnjs` maps written by
vite-kiln-edit-init.js.
Net effect: sites can drop the clay-kiln branch pin and go back to the
published clay-kiln release. No clay-kiln changes required to run on Vite.
Made-with: Cursor
The Vite vue2 plugin was injecting raw <style> source into the bundle without running PostCSS, so nested `&-foo` selectors, `@mixin`, and simple variables reached the browser as invalid CSS that no UA could parse. Kiln plugin modals built from local .vue files (article-picker, agora, mediaplay-picker, β¦) consequently rendered unstyled. The legacy @nymag/vueify Browserify pipeline runs the chain postcss-import β autoprefixer β postcss-mixins β postcss-nested β postcss-simple-vars over every Vue <style> block. This commit mirrors that chain in the Vite plugin, resolving each PostCSS plugin from the host project's node_modules so versions stay aligned with the host's CSS pipeline (e.g. PostCSS 7 in nymag/sites). Resolution is best-effort: any plugin missing from the host project is silently dropped and the chain continues with whatever resolved. If PostCSS itself is missing, the original source is passed through unchanged β no worse than the pre-fix behaviour. Errors during processing emit a Rollup warning rather than failing the build. Verified locally on nymag/sites: public/css/_kiln-plugins.css now contains zero raw `&-` selectors (was 180), and the agora / article-picker modal styles render correctly. Made-with: Cursor
CLAYCLI_COMPILE_SCRIPTS_SKIP_STYLE_POSTCSS was treating two unrelated
concerns as a single switch:
1. The PostCSS plugin chain (cssImport β autoprefixer β mixins β
nested β simple-vars), needed for every <style> block to flatten
`&-foo` nesting and resolve simple variables.
2. The sass/scss preprocessor (node-sass β dart-sass), introduced to
dodge node-sass native-binding failures in some Docker stages.
When the env var was set, the chain in (1) was disabled too, so kiln
plugin <style> blocks reached the browser as nested CSS no UA can
parse. Modals built from local .vue files (article-picker, agora,
mediaplay-picker, β¦) consequently rendered unstyled β even on sites
served by the existing Browserify pipeline.
This commit decouples them: the PostCSS chain now always runs; the env
var only swaps in the dart-sass preprocessor.
Plugins are resolved from the host project's node_modules first so the
chain runs against the host's PostCSS major. If the host has no PostCSS
installed at all, claycli's bundled copies are used instead. Mixing the
two trees is avoided β picking individual plugins across versions
triggers "PostCSS plugin X requires PostCSS Y" failures because each
plugin's `Symbol.for('postcss')` is anchored to whichever postcss it
was installed alongside. Plugins absent from the chosen tree are
dropped silently (postcss-mixins is a transitive of vueify only and may
legitimately be absent from a host).
Verified locally on nymag/sites: clay compile scripts emits a 65 KB
_kiln-plugins.css with zero raw `&-` selectors (was 56 KB / 180 nested
matches), and the agora / article-picker modals render correctly.
Companion to the equivalent fix in lib/cmd/vite/plugins/vue2.js so the
two pipelines produce equivalent output for kiln plugin SFCs.
Made-with: Cursor
Made-with: Cursor # Conflicts: # .github/scripts/deploy-docs.sh # .github/scripts/release.sh # .github/workflows/ci.yml # lib/cmd/import.test.js # package.json
|
coverage: 64.054% (-28.0%) from 92.014% β jordan/yolo-update into master |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
clay vitecommand (lib/cmd/vite/) that runs alongside the legacyclay compileBrowserify pipeline, producing native ES module bundles with content-hashed filenames, code splitting, and PostCSS 8 styling. The two pipelines are designed to coexist during a phased rollout β sites can opt in viaCLAYCLI_VITE_ENABLED/CLAYCLI_VITE_SITESwithout changing any application code, and consumer projects can keep building withclay compileuntil they're ready to flip the switch.This PR also patches several long-standing issues in the existing Browserify pipeline that surfaced during integration testing, so the legacy path is materially better even before any site enables Vite.
What's new
clay vitecommand (lib/cmd/vite/)scripts.jsβ Vite build with esbuild minification. Emits content-hashed chunks plus a_manifest.jsonmapping entry keys to filenames + their static imports.generate-bootstrap.jsβ emits.clay/vite-bootstrap.js, a single ESM entry loaded via<script type="module">. Scans the DOM fordata-uriattributes and dynamically imports the matching component chunk on demand. Now also stubswindow.modulessynchronously at the top of the bundle so clay-kiln's preloader (Object.keys(window.modules)) doesn't crash before any component loads.generate-kiln-edit.jsβ emits.clay/vite-kiln-edit-init.js, the edit-mode aggregator that pulls in every model.js / kiln.js so kiln has all schemas at edit time.generate-globals-init.jsβ emits.clay/_globals-init.jssowindow.DS,window.Eventify, and friends are available before any component mounts.styles.jsβ compiles styleguides via PostCSS 8 nested under claycli's ownnode_modules, isolated from the host's CSS toolchain.fonts.js,templates.js,media.jsβ feature parity withclay compilefor non-JS assets.Plugins (
lib/cmd/vite/plugins/)browser-compatβ stubs Node-only modules and rewritesprocess.env.*βwindow.process.env.*. Honours thelenientBrowserExternalizeflag (described below) so projects mid-migration aren't blocked on a single transitive Node-only import.client-envβ generatesclient-env.jsonfrom whitelistedprocess.envvalues at build time via Rollup transform hook (was a ~30 s grep scan in the legacy pipeline).manual-chunksβ groups vendor and component chunks to keep per-page payloads small under CJS-mode builds; falls back to Rollup's nativeexperimentalMinChunkSizeonceclientFilesESM:true.missing-moduleβ silences optional server-onlyrequire()calls that Vite would otherwise error on.service-rewriteβ redirectsservices/universal/*imports toservices/client/*equivalents in browser bundles.vue2β Vite-compatible Vue 2 SFC transform replacing@nymag/vueify. Now runs the host project's PostCSS chain (cssImport β autoprefixer β mixins β nested β simple-vars) over every<style>block so kiln plugin modals (article-picker, agora, mediaplay-picker, etc.) render with the correct styles instead of raw nested PostCSS source.Sticky events (
stickyEventsconfig key +.clay/vite-bootstrap.jsshim)Patches
window.addEventListenerso late subscribers are immediately replayed via a microtask if the event has already fired. Required because under ESM dynamic imports the authauth:initevent can fire before listeners register, where the synchronous Browserify bundle never had this race. Listed events are configurable per project; long-term path is to migrate consumers toauth.onReady()and remove the shim.lenientBrowserExternalizeflagVite wraps every unresolved Node-only import in a Proxy that throws on first property access β strictly correct, but blocks bundles on any latent isomorphic
require('cheerio')etc. Setting this flag inclaycli.config.jsmakes claycli intercept Vite's__vite-browser-externalvirtual and return an empty ESM module instead, restoring the silent-undefined behaviour Browserify and raw Rollup had. Use it during migration; clean up component-by-component.clay compileimprovements (legacy Browserify path)These changes ship inside this PR because they're prerequisites for running both pipelines from the same claycli version, but they benefit Browserify-only sites too:
scripts.jsβCLAYCLI_COMPILE_SCRIPTS_SKIP_STYLE_POSTCSSno longer disables the Vue SFC PostCSS chain. The env var was conflating two unrelated knobs (the PostCSS chain and the node-sass β dart-sass swap); now it only controls the sass swap, and the PostCSS chain always runs. Plugins resolve from the host'snode_modulesfirst so PostCSS 7 hosts (sites) don't pull claycli's PostCSS 8 chain by accident.get-script-dependencies.jsβ hardened against missing_registry.jsonat startup.fonts.js,media.js,styles.js,templates.js.CI
Picks up master's CircleCI β GitHub Actions migration (#240) via merge:
.github/workflows/ci.ymlβ Node 18 / 20 / 22 matrix, concurrency control, coveralls upload on Node 20.deploy-docsjob β auto-deploys docs on master push and onstgtag.deploy-packagejob β auto-publishes to npm when av*.*.*tag is pushed (uses--tag=prereleasefor hyphenated versions).releasescript inpackage.jsonnow points at.github/scripts/release.sh.Docs
CLAY-VITE.mdβ architecture overview, plugin reference, manifest format, sticky-events explanation, performance numbers.BUNDLER-COMPARISON.mdβ measured comparison between Browserify, esbuild (clay build), and Vite (clay vite).Test plan
npm run lintcleannpm testβ 403/403 passingnymag/sitesrunning both pipelines locally:clay compile scriptsproduces flat CSS in_kiln-plugins.css(0 raw&-selectors, was 180 before the PostCSS fix)clay viteproduces a complete_manifest.jsonand.clay/vite-bootstrap.jsclay-kiln-edit.js) loads withoutObject.keys(window.modules)crash-/_in their hash now correctly receive immutable cache headers downstream insites/app.jsnymag/sites/jordan/staging-ready-rollup) builds and renders against this branch