Generate boxel-ui component Specs for AI agent discovery (CS-10527) by jurgenwerk · Pull Request #4809 · cardstack/boxel

jurgenwerk · 2026-05-13T12:50:35Z

Summary

Auto-generate one Spec card per @cardstack/boxel-ui component into the catalog so the software-factory agent can discover and reuse UI primitives via realm search instead of hand-rolling HTML.
The agent-side behavior is opt-in via --enable-boxel-ui-discovery on the factory CLI. Without the flag, the agent has zero awareness of boxel-ui components (no skill loaded, no system-prompt exception) and the factory runs exactly as it did before this PR. With the flag, the discovery skill + catalog-search exception are added to the system prompt.
Resolves CS-10527.

Pieces

What	Where
Generator (parses each `usage.gts`, emits Spec JSON with keyword-rich `cardDescription` and a structured `readMe`)	`packages/boxel-ui/addon/bin/generate-component-specs.mjs`
`pnpm generate:component-specs` script	`packages/boxel-ui/addon/package.json`
CodeRef prefix mapping `@cardstack/boxel-ui/` → `https://packages/@cardstack/boxel-ui/` so spec `ref.module` resolves	`packages/host/app/lib/externals.ts`
New factory skill (mandatory rule, enumerate-first, catalog query, self-audit, fallback path) — gated on the feature flag	`packages/software-factory/.agents/skills/boxel-ui-component-discovery/SKILL.md`
`--enable-boxel-ui-discovery` CLI flag, plumbed through `FactoryEntrypointOptions` → `IssueLoopWiringConfig` → `DefaultSkillResolver` constructor + `ContextBuilder` → `AgentContext.enableBoxelUiDiscovery` → template variable	`factory-entrypoint.ts`, `factory-issue-loop-wiring.ts`, `factory-skill-loader.ts`, `factory-context-builder.ts`, `factory-agent/{types,claude-code,opencode}.ts`
System prompt: `{{#if enableBoxelUiDiscovery}}` wraps the catalog-search exception and `Catalog realm: {{catalogRealm}}` line — both disappear cleanly when the flag is off	`packages/software-factory/prompts/system.md`
`{{catalogRealm}}` template variable (derived from target realm origin) so the agent never guesses staging/prod hosts	`factory-target-realm.ts` (new `deriveCatalogRealmUrl`)
Resolver instrumentation: logs the resolved skill list per issue so future debugging doesn't require code reading	`factory-skill-loader.ts`
CI mirror workflow: clones `cardstack/boxel-catalog`, regenerates from `usage.gts`, pushes the diff on merge to main	`.github/workflows/mirror-boxel-ui-specs.yaml` ⚠️ needs `BOXEL_CATALOG_PUSH_TOKEN` secret
`docs/spec.md` Section 3 (drops "no support for examples", documents the generator + dev workflow + cross-repo gotchas)	`docs/spec.md`
Three example briefs that exercise the discovery loop	`packages/software-factory/realm/Wiki/{delete-confirmable-note,support-ticket-form,product-faq}.json`

Architecture (briefly)

usage.gts is the source of truth in this repo.
Generator extracts the primary <FreestyleUsage> block per component → markdown readMe with Import / API table / Example / CSS Variables.
Output is written only to packages/catalog/contents/Spec/ — the working tree of the deployed cardstack/boxel-catalog repo (gitignored from boxel). No generated specs are committed to the boxel repo.
CI mirror workflow regenerates fresh on every merge and pushes to cardstack/boxel-catalog.
Realm-server file-watcher reindexes locally (when catalog/contents/ exists). Deployed catalog reindexes on next startup via the existing start-staging.sh → setup:catalog-in-deployment flow.
Agent-side discovery is opt-in. The feature flag controls whether the agent's system prompt mentions the catalog at all.

Feature-flag rationale

Pre-CS-10527 factory runs completed inside their 50-turn budget on most briefs. Adding the discovery skill (~80 lines of always-loaded content) increases prompt bloat and modestly competes for the agent's attention with other always-loaded references like dev-technical-rules.md. Putting the skill behind a flag means:

Production / day-to-day factory runs are unchanged. Turn budget is preserved.
Experimentation with component discovery is opt-in per run.
We can iterate on the skill's wording and the discovery experience without affecting unrelated factory work.

Verified end-to-end (with the flag)

Ran the factory loop against the product-faq brief — a brief that describes "collapsible disclosure", "compact tag-style indicator", and "primary action button" in plain language, never naming any component:

import { Accordion, Button, Pill } from '@cardstack/boxel-ui/components';

All three landed correctly from the catalog. Zero raw <button> / <input> / <details> in the output. Card passed lint, parse, evaluate, instantiate, and tests.

Test plan

Configure BOXEL_CATALOG_PUSH_TOKEN repo secret with write access to cardstack/boxel-catalog.
Run the generator locally and confirm the catalog realm indexes the 52 new specs.
Run factory without the flag against any normal brief — verify the agent sees no boxel-ui content (grep -c "boxel-ui-component-discovery\|Catalog realm:" /tmp/factory-run.log should be 0) and the run completes as before.
Run factory with --enable-boxel-ui-discovery against product-faq (or another UI-heavy brief) — verify the agent imports from @cardstack/boxel-ui/components.
After merge: mirror workflow runs and opens/pushes to cardstack/boxel-catalog.

Follow-ups (out of scope here)

Same skill in packages/skills-realm/contents/Skill/ for the in-host Matrix code agent (this PR covers the factory only).
Improve cardDescription quality for the ~14 components whose usage.gts lacks a top-level @description (the generator falls back to a placeholder for those today).
A short gts-pitfalls reference for known parse traps (@tracked on fields of inline class expressions, generics on class-property initializers) that consume turn budget today.
Verify the mirror workflow against the real cardstack/boxel-catalog repo (requires the token + a real PR cycle).

🤖 Generated with Claude Code

github-actions · 2026-05-13T12:55:05Z

Preview deployments

Host staging preview

Host production preview

Boxel UI staging preview

Host Test Results

1 files ±0 1 suites ±0 1h 45m 23s ⏱️ + 3m 56s
2 661 tests ±0 2 646 ✅ ±0 15 💤 ±0 0 ❌ ±0
2 680 runs ±0 2 665 ✅ ±0 15 💤 ±0 0 ❌ ±0

Results for commit 2de436c. ± Comparison against earlier commit 61b00ec.

Realm Server Test Results

1 files ±0 1 suites ±0 11m 55s ⏱️ + 1m 2s
1 365 tests ±0 1 365 ✅ ±0 0 💤 ±0 0 ❌ ±0
1 444 runs ±0 1 444 ✅ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 2de436c. ± Comparison against earlier commit 61b00ec.

Publishes one Spec card per @cardstack/boxel-ui component into the catalog so the software factory agent can discover and reuse UI primitives by querying the realm instead of hand-rolling HTML. Generator (packages/boxel-ui/addon/bin/generate-component-specs.mjs): walks each component's usage.gts, extracts the primary FreestyleUsage block (args, description, example, CSS vars), and emits one JSON Spec per component. Wired as `pnpm generate:component-specs`. Writes two outputs: an in-repo snapshot under test/fixtures/specs/ (52 files) for the CI drift gate, and the live tree at packages/catalog/contents/Spec/ for local realm-server file-watcher reindex. Normalizes internal Boxel- prefixed tag names in the example block to the public export name (`<BoxelInput>` → `<Input>`) and inlines class-field array literals for `@options` so enum values are listed verbatim. Resolver: registerCardReferencePrefix('@cardstack/boxel-ui/', …) in host/app/lib/externals.ts so a Spec ref pointing at @cardstack/boxel-ui/components resolves via the existing fake-packages URL scheme. Agent wiring: new factory skill `packages/software-factory/.agents/skills/boxel-ui-component-discovery/` that teaches the discovery recipe and makes the rule mandatory before any UI is written. Loaded automatically by factory-skill-loader on the same GTS-keyword trigger as ember-best-practices. The system prompt gains a corresponding catalog-search exception under "Stay in your target realm" and surfaces the catalog realm URL (derived from the target realm origin) so the agent doesn't probe staging/prod hosts. boxel-development pointers reinforce the rule from the always-loaded skill. CI: ci-lint runs `generate:component-specs --check` against the boxel-ui snapshot to catch missed regenerations. mirror-boxel-ui-specs.yaml publishes the generated specs to cardstack/boxel-catalog on merge to main (needs BOXEL_CATALOG_PUSH_TOKEN secret — documented inline). Test artifacts: two example briefs under packages/software-factory/realm/Wiki/ (delete-confirmable-note, support-ticket-form) exercise the discovery loop against Modal/Button and Input/Select/Button/Pill respectively.

jurgenwerk force-pushed the cs-10527-component-specs-for-searchable-reusable-ui-components branch 7 times, most recently from ea74310 to 61b00ec Compare May 14, 2026 08:38

jurgenwerk force-pushed the cs-10527-component-specs-for-searchable-reusable-ui-components branch from 61b00ec to 2de436c Compare May 14, 2026 09:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate boxel-ui component Specs for AI agent discovery (CS-10527)#4809

Generate boxel-ui component Specs for AI agent discovery (CS-10527)#4809
jurgenwerk wants to merge 1 commit into
mainfrom
cs-10527-component-specs-for-searchable-reusable-ui-components

jurgenwerk commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jurgenwerk commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Pieces

Architecture (briefly)

Feature-flag rationale

Verified end-to-end (with the flag)

Test plan

Follow-ups (out of scope here)

Uh oh!

github-actions Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Preview deployments

Host Test Results

Realm Server Test Results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jurgenwerk commented May 13, 2026 •

edited

Loading

github-actions Bot commented May 13, 2026 •

edited

Loading