feat: harden generator and generated site security - #19
Merged
Conversation
- Drop next, react, and react-dom from the CLI's own dependencies; they belong to the generated app's package.json, not the CLI runtime. - Add scripts/clean.mjs and route build/prepare through it so a stale dist/ is never shipped or left over from a prior package manager. - Add scripts/verify-package.mjs (test:package) to catch stale compiled output and a package-manager-specific prepare script. - Add scripts/smoke-generated-site.mjs (smoke:generated) to generate a fixture site, then type-check, lint, and build it end to end. - Run format:check, test:package, and smoke:generated in CI, and bump the pinned pnpm setup action to 11.17.0. - Declare packageManager: pnpm@11.18.0 and engines.node >=22, ignore /.pnpm-store, and document the new scripts in CONTRIBUTING.
- Claim output directories safely (src/lib/output-safety.ts): refuse to overwrite an unrelated non-empty directory, recognize a Doccupine marker file and legacy generated apps, and resolve every generated path so it cannot escape the output directory. - Write every generated file atomically (writeFileAtomic) so the dev server and file watcher never observe a partially written file. - Serialize watcher-triggered mutations through a queue so concurrent file events cannot race the shared aggregate files (layout, sitemap, llms files, API manifest). - Fail generation with an actionable error on route collisions and on unsafe or duplicate section slugs/directories, instead of silently letting one page overwrite another. - Validate doccupine.json (validateConfig): reject overlapping watch/output directories, invalid ports, and malformed openapi config, with a pointer to `doccupine config --reset`. - Cache dependency installs with a package.json + package-manager fingerprint (.doccupine-install), skipping reinstall when nothing changed, and add a --skip-install flag. - Add a `generate` alias for `build`, kept decoupled from installing dependencies or starting the dev server. - Always generate sitemap.xml and robots.ts, even without a configured site URL, and read NEXT_PUBLIC_SITE_URL for both instead of only config.json. - Build an API-reference index page that links every generated OpenAPI endpoint, grouped by spec and tag. - Escape generated metadata (title, description, icon, image) with JSON.stringify instead of interpolating into template literals, closing a code-injection surface for untrusted frontmatter values.
- Document the new /api-reference index page, and clarify that the playground proxy blocks private, internal, metadata, and reserved network addresses, with loopback servers such as localhost allowed only when the spec declares them explicitly. - Document that SITE_PASSWORD protection now also covers the API playground proxy, and that every content route re-checks its own session instead of relying solely on the middleware. - Document the DOCS_API_KEY / SITE_PASSWORD interaction for the MCP endpoint: API-key auth when configured, otherwise the same gate session as the rest of the site. - Document --skip-install, the `generate` alias for `build`, and the output-directory safety and section-slug validation rules. - Document the `updated` frontmatter field, and that sitemap.xml and robots.txt are now always generated even without a configured site URL. - Document the services/mcp/docs-content.json manifest that now powers MCP content indexing, replacing filesystem scanning of the generated app. - Add the Space component reference page and link it from the components overview.
- Require the visitor's unlocked gate session (or DOCS_API_KEY for MCP) inside the rag, search, playground, and mcp route handlers themselves via src/templates/lib/access.ts, not just in middleware, so a direct route invocation can no longer bypass SITE_PASSWORD. - Extend the playground proxy's SSRF guard to permanently block reserved, multicast, and IPv4-mapped IPv6 targets, and fail closed for any address outside an explicitly declared loopback server. - Match the playground allowlist's base path on a path-segment boundary so a shared prefix (e.g. /v1x/... vs /v1/...) is no longer treated as a match. - Bound the request rate limiter's in-memory map so a client rotating spoofed forwarding headers cannot grow it without limit. - Compare API keys and bearer tokens with a constant-time comparison. - Replace filesystem scanning of the generated app for MCP content with a build-time services/mcp/docs-content.json manifest, traced into the serverless functions so the docs corpus is not inlined into the function bundle and the server never scans user-reachable paths at runtime. - Add src/templates/security.test.ts asserting these boundaries stay in place.
- Explain the output-directory claiming rules and the `doccupine config --reset` recovery path for an invalid config. - Document `doccupine generate` as an alias for `build`, and the new --skip-install flag. - Explain that /sitemap.xml is always served (empty without a configured site URL) and that NEXT_PUBLIC_SITE_URL is baked in at build time, so changing it needs a redeploy. - Document that the OpenAPI integration now generates an /api-reference directory index in addition to per-operation pages. - Document that SITE_PASSWORD protection now also covers the /api/playground proxy, and that the MCP endpoint falls back to the gate session when DOCS_API_KEY is not configured.
Long inline `code`/`kbd` spans previously stretched to fill the container width. Constrain them to max-content so they hug their text like other inline elements.
Update the generated app's posthog-js dependency from ^1.407.8 to ^1.408.0.
- rateLimit: stop keying buckets off the client-supplied X-Forwarded-For header, which let anyone bypass the limit by rotating a spoofed value. Derive identity only from headers that a recognized hosting platform (Vercel, Cloudflare, Fly) itself overwrites, falling back to one shared bucket for unknown/self-hosted deployments instead of trusting an attacker-controlled address. - ssrfGuard: rewrite IPv6 blocking to parse the full 128-bit address instead of matching prefixes with startsWith, which could both over- and under-match adjacent ranges. Correctly resolves IPv4-mapped, IPv4-compatible, IPv4-translated, and NAT64 forms to their embedded IPv4 address before applying the IPv4 block list, and restricts the allowPrivate loopback opt-in to non-production so a generated spec cannot reach a private target in a deployed build. - mcp route + server: bound request bodies, protocol batches (at most one tools/call per request), tool arguments, search result counts, and serialized responses; wire AbortSignal through search, embedding, and tool calls so a cancelled request actually stops work instead of finishing unattended; validate both REST and protocol tool calls against shared strict zod schemas. - rag route: add RAG_API_KEY bearer auth for server-to-server access on public sites (browser chat cannot safely hold the secret), bound request size, and wire request cancellation through indexing, search, and model streaming. - gate and playground routes: switch to the new rateLimit(Request) signature. - env.example / gitignore: document RAG_API_KEY and DOCS_API_KEY, and ignore every .env* variant instead of only .env*.local. - security.test.ts: cover the above.
- OpenApiRegistry.load: build the next registry into a scratch instance and swap it in only after every configured spec parses and ingests successfully, then throw on failure. Previously a spec that failed to parse was logged and skipped in place, leaving the live registry half-updated instead of surfacing the error. - safeMatter: detect YAML-incompatible tagged frontmatter delimiters (e.g. a language tag right after the opening `---`) and treat them as unsupported instead of handing them to gray-matter, which can invoke a non-YAML parsing engine for those tags.
- SearchModalContent: proper dialog + combobox model (role="dialog", aria-modal, role="combobox"/"listbox"/"option", aria-activedescendant), a focus trap with focus restored to the opener on close, and a polite live region announcing result counts. - Chat: mobile chat becomes a real modal (role="dialog", inert + aria-hidden while closed, focus trap, focus restored to the opener on close), autoscroll only when already near the bottom, and respects prefers-reduced-motion. - Accordion / sidebar groups: collapsed content is inert and aria-hidden instead of just visually hidden; sidebar toggles gain aria-controls wired to the region they open. - Tabs: implements the standard horizontal ARIA tabs keyboard pattern (role="tablist"/"tab"/"tabpanel", roving tabIndex, Arrow/Home/End). - eslint.config: enable the complete jsx-a11y recommended rule set, with narrow scoped exceptions for the two generated components that intentionally break a rule (ApiPlayground's user-controlled media preview, SiteGate's autofocus login field). - New accessibility.test.ts pins the above down.
- src/cli.ts: commander wiring (watch/build/config commands), dependency-install caching, and process orchestration, extracted from src/index.ts. - src/mdx-to-nextjs-generator.ts: the MDXToNextJSGenerator engine (scaffolding, watchers, page/layout regeneration), also extracted from src/index.ts. - src/lib/generated-artifacts.ts: new GeneratedArtifacts class that tracks generated route and Markdown-mirror ownership in .doccupine-artifacts.json, migrating from the older .doccupine-llms-manifest.json / .doccupine-api-manifest.json files. Watch-mode renames and deletes now consult this registry instead of guessing routes from filenames, so a changed section removes only the files it owns. - src/index.ts shrinks to a thin re-export of the split modules. - Add matching test files for the new modules.
pnpm test now runs pnpm build && vitest run, so tests always exercise a fresh dist output locally and in CI. Drop the now-redundant explicit pnpm build step from the CI workflow.
Update the seeded sample docs to match this round of changes: - ai-assistant / authentication: RAG_API_KEY for server-to-server /api/rag access, and that .env* (not just .env.local) is ignored. - api-playground: IPv6-translated private targets are blocked too, loopback servers are development/test-only, and proxy requests are rate limited per trusted client address. - commands: starter docs are only seeded into an empty source directory, non-YAML frontmatter is rejected, and generated route ownership is tracked in .doccupine-artifacts.json. - model-context-protocol: get_doc/list_docs work without an embedding provider, rate limiting uses the platform's trusted client address, and the request/argument/result size limits. - tabs: document the horizontal ARIA keyboard pattern.
- README: note RAG_API_KEY, the renamed .doccupine-artifacts.json (was .doccupine-llms-manifest.json), and that get_doc/list_docs work without an embedding provider. - SECURITY: document RAG_API_KEY for protecting paid RAG spend on public sites, and clarify that /api/mcp falls back to the site gate when DOCS_API_KEY is unset.
Doc pages are force-static, so the server always renders the light theme; the previous fix (6d57864) resolved the mode before paint via a "dark" class but still had to hide the body until Cherry's ClientThemeProvider swapped in themeDark on mount, because theme.ts still resolved colors/shadows to literal hex per mode. Two things combined to leave a residual flash: the provider strips its own "dark" class for a frame on mount while it reconciles the server-rendered light theme against the cookie, and nothing painted purely from CSS custom properties, so anything keyed off that class (or off the literal theme object) had a light frame to flash through. - app/theme.ts: every painted color and shadow is now a var(--color-*) / var(--shadow-*) reference built by a new cssVarTokens helper, so the CSS styled-components generates is identical in both modes. theme and themeDark now differ only in isDark. - lib/layout.ts: the blocking theme-init script only has to decide the mode and stamp it on <html> before paint; drop the body hiding, the forced #000 background, the transition suppression, and the MutationObserver that existed only because the server always rendered light. - The visible mode is keyed to a data-theme attribute we own, not to Cherry's "dark" class, since that class is the one the provider strips for a frame on mount. Selectors move to :root[data-theme="dark"] in GlobalStyles, Callout, Code, Badge, and CopyButton; CherryThemeProvider gains a ThemeModeAttribute component that mirrors later mode changes onto the attribute. - GlobalStyles declares color-scheme per mode (previously set inline on <html>, which stuck to the first visit's mode after a toggle). - CherryThemeProvider now owns the theme-color meta tag ($themeColor={false}), because Cherry would otherwise copy the literal string "var(--color-light)" into it. - Card / Steps: icon tint moves into CSS, since var() does not resolve inside an SVG presentation attribute (lucide's color prop). - DemoTheme: presets now only write --color-* overrides. - New utils/polishedCompat.ts plus a turbopack.resolveAlias in next.config.ts (registered in lib/structures.ts): published cherry-styled-components (0.2.11) calls polished's rgba/darken/lighten on theme colors, and polished throws on a var() reference, crashing the render. The shim performs those three operations with color-mix instead. The dependency pin stays ^0.2.11, so it picks up 0.2.12 (which needs no shim) automatically once that is published. - New theming.test.ts locks the contract down: mode resolved before first paint onto data-theme, nothing hidden while waiting for hydration, every token painted through a custom property, icons tinted in CSS, and the polished shim wired up.
StyledChat and StyledChatForm transitioned "all", which animates visibility, so the panel stayed hidden for the whole 0.3s slide-in and the focus() call that follows opening was a silent no-op. Both now list their transitions explicitly with "visibility 0s linear": snapping visible immediately on open, and delaying the hide on close so the slide-out still animates.
Revert an unwanted restyle: the search input, its Ask-AI pill, and the chat input go back to plain styled.input / styled.button instead of Cherry's Input/Button (the five affected styled blocks are byte-identical to HEAD again). The three accessibility.test.ts assertions that described the Cherry adoption itself are dropped; every other accessibility assertion is untouched. Also, in the search modal: - Hide the platform clear button on the type="search" field (::-webkit-search-cancel-button / -decoration), since it duplicated the modal's own close control. - Add an 8px gap between the spinner and "Searching documentation...".
luangjokaj
changed the title
cherry-styled-components 0.2.12 replaced its polished-based color math with color-mix and dropped polished from its dependency tree, which is what the compatibility shim added in e7d328a existed to work around. With 0.2.12 published, migrate off the shim. - package.ts: bump the generated app's cherry-styled-components pin from ^0.2.11 to ^0.2.12. This is what makes the removal safe: it invalidates the dependencyFingerprint install stamp (forcing a reinstall) and the new range no longer resolves to 0.2.11, so no project can still be running the version the shim protected against. - utils/polishedCompat.ts: delete the shim (rgba/darken/lighten reimplemented with color-mix). - structures.ts: drop the shim's import and its appStructure entry, and add "utils/polishedCompat.ts" to obsoleteFiles so projects generated before this change delete their stale copy on the next run. - next.config.ts: remove the turbopack.resolveAlias mapping polished to the shim, in both the PostHog and non-PostHog branches. - theming.test.ts: drop the test case asserting the shim wiring and its three now-unused imports. Verified in a regenerated fixture: the stale shim file and next.config alias are gone, pnpm install upgrades cherry-styled-components to 0.2.12 on its own with polished absent from the tree, and the fixture passes type-check, lint, and next build. In headless Chrome against 0.2.12 with no shim: no light frame during load, correct paint in both modes with every JS chunk blocked, no html class flicker, and the ThemeToggle knob now positions itself before hydration. CLI: pnpm build, 142 tests (12 files), and pnpm format:check all pass.
- Prompt.ts: switch the header's text mixin from styledSmall to styledText, dropping the now-unused styledSmall import. - Change the header's align-items from center to flex-start and give the leading icon a 4px top margin, so it lines up with the first line of the now-larger text instead of the vertical center of the whole (possibly multi-line) block.
Pack the built CLI with npm, install it into an isolated production project, and verify both its module import and installed binary. Reject missing or unexpected tarball contents.
Render spec-controlled prose as inert JSX text and resolve only local OpenAPI files inside rootDir. Snapshot references before dereferencing to prevent network access, path escapes, and mutable rereads.
Reject symlinked generated paths and require strict legacy ownership fingerprints before claiming output. Validate and roll back hot config reloads while preserving user-owned public aggregate files.
Classify API, MCP, Next.js, and analytics paths by complete route segments. Avoid treating similarly prefixed documentation routes as reserved or bypassing the generated site gate.
Start transform and opacity transitions immediately when chat closes, then delay visibility for 300ms so the closing animation can finish.
luangjokaj
changed the title
- Add authentication guidance for site passwords, sessions, and API keys. - Document publish validation, granular discards, navigation slug rules, and API spec requirements. - Expand Agent Sync setup, auto-linking, and repository workflows, and align related platform docs.
luangjokaj
changed the title
- Explain that $horizontal belongs inside a line of inline content, where the gap has something to sit between. - Replace the two side-by-side buttons, which had no real inline gap to demonstrate, with a single button whose label and icon are spaced apart by the Space component.
Secure atomic output mutations and validate claimed output-root identity before generated operations. Read MDX, starter documentation, configuration, and public assets without following links. Restrict local OpenAPI references to the configured spec directory and safe extensions. Bound JSON request envelopes across generated routes.
Keep chat and search mutually exclusive while transferring opener ownership between dialogs. Restore focus only after the active overlay closes.
luangjokaj
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
High-severity review follow-up
claimed output-root identity before mutations, and rejects replaced roots or
symlinked path components.
files through validated no-follow source paths. Public copies preserve binary
data and reject symlink and path-swap escapes.
$reffiles to the configured root specdirectory, approved JSON/YAML extensions, and non-hidden paths while keeping
the explicitly configured root filename unrestricted.
playground, RAG, and MCP routes. The playground separately caps decoded UTF-8
and base64 request payloads.
dialogs, and restores focus only after the active modal closes.
symlink replacement, source path races, restricted OpenAPI references,
request-envelope limits, abort handling, and modal focus ownership.
Generator safety and packaging
configuration and section routes before changing output.
deletes remove only files owned by the changed source.
after overrides are deleted.
--skip-install, and providesgenerateas an alias forbuild.installed binary execution.
Generated-site security and accessibility
hardens rate-limit identity, and tightens SSRF handling for IPv4 and IPv6.
propagating cancellation through RAG and MCP work.
semantics, keyboard behavior, focus containment, and reduced-motion support.
traps.
Generated documentation and presentation
the shared code component.
the API playground, and Agent Sync workflows.
theme state synchronized without hydration flash.
Verification
pnpm test: 204 tests passedpnpm test:package: 375 files verified, isolated production install passed,package import succeeded, and installed binary 0.0.140 executed
pnpm smoke:generated: generated type-check, ESLint, and Next.js productionbuild passed
pnpm format:checkgit diff --checkTest plan