Skip to content

🪄 fix: Ground the Default Artifacts Prompt in What the Renderer Supports - #17024

Merged
danny-avila merged 3 commits into
devfrom
danny-avila/artifacts-default-prompt
Oct 16, 2026
Merged

danny-avila merged 3 commits into
devfrom
danny-avila/artifacts-default-prompt

Conversation

@danny-avila

Copy link
Copy Markdown
Collaborator

Pull Request

Summary

The built-in artifacts prompt (Normal and shadcn/ui modes) costs about 2,500 tokens per request, ~5,100 to 5,800 with shadcn/ui. Several of its rules don't match the renderer. It sends models to a /api/placeholder/W/H image route that LibreChat doesn't serve, bans Tailwind arbitrary values that the Play CDN compiles, says a bare React element renders when a default export is required, and in shadcn/ui mode documents sheet and aspect-ratio, which aren't mounted into the sandbox, so those imports fail to build. It also doesn't mention the grammar edges that actually break a directive, such as a line of just ::: inside the content, which ends the artifact even inside its code fence. The prompt was kept as two near-identical copies, an XML one for the anthropic provider and a Markdown one for everything else (Claude on Bedrock already got the Markdown copy), plus a third, older copy in api/app/clients/prompts/artifacts.js that the Assistants endpoints still loaded.

This PR replaces the prompt with one shorter version for every provider. Each rule maps to a renderer constraint or to a failure measured in real usage (below). The Assistants endpoints now import the same prompt from @librechat/api, and the legacy copies are deleted. The shadcn/ui component list now comes from one shared constant, artifactUIComponents in librechat-data-provider. The client mounts those files from it and the prompt lists their names, so the prompt can't mention a component the sandbox lacks.

Default prompt: 2,546 → 825 tokens (o200k; 2,513 → 825 for the anthropic provider). shadcn/ui: 5,133 → 3,251 (5,784 → 3,892 for anthropic, whose component docs stay in XML).

Usage findings (demo, aggregate)

These numbers come from the demo deployment's assistant messages containing :::artifact. I counted directives only when :::artifact{ starts a line outside a code fence; 216 matches were mentions inside code or prose and are excluded. Usage is thin, so treat it as supporting evidence; the renderer constraints carry most of the weight.

  • Volume: the last 60 days had only 5 artifact messages (3 users, one Normal-mode agent; all text/markdown). One was cut off unclosed when its generation was interrupted. Across all history (Aug 2024 to Oct 2026) there are 198 directives in 168 messages and 110 conversations, from 4 users; 94% came from one admin account. The last 12 months have 50.
  • Types (all time): Mermaid 52, React 41, SVG 41, HTML 36, Markdown 11. 20 (10%) used types the renderer doesn't support: text/plain 7, application/javascript 4, application/vnd.ant.mermaid 3, application/json 2, the template's literal mime-type 2, and text/css and text/javascript 1 each. Unknown types are previewed as HTML. None occurred in the last 12 months.
  • Syntax: all 198 had the attributes in order and in double quotes, and none had a split opening line. 12 (6%) were missing the closing fence and :::; only 1 of those is from the last 12 months, the interrupted message above. 4 had no fence. 1 had an inner 3-backtick fence close its 3-backtick outer fence early. Since the 4-backtick rule landed (🧵 fix: Preserve Fenced Markdown Artifacts #14121), 12 of 12 used 4-backtick fences.
  • Unnecessary artifacts: 49 of 198 (25%) had fewer than 15 lines, including 24 of the 52 Mermaid diagrams. In the last 12 months it was 5 of 50, 4 of them Mermaid. A ```mermaid code block already renders inline in chat.
  • React: 5 of 41 imported modules that aren't available (4 imported project-local ~/ or ./ files, i.e. app code that can't run in a sandbox; also react-hook-form and lodash). 4 of 41 had no default export. None used Tailwind arbitrary values or @/ shadcn paths.
  • HTML: 2 of 36 used the nonexistent /api/placeholder route. 9 loaded external images and 1 loaded a script from outside cdnjs; both load fine, since there is no CSP in the preview.
  • SVG: 0 of 41 lacked a viewBox.
  • Truncation: no "rest of the code" markers in real artifacts.
  • Updates: in conversations that already had an artifact, 28 reused an identifier and 53 started a new one.
  • Follow-ups: 74 artifact messages got a user reply. 8 read as failures (4 blank or not rendering, 3 errors, 1 other), mostly on 2024-era Mermaid artifacts. None in the last 12 months.
  • Client logs: no artifact, Sandpack, or Mermaid errors in RUM since it started (Oct 4). The panel doesn't report Sandpack errors, so this proves nothing either way.

Prompt changes and their evidence

Change Evidence
One prompt for every provider; the Assistants endpoints use it too Bedrock Claude (62 artifacts) already got the Markdown copy with no malformed directives. Three copies had drifted (the legacy one still said lucide 0.263.1).
Format rules: opening line on one line, no space before {, nothing after }, double quotes, no " in values, close with ::: The micromark grammar rejects each of these; the new tests pin them.
New: a bare ::: line inside the content ends the artifact The container checks every line for a closing fence before it parses inner code fences; tested.
Kept: fence longer than any inner fence 1 early close under 3-backtick fences, and full compliance since #14121.
Whole content on update, reuse identifier (one line, no examples) The client renders each message's artifact from its own content and never merges diffs.
"Use exactly one of these types; other code belongs in a code block" 10% unsupported types historically, which preview as HTML.
Short Mermaid diagrams go in a ```mermaid block 24 of 52 Mermaid artifacts were under 15 lines, and those blocks render inline.
Removed /api/placeholder; draw placeholders with inline SVG or CSS The route doesn't exist (the URL returns the bundler's HTML), and 2 HTML artifacts used it.
Removed the "no arbitrary values" ban Tailwind Play CDN 3.4.17 compiles them, and usage showed none anyway.
React: default export, no required props; removed "React elements" and class components The entry does import App from "./App"; 4 of 41 had no default export.
React imports: exact list incl. react-dom, lucide-react 0.394.0 (matches ^0.394.0), shadcn names from artifactUIComponents, not @/ 5 of 41 bad imports; the template has no @/ alias.
"Code needing other packages, local files or a server: use a code block" 4 of 41 React artifacts were project code importing ~/ files.
Markdown: raw HTML, math, and Mermaid don't render The marked page drops raw HTML and has no math or Mermaid support.
SVG viewBox kept, reworded The preview stretches the root <svg>, so only a viewBox scales it.
HTML: cdnjs-only scripts kept Kept as policy, consistent with the published docs template; not enforced by the runtime.
Removed: "self-deprecating humor", "offer to install shadcn" (they're mounted), the duplicate example sets, the text/md note (aliases are normalized) No measured failure or renderer constraint behind them.
shadcn/ui mode: removed sheet and aspect-ratio docs; removed the duplicated moreExtraComponents block (same output) Neither file is mounted, so those imports fail.
shadcn/ui docs now follow a blank line They were concatenated straight onto the last line of the base prompt.

How it works

librechat-data-provider  artifactUIComponents  { 'alert-dialog': source, ... }
├── client/src/utils/artifacts.ts   sharedFiles['/components/ui/<name>.tsx']  (mounted into Sandpack)
└── packages/api/src/prompts/artifacts/index.ts   "where name is one of: <keys>"
        ├── agents: initialize.ts   generateArtifactsPrompt({ endpoint: provider, artifacts })
        └── assistants / azureAssistants build.js   (was the legacy api/ copy)

Type of change

  • Bug fix
  • Refactor
  • Tests / tooling / CI

Testing

I measured usage on the demo MongoDB, read-only and aggregate only. I checked each rule against the client parser (remark-directive / micromark), the Sandpack templates and dependencies, and the Mermaid, Markdown and SVG renderers.

Tested environments/configuration:

  • Node 24.16.0, Windows; o200k_base / cl100k_base token counts via js-tiktoken

Automated tests:

  • packages/api/src/prompts/artifacts/index.spec.ts (new): custom mode returns null; every provider gets the same default prompt; shadcn/ui docs follow a blank line (XML for anthropic); the listed component names equal artifactUIComponents; every /components/ui/* import in either mode resolves to a mounted file; no /api/placeholder; the example is a well-formed artifact.
  • client/.../MarkdownBlocks.artifacts.test.tsx: added directive-grammar cases. The documented opening line renders, and these do not: a space before {, attributes split across lines, text after }, an escaped quote. A bare ::: line closes the artifact inside its fence.
  • npx jest src/prompts/artifacts (packages/api); npx jest src/utils/__tests__/artifacts.test.ts src/components/Artifacts src/hooks/Artifacts and the Markdown artifact suite (client): all pass.
  • npx tsc --noEmit in packages/api; npm run typecheck in client; data-provider build (tsc); ESLint and sort-imports on touched files.

Screenshots / recordings

No user-facing change. The mounted shadcn/ui files are identical; only the prompt text sent to models changes.

Risk / compatibility

  • Provider behavior: the anthropic provider now gets the Markdown-formatted prompt that every other provider (including Claude on Bedrock) already used. Its shadcn/ui component docs stay in XML.
  • Assistants endpoints: they now get the current prompt instead of the legacy copy.
  • Custom mode: unchanged; it injects nothing.

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors
  • User-facing or complex behavior is documented where necessary
  • Required dependency changes have been merged/published
  • Required documentation PR: 📝 docs: Document Custom Artifact Instructions docs#825 describes the current prompt (≈2,700 / ≈6,000 tokens, placeholder and arbitrary-value notes) and needs a follow-up once this merges

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Pass 1 of 3 on head 3fdfc4a.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T14:40:13.821709Z 77e5b4e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 3fdfc4a0fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

Pass 2 of 3 on head 3fdfc4a.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 3fdfc4a0fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

This is the final review for this PR (pass 3 of 3) of head 3fdfc4a. Please flag only blocking issues: correctness, security, or data loss.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 3fdfc4a0fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila

Copy link
Copy Markdown
Collaborator Author

@codex review

This is the final review for this PR (pass 4 of 4) of head 77e5b4e. Please flag only blocking issues: correctness, security, or data loss.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 77e5b4ea14

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danny-avila
danny-avila merged commit 0a1f4df into dev Oct 16, 2026
46 checks passed
@danny-avila
danny-avila deleted the danny-avila/artifacts-default-prompt branch October 16, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant