Repository navigation
refactor: Assemble the System Prompt from Structured, Kind-Classified Parts - #17145
Open
Shabonasar wants to merge 10 commits into
Open
Shabonasar wants to merge 10 commits into
Shabonasar wants to merge 10 commits into
Conversation
The per-request web search context (current date/time with milliseconds) was joined before additional_instructions, which carry mostly static text such as the artifacts prompt. Provider-side prefix caches stopped at the timestamp, so the static text after it was recomputed on every request. Move the dynamic context to the end via buildAdditionalInstructions and cover the ordering with unit tests.
…che-order # Conflicts: # packages/api/src/agents/initialize.ts
…e date changes in history The web search runtime context and temporal special variables were anchored to the turn start with millisecond precision. The system message precedes the whole history, so every turn invalidated the cached history and new conversations could not reuse the system prompt (LibreChat-AI#16877). - Web search runtime context states the conversation start in the user's timezone with minute precision; temporal variables resolve against the same persisted anchor. - A user message sent on a later local date than the model last saw gets a <context_update> block in its prompt copy only, derived from stored createdAt values so it renders identically on later turns; compaction boundaries follow the agents SDK formatter. - Optional endpoints.agents.promptMessageTimestamps adds the send time to every user message; the turn's user message is stored with createdAt = turnStartedAt on every save path. - Provider payload stability tests through createRun for OpenAI, an OpenAI-compatible endpoint, Anthropic, OpenRouter, Bedrock and Google.
… routes The browser resolved promptPrefix with its own clock on every send, so turn 1 of a spec-backed conversation carried the raw template while turn 2 carried resolved text in a different system-prompt position. Only Assistants routes resolve on the client now; the server resolves against the conversation start.
… of a conversation
Adds `endpoints.agents.promptCache.stableContext` (default off). Off, the prompt is
built as before. On:
- prompt time (`{{current_datetime}}` and friends, the web search runtime context) is
anchored to the conversation start, and a date change is announced by a
`<context_update>` block at the start of the first user message sent after it;
- the conversation-stable part of `additional_instructions` is merged into the system
instructions for Anthropic and OpenRouter with prompt caching, while memory, RAG and
one-turn queued file context stay in `additional_instructions`, where the SDK moves
them next to the newest user message;
- dynamic tool context goes after the additional instructions;
- the user message is inserted with the turn start as `createdAt`, so later turns render
the same stamp.
`promptMessageTimestamps` (default off) additionally prefixes each user message with the
time it was sent.
The history notes, the instruction split and the lever live in `packages/api`; the
controller only calls them. `applyContextToAgent` accepts the run context as parts and
records which are volatile, so no text is split by matching.
…assified parts Producers (project, memory guard, artifacts, skills catalog, RAG, memory, agent context files, always-apply skills of isolated subagents) contribute keyed parts through contributeContext instead of concatenating strings into additional_instructions. One registry fixes the canonical order and classifies every part, tool context map entries included, as static, conversation or turn. A runtime key the registry does not know counts as turn, so a new producer cannot break the stable prefix by default. assembleSystemPrompt builds the strings handed to the SDK. Once parts exist they are the only source of the tail; a direct write to additional_instructions is ignored with a warning. Without promptCache.stableContext it reproduces the previous request; with it, createRun moves only the turn parts with the prompt-cache tail and merges the rest into the instructions. The assembled text is byte-identical to string concatenation for both values of the lever, guarded by a snapshot recorded on the previous implementation. Isolated subagents with an always-apply skill place the skill ahead of the tool runtime context when stableContext is on.
This was referenced Oct 15, 2026
codegraph-librechat
Bot
added
🗺️ Backend Infra
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The system prompt tail was built by concatenating strings from many producers (project, memory guard, artifacts, skills catalog, RAG, memory, agent files, subagent skills), so nothing said which text is static, which is stable for a conversation and which changes every turn; a new producer could silently break the stable prefix. This PR makes producers contribute keyed parts to one registry that fixes the canonical order and classifies each part as static, conversation or turn. An unknown runtime key counts as turn. It changes no behaviour: with
endpoints.agents.promptCache.stableContextoff (the default) the assembled text is byte-identical to the previous concatenation; with it on, the same text is only split into the turn parts and the rest.Related to #17102. Stacked on #16369 (PR1). No SDK dependency.
How it works
additional_instructionsis ignored with a warning.Measurements
No behaviour change, so no measurement. A snapshot of 23 cases, recorded on the previous implementation, is asserted with the lever off; the lever-on layout is covered by
promptPrefixStability.test.tsandsystemParts.spec.ts.Type of change
Testing
Unit tests only; the lever needs no manual setup.
Automated tests: the test files this PR adds or changes, run per workspace with
npx jest <files>(mongod-backed suites against a real mongod): packages/api 203, api 457 tests passing. Files:openai.spec.js,responses.unit.spec.js,client.test.js,promptPrefixStability.test.ts,systemParts.snapshot.test.ts,context.spec.ts,run.spec.ts,systemParts.spec.ts.Static checks:
tsc --noEmitin packages/api; eslint and prettier on the touched files.Screenshots / recordings
No user-facing change.
Risk / compatibility
None expected: with the lever off the output is byte-identical to the previous implementation (snapshot).
Checklist