blog: Stop Growing Your System Prompt - #4099
Draft
benfrank241 wants to merge 1 commit into
Draft
Conversation
Argues that the default fix for an agent that forgets, adding another line to the system prompt, has a structural ceiling. Opens by conceding what the prompt is genuinely good at: it is immediate, certain to be seen, and inspectable. Then the costs. The prompt is paid on every turn regardless of the task, it crowds out the work itself, and relevance falls as size grows, so adding a rule can make an existing rule less likely to be followed. It also only ever grows, because adding is cheap and deleting requires knowing a line is obsolete. The distinction the post turns on is instructions versus knowledge. Instructions are unconditional and belong in the prompt. Knowledge is unbounded, mostly irrelevant to the current task, and belongs somewhere retrievable, where the cost is per-request rather than permanent. Uses the documented recall budget defaults to make that concrete. Closes on the honest limits: retrieval can miss, adds a dependency, and needs time before it beats a well-written prompt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AE4NCnCKXUzNDF3N9FaQja
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Argues that the default fix for an agent that forgets — adding another line to the system prompt — has a structural ceiling.
Opens by conceding what the prompt is good at: immediate, certain to be seen, inspectable. A strawman would be easy to dismiss.
Then the costs:
The distinction it turns on: instructions vs knowledge. Instructions are unconditional and belong in the prompt. Knowledge is unbounded and mostly irrelevant to the current task, so putting it in the prompt means paying for all of it to get any of it. Uses the documented recall budget defaults (
fixed: low 100, mid 300, high 1000 per method per fact type; plusadaptive) to make the per-request alternative concrete.Closes on honest limits: retrieval can miss, adds a dependency, and needs time before it beats a well-written prompt. "For a script you'll run twice, put it in the prompt."
Composite 87.8/100, specificity 100, readability 94. All cross-links verified live.
Draft — for review, not merge.
🤖 Generated with Claude Code
https://claude.ai/code/session_01AE4NCnCKXUzNDF3N9FaQja