Skip to content

Latest commit

 

History

History
357 lines (271 loc) · 15.3 KB

llm-wiki-pattern-tutorial.md

File metadata and controls

357 lines (271 loc) · 15.3 KB

The LLM Wiki Pattern: Building a Compounding Knowledge Base with Claude Code

A hands-on, self-paced companion to the talk. Work through it at your own pace; each step takes a few minutes.

If you've ever started a fresh Claude session and had to re-explain the same background before you could ask your real question — who's on the team, what a project's shorthand means, a decision made three sessions ago — you've felt the pain this pattern solves. Every new session starts from zero unless that context lives somewhere the model can pick back up.

Andrej Karpathy proposed an answer in a widely-shared 2026 gist: the LLM Wiki pattern. Instead of re-explaining context every session (or, if you're using retrieval-augmented generation, re-retrieving and re-synthesizing it from raw chunks every query), you have an LLM compile your source material once into a structured, cross-referenced Markdown wiki — and then query that compiled artifact directly. Think of it like compiling code: the compile step is expensive, but you pay for it once, and every run after that is fast and cheap.

This tutorial walks through building your own LLM wiki using Claude Code, entirely in Markdown, with no vector database. We'll build a small one together from a set of sample documents, then look at what a mature, real-world wiki grows into.

Get the code from GitHub before you start — BCM-HGSC/llm-wiki-pattern-tutorial — Step 1 below walks through cloning it.


The core idea

The pain this solves has two faces. One is session-to-session: without somewhere for it to live, context you and Claude worked out together evaporates the moment the session ends, and you re-explain it next time. The other, if you're doing retrieval-augmented generation, is query-to-query: RAG chunks your documents, embeds them, and on each query retrieves whatever's most similar — which isn't the same as curated or organized, and the retrieval starts over every time.

The LLM Wiki pattern:

  1. Feed your raw sources to an LLM once
  2. It compiles them into interlinked Markdown pages — one per concept, with summaries, cross-references, provenance, and flagged contradictions
  3. You query the compiled wiki directly, pointing Claude at just the pages a question needs
  4. When you add a new source, the LLM updates the wiki incrementally — it doesn't start over

Two things fall out of this. First, selective retrieval without an embedding pipeline — because the wiki is already organized by concept, you (or Claude) can pull in exactly the pages a question needs instead of dumping every raw source into context. Second, and the bigger everyday win: context stops evaporating between sessions. The wiki is where the business context you've already explained once goes to stay explained — you stop re-introducing the same background at the start of every conversation.


The three-layer architecture

my-wiki/
├── raw/          # Layer 1: immutable source material
├── wiki/         # Layer 2: LLM-generated, cross-referenced pages
└── CLAUDE.md     # Layer 3: instructions that govern how the wiki is maintained

Layer 1 — Raw sources. PDFs, articles, transcripts, notes, screenshots. These are never edited after they land in raw/. They're your ground truth — the wiki always traces back to them.

Layer 2 — The wiki. Markdown pages, one per concept, each containing a summary, wiki-links ([[other-concept]]) to related pages, and provenance notes pointing back to the raw source. This is the layer you actually query.

Layer 3 — Instructions (CLAUDE.md). A short Markdown file that tells the agent how to behave when compiling or updating the wiki: naming conventions, page structure, how to handle contradictions, when to create a new page vs. extend an existing one, and how to maintain the index and log.

A note on this third layer. The original pattern calls this the "schema," and some writeups implement it as a bare schema.md. But Karpathy's own framing describes it as a CLAUDE.md or AGENTS.md file — the standard instructions file coding agents already read automatically. We use CLAUDE.md here because it is both more useful (Claude Code loads it on its own at session start) and truer to the original. It's the wiki's operating manual, not just a data schema.

This pattern works best for personal-scale knowledge bases — Karpathy's guidance is roughly under 100,000 tokens of compiled wiki content, comfortably within a single context window.


What we'll build

We'll compile a small knowledge base for a fictional volunteer nonprofit, the Ridgeline Community Observatory (RCO), from four short source documents:

File Type What it is
membership-policy.pdf PDF Membership tiers, dues, equipment-reservation rules
visit-us.html HTML The observatory's public "Visit Us" web page
board-meeting-2026-06.txt transcript Notes from a monthly board meeting
treasurer-note.eml email The treasurer's note to members

These four files live in the sample-docs/ folder next to this tutorial. They deliberately disagree in one place — watch for it; surfacing that disagreement is the payoff.


Step 1: Clone the repo

Everything you need — this tutorial, sample-docs/, starter/, and the reference demo-wiki/ — lives in one GitHub repo:

git clone https://github.com/BCM-HGSC/llm-wiki-pattern-tutorial.git
cd llm-wiki-pattern-tutorial

Every path in the rest of this tutorial (sample-docs/, starter/, ...) is relative to this cloned folder.

Step 2: Set up the folder

You can copy the provided starter/ scaffold, which already contains a CLAUDE.md and empty raw/ and wiki/ folders:

cp -r starter my-wiki
cd my-wiki

Or build it by hand:

mkdir -p my-wiki/raw my-wiki/wiki
cd my-wiki
touch CLAUDE.md

Step 3: Write your instructions (CLAUDE.md)

This is the one piece of hand-written configuration, and the heart of the pattern. It doesn't need to be long. The starter CLAUDE.md covers the essentials; the parts that matter most are:

  • raw/ is immutable — the wiki always traces back to it, never overwrites it.
  • Page structure — each page has a one-paragraph summary, a ## Details section, a ## Related section with [[wiki-links]], and a ## Sources section citing the raw file(s) it came from.
  • Contradictions — if a new source contradicts an existing page, don't overwrite silently; add a ## Contradictions section noting both claims and their sources.
  • Index and log — maintain wiki/index.md (a flat list of pages) and wiki/log.md (an append-only record of each compile/update).

Open the starter CLAUDE.md and skim it now, so you know what you're asking Claude to follow.

Step 4: Drop in your raw sources

Copy the four sample documents into raw/:

cp ../sample-docs/* raw/

No preprocessing needed — Claude Code reads PDFs, HTML, and text files directly.

Step 5: Launch Claude Code and compile

From inside my-wiki:

claude

Because CLAUDE.md is loaded automatically, your compile prompt can be short:

Compile this wiki. Read every file in raw/ and follow CLAUDE.md exactly:
create one page per concept in wiki/, cross-link related pages, cite sources,
and write index.md and log.md when you're done.

Claude Code will read the instructions, process each raw file, and write out the wiki pages — cross-linking related concepts as it goes. Expect pages roughly like membership.md, equipment.md, events.md, and governance.md, plus index.md and log.md.

Step 6: Look for the contradiction

Open wiki/membership.md. The board raised individual dues from $40 to $50, effective 2027 — but the policy PDF and the public web page still say $40. A good compile will not silently pick one. It should carry a ## Contradictions (or similarly flagged) section noting:

  • the PDF and web page say $40 (older sources),
  • the meeting transcript and treasurer's email say $50 effective 2027-01-01,
  • and that the older public-facing docs likely just haven't been updated yet.

If Claude did silently pick one value, that's a teaching moment: tighten the contradiction rule in CLAUDE.md and recompile. This is how you tune a wiki.

Step 7: Query the wiki

Querying is just... asking. Point Claude at wiki/, not raw/:

Based on wiki/, what will individual membership cost in 2027, and what's the
source for that? What equipment can members reserve, and what are the limits?

Because the wiki is pre-organized, cross-referenced, and carries its own provenance, Claude can pull in just the pages a question needs instead of reasoning over every raw source — faster, and less prone to missing-context mistakes.

Step 8: Add a new source incrementally

This is the compounding part. Suppose a new flyer arrives announcing a public telescope-building workshop. Create a quick source file:

cat > raw/2026-07-workshop-flyer.txt <<'EOF'
RCO Public Workshop — Build Your Own Telescope
Saturday, September 19, 2026, 10 AM–3 PM, Millbrook Library.
Free for members, $15 for non-members. Materials provided.
Led by Tom Fletcher, Equipment Lead.
EOF

Then prompt:

A new source was added to raw/: 2026-07-workshop-flyer.txt. Update the wiki
incrementally per CLAUDE.md — do not rebuild from scratch. Update only the
pages this affects, refresh index.md, and append a log.md entry.

Notice what Claude doesn't do: it doesn't re-read everything and regenerate all pages. It touches events.md, maybe governance.md (Tom's role), the index, and the log. Each addition makes the wiki richer without redoing prior work.

Step 9: Commit

Everything is Markdown, so version control is free and meaningful:

git init && git add -A && git commit -m "Compile RCO wiki from initial sources"

Commit after each compile or update. You get a full, reviewable history of how your knowledge base evolved — including how contradictions were surfaced and resolved.


The structure is the invariant — everything else adapts

Here's the most important lesson, and the reason this tutorial uses CLAUDE.md instead of a rigid schema.md: the three layers are the pattern. Everything else is a design choice you make for your domain. Karpathy's gist was deliberately abstract and expected you to instantiate your own version.

To see how far an instantiation can drift while staying faithful to the three tiers, look at a mature real-world example — HGSC's own team knowledge base, BCM-HGSC/submissions-kb (an HGSC-internal repo; sign in with your HGSC GitHub account to view it). It keeps all three layers but looks nothing like our starter:

Our tutorial wiki A mature instantiation (submissions-kb)
CLAUDE.md with basic rules CLAUDE.md as a full operating manual: voice/style, an inbox-processing protocol, a source-type disposition table, sanitization rules, curation checkpoints
flat wiki/ topical top-level dirs (people/, systems/, terminology/, projects/, operations/), each with its own README index
raw/ sources/ with an annotated INDEX.md for provenance, plus a gitignored inbox/ staging area for un-processed material
log.md dropped — git history serves that role
a [[wiki-link]] convention Obsidian-backed links, relative-path cross-references, strict ISO dates

Same skeleton, completely different body. When you build your own, start simple like our starter and let structure emerge from what your material actually needs — don't copy either example wholesale.

Read its CLAUDE.md and README.md to see how an instructions layer grows up.

A confession about how submissions-kb actually grew

Worth being honest about, since it's a real deviation from the pattern described above: a lot of submissions-kb's content didn't arrive by dropping a file into sources/ or inbox/ first. It arrived through conversation. Someone would explain something to Claude mid-session — a decision, a piece of context about how a system works — and, as part of that same session, Claude would go update an existing wiki page or create a new one with the new information, right then.

That's not what a pure instantiation of the pattern would do. Followed strictly, the new material should have gone into inbox/ (or raw/, in our starter's terms) as its own dated source file first, and then a separate compile step should have folded it into wiki/ — preserving the raw/wiki separation and the provenance trail. In practice, the two steps collapsed into one, in the middle of a conversation, because that's simply the more convenient path when you're already talking to Claude about the thing you want it to remember.

It still worked — the wiki got the information either way — but it's a real gap: some claims in submissions-kb trace back to "a conversation on some date" rather than a filed, citable source. If you're building your own wiki and care about strict provenance, decide up front whether conversational updates are allowed to write to wiki/ directly, or whether your CLAUDE.md should require Claude to first save new information as a source before compiling it in.


Practical tips

  • Keep raw/ immutable. To correct a source, add a corrected version rather than editing the original — you want an audit trail.
  • Use git. Since everything is Markdown, commit after each compile step. You get version history on your knowledge base for free.
  • Watch the token budget. Once your wiki approaches six figures of tokens, split it into sub-wikis by topic or summarize more aggressively in CLAUDE.md.
  • Don't skip contradictions. The temptation is to let the LLM quietly overwrite old claims with new ones. Force it to surface conflicts — that's often the most valuable output of the whole exercise.
  • Tune the instructions, not the output. If the wiki comes out wrong, fix CLAUDE.md and recompile rather than hand-editing pages. The instructions are the program; the pages are its output.

When this pattern isn't the right fit

  • Multi-user or enterprise settings. There's no access control, no concurrency handling, no transactional guarantees — it's a single-user, single-writer pattern. (submissions-kb handles this with git review/merge as the checkpoint, not the tool itself.)
  • Very large corpora. Beyond personal-scale (roughly 100K+ tokens of compiled wiki), you likely still want retrieval underneath, possibly with the wiki as a curated layer on top.
  • Highly dynamic data. If your source material changes faster than you can review compiled updates, the "compile once" model starts to strain.

Further reading

  • Andrej Karpathy's original gist introduced the pattern in April 2026 and popularized the "compile, don't retrieve" framing.
  • Search "Karpathy LLM wiki" for community writeups and at least one open-source desktop implementation that extends the pattern with a full UI.
  • BCM-HGSC/submissions-kb (HGSC-internal), for a lived-in instantiation of the pattern.