feat: --no-cache-thoughts for reasoning-model multi-turn cache alignment#29

Draft

DavidBellamy wants to merge 11 commits into

prodfrom

feat/no-cache-thoughts-llm360-fork

Commits on May 26, 2026

feat: foundation for --no-cache-thoughts with position-preserved radix entries
DavidBellamy
committed
feat: split-insertion routing + non-contiguous position support
DavidBellamy
committed
feat: ForwardBatch.init_new uses build_extend_positions helper
DavidBellamy
committed
feat: plumb cached_positions from match_prefix through to ForwardBatch
DavidBellamy
committed
feat: decode-path position offsets for non-contiguous cache hits
DavidBellamy
committed
feat: stub split kwarg in non-RadixCache backends (graceful no-op)
DavidBellamy
committed
test: BBQ smoke + --no-cache-thoughts E2E (uncovers chat-template gap)
DavidBellamy
committed
feat: strip <think> priming tail from --no-cache-thoughts cached prompt
DavidBellamy
committed
feat: revert priming-strip; align cached entry with TITO buffer + add TITO E2E
DavidBellamy
committed

Commits on Jun 7, 2026

fix(no-cache-thoughts): page-align relocated answer KV; cap split at committed length
DavidBellamy
committed

Commits on Jun 11, 2026

fix(no-cache-thoughts): initialize KV-copy config for MHA pools
DavidBellamy
committed