Skip to content

fix(pd): correct DeepSeek V4 layerwise cache handoff#660

Draft
lucifer1004 wants to merge 2 commits into
lightseekorg:mainfrom
lucifer1004:agent/pd-grouped-cache-transfer
Draft

fix(pd): correct DeepSeek V4 layerwise cache handoff#660
lucifer1004 wants to merge 2 commits into
lightseekorg:mainfrom
lucifer1004:agent/pd-grouped-cache-transfer

Conversation

@lucifer1004

@lucifer1004 lucifer1004 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Motivation

DeepSeek V4 stores SWA, compressed KV, compressor state, and indexer state in independently paged cache groups. PD previously applied the legacy KV page indices to every contiguous buffer, so a Mooncake transfer could succeed while copying data between the wrong physical pages.

The DeepSeek V4 attention path also bypassed the common layerwise cache-readiness publication used by PD transfers.

Changes

  • publish cache readiness from the DeepSeek V4 attention path
  • associate each DeepSeek V4 contiguous buffer with its cache group
  • carry source and destination group page tables and logical base offsets through the Mooncake control path
  • match source and destination pages by logical position before constructing RDMA ranges
  • preserve the existing flat-index path for ungrouped cache pools
  • mirror group-aware address construction in the async manager without changing backend selection or wiring

Compatibility

Ungrouped cache pools retain their existing transfer behavior. Grouped-cache transfers require both peers to run the updated code; missing grouped metadata fails explicitly instead of copying incorrect pages.

This PR does not restore or enable mooncake_async executor wiring. Current executors continue to select the synchronous Mooncake managers.

Validation

  • prek -C tokenspeed run --all-files
  • 172 relevant runtime tests passed, with 1 skipped and 2 known SM120 CUDA cases deselected
  • sync Mooncake layerwise PD qualified with TP2/EP2 and one prefill plus one decode replica
  • full GSM8K: 93.63% flexible exact match and 93.48% strict exact match over 1,319 samples
  • random 8K input / 1K output at concurrency 1 through 32: 252/252 requests completed
  • no transfer, memory-registration, bad-address, CUDA, or OOM errors observed

The model qualification used the SM120 enablement stack from #648 plus this PD patch; those kernel changes are intentionally excluded from this PR.

Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant