feat(qwen35): add opt-in DFlash speculative decoding

[CAICAIIs]

Summary

This PR adds an opt-in Qwen3.5 DFlash speculative decoding path behind --dflash-draft-model-path.

The default Qwen3.5 serving path is unchanged unless a draft model path is provided. The initial supported path is intentionally conservative: single active greedy request, no logprobs, single GPU, and normal target decode fallback for unsupported request shapes.

What Changed

• Adds Qwen3.5 DFlash draft model config, loading, memory reservation, scratch buffers, and scheduler state.
• Wires --dflash-draft-model-path through the server for Qwen3.5.
• Adds hidden-state capture during Qwen3.5 prompt prefill for DFlash context.
• Adds speculative draft/verify/commit flow for one active greedy request.
• Handles Qwen3.5 hybrid state carefully: full-attention KV, recurrent state, conv state, and sequence length.
• Uses a conservative commit path that restores canonical state and replays the accepted target span through normal decode.
• Falls back to normal decode for multi-active batches, logprobs, and unsupported runtime cases.
• Rejects unsupported launch combinations such as DFlash with LoRA, KV offload, tensor parallel launch, or decode-overlap.
• Adds scheduler-level coverage to prove the DFlash path is actually entered and matches plain greedy output.

Validation

Ran the following checks:

cargo fmt --all --check
git diff --check

GPU correctness gates:

OPENINFER_TEST_MODEL_PATH=<Qwen3.5-4B> \
OPENINFER_DFLASH_TEST_MODEL_PATH=<Qwen3.5-DFlash> \
cargo test --release -p openinfer-qwen35-4b --features qwen35-4b \
  --test dflash_speculative_gate -- --nocapture --test-threads=1

OPENINFER_TEST_MODEL_PATH=<Qwen3.5-4B> \
cargo test --release -p openinfer-qwen35-4b --features qwen35-4b \
  --test speculative_verify -- --nocapture --test-threads=1

OPENINFER_TEST_MODEL_PATH=<Qwen3.5-4B> \
cargo test --release -p openinfer-qwen35-4b --features qwen35-4b \
  --test e2e_scheduler -- --nocapture --test-threads=1

Server config checks:

cargo test -p openinfer-server --features qwen35-4b qwen35_accepts_dflash -- --nocapture
cargo test -p openinfer-server --features qwen35-4b qwen35_dflash_rejects_decode_overlap -- --nocapture

Claim Boundary

This PR is a correctness and usability slice. It proves that Qwen3.5 can run an explicit DFlash path without changing the default engine path.

It does not claim concurrent throughput improvement yet. Multi-active speculative verify/commit, CUDA Graph capture for verify, and serving benchmark/profile closure are kept as future work.

Future Work

• Add batched multi-active DFlash verify/commit for c4/c8/c16 serving throughput.
• Replace conservative replay commit with a lower-overhead hybrid-state commit path once correctness gates cover prefix accept/reject cases.
• Add CUDA Graph capture for fixed-shape verify spans.
• Add same-commit A/B serving benchmarks for baseline Qwen3.5 vs Qwen3.5+DFlash.
• Add nsys profiles covering draft forward, target verify, commit/replay, sampling, and scheduler overhead.
• Expand public benchmark docs only after same-host data shows stable throughput gains.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d6298790b0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Copilot]

Pull request overview

Adds an opt-in DFlash speculative decoding path for the Qwen3.5 model line, enabled only when a draft model path is provided. The change integrates draft loading, prefill hidden-state capture, speculative draft/verify/commit logic, and conservative fallbacks/rejections so the default Qwen3.5 serving path remains unchanged unless explicitly enabled.

Changes:

• Wire --dflash-draft-model-path through openinfer-server into the Qwen3.5 engine launch and add CLI validation to reject incompatible flags (LoRA, KV offload, TP, decode-overlap).
• Implement Qwen3.5-side DFlash draft model loading, memory reservation, prefill hidden capture, speculative draft/verify/commit flow, and state management (KV + recurrent/conv).
• Add correctness-oriented tests and docs, plus supporting kernel/FFI and KV/page-pool truncation utilities needed for rollback/commit.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
openinfer-server/src/main.rs	Pass dflash_draft_model_path through Qwen3.5 engine startup.
openinfer-server/src/config.rs	Validate/consume DFlash CLI arg for Qwen3.5; add server-side tests.
openinfer-qwen35-4b/Cargo.toml	Register new DFlash scheduler gate test target.
openinfer-qwen35-4b/src/lib.rs	Expose engine startup with optional DFlash draft path; add new modules.
openinfer-qwen35-4b/src/scheduler.rs	Add opt-in DFlash scheduler state and speculative decode integration + fallbacks.
openinfer-qwen35-4b/src/dflash.rs	Implement DFlash draft model execution and context handling.
openinfer-qwen35-4b/src/dflash/config.rs	Parse/validate DFlash draft config vs target config.
openinfer-qwen35-4b/src/dflash/loading.rs	Load DFlash draft weights from safetensors shards.
openinfer-qwen35-4b/src/dflash/reservation.rs	Compute conservative GPU memory reservation for draft + scratch.
openinfer-qwen35-4b/src/dflash/scratch.rs	Allocate/reuse lane-level batched scratch for draft forward.
openinfer-qwen35-4b/src/dflash/state.rs	Track per-request DFlash KV/context state and rollback backups.
openinfer-qwen35-4b/src/unified_forward.rs	Add prefill hidden-state capture plumbing for DFlash context.
openinfer-qwen35-4b/src/prefill.rs	Add verify forward path + capture support and shared verify execution helpers.
openinfer-qwen35-4b/src/verify_buffers.rs	New fixed scratch buffers for target verify spans.
openinfer-qwen35-4b/src/speculative.rs	Add executor-level speculative verify API and commit/rollback logic.
openinfer-qwen35-4b/src/weights.rs	Reserve memory for DFlash and add embedding helper for draft path.
openinfer-qwen35-4b/src/recurrent_state.rs	Add device-to-device recurrent/conv state copy helper.
openinfer-qwen35-4b/src/recurrent.rs	Relax scratch size assertions to support reused larger buffers.
openinfer-qwen35-4b/src/prefill_buffers.rs	Track scratch capacity and add set_rows + capacity helpers.
openinfer-qwen35-4b/src/ops.rs	Re-export additional core ops helpers used by capture/verify paths.
openinfer-qwen35-4b/src/kernel_plan.rs	Update kernel-plan doc pointer to new prefill function name.
openinfer-qwen35-4b/src/executor.rs	Refactor slot compaction via graph copy helper; add debug state summaries.
openinfer-qwen35-4b/src/batch_decode_graph.rs	Add slot<->slot and slot<->state recurrent/conv state copy helpers.
openinfer-qwen35-4b/tests/dflash_speculative_gate.rs	Scheduler-level DFlash gate test (lossless vs plain greedy + fallback checks).
openinfer-qwen35-4b/tests/speculative_verify.rs	Executor-level verify correctness tests vs decode/prefill oracles.
openinfer-kernels/src/ops/attention.rs	Add single-request causal-window prefill attention op (FFI wrapper).
openinfer-kernels/src/ops.rs	Re-export new attention op.
openinfer-kernels/src/ffi/shared.rs	Add FFI signature for causal-window prefill kernel.
openinfer-kernels/src/ffi/qwen35.rs	Add FFI signature for batched QK norm + RoPE + KV write prefill prep.
openinfer-kernels/csrc/shared/paged_attention.cu	Implement causal-window single-prefill kernel binding.
openinfer-kernels/csrc/qwen35/prefill_attention_hd256.cu	Implement batched Q/K norm+RoPE+KV write prep for Qwen3.5 verify.
openinfer-core/src/page_pool.rs	Add OwnedPagePermit::truncate for returning tail pages early.
openinfer-core/src/kv_pool.rs	Add KvState::truncate_to for KV rollback/commit flows.
openinfer-core/src/ops.rs	Re-export new causal-window prefill op from kernels.
docs/models/qwen35/roadmap.md	Update Qwen3.5 roadmap to mention opt-in DFlash MVP + freshness date.
docs/models/qwen35/dflash-speculative-decoding.md	New doc describing DFlash scope, enablement, and validation gates.
docs/index.md	Add routing-table entry for the new DFlash doc.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 85462f25f5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".