PR: feat(qwen3): n-gram speculative decoding#349

Open

wjinxu wants to merge 34 commits into

openinfer-project:mainfrom

wjinxu:feat/qwen3-ngram-proposer

Commits on Jun 11, 2026

feat(qwen3): add n-gram (prompt-lookup) speculative proposer

wjinxu
and
cursoragent
committed
feat(qwen3): add greedy speculative acceptance + spec config

wjinxu
and
cursoragent
committed
feat(kv-cache): expose speculative schedule/apply on RequestKv

wjinxu
and
cursoragent
committed
docs(qwen3): design note for n-gram speculative decoding

wjinxu
and
cursoragent
committed
feat(qwen3): wire speculative verify forward into the executor

wjinxu
and
cursoragent
committed
docs(qwen3): mark speculative verify forward landed in design note

wjinxu
and
cursoragent
committed
test(qwen3): GPU validation that n-gram speculative decode is lossless

wjinxu
and
cursoragent
committed
docs(qwen3): record GPU-validated lossless property for ngram spec decode

wjinxu
and
cursoragent
committed
feat(qwen3): expose execute_speculative on the ModelExecutor trait

wjinxu
and
cursoragent
committed
feat(qwen3): wire n-gram speculative decode into the scheduler

wjinxu
and
cursoragent
committed
docs(qwen3): record scheduler wiring landed; remaining = config knob + perf

wjinxu
and
cursoragent
committed
feat(qwen3): env switch for n-gram speculative decode

wjinxu
and
cursoragent
committed
test(qwen3): n-gram speculative decode speedup benchmark

wjinxu
and
cursoragent
committed
refactor(qwen3): abstract speculative decode behind a proposer trait

wjinxu
and
cursoragent
committed
docs(qwen3): refresh speculative module header for the proposer abstraction

wjinxu
and
cursoragent
committed
refactor(qwen3): push n-gram env parsing down to NgramConfig::from_env

wjinxu
and
cursoragent
committed
refactor(qwen3): honest speculative docs + tighten cohesion

wjinxu
and
cursoragent
committed
fix(qwen3): make speculative step transactional and budget-capped

wjinxu
and
cursoragent
committed
fix(qwen3): gate speculation to greedy, no-logprobs requests

wjinxu
and
cursoragent
committed

Commits on Jun 24, 2026

Commits on Jun 25, 2026

Commits on Jun 26, 2026

Merge remote-tracking branch 'origin/main' into feat/qwen3-ngram-proposer
wjinxu
committed