metric(qwen3): report prefix-cache query/hit

docs/index.md

@@ -143,7 +143,7 @@ Organized by domain (model line / subsystem / playbook / lesson) instead of by l
 | `subsystems/frontend/simulated-inference-engine.md` | CPU-only simulated model crate for vLLM/OpenAI frontend and `vllm bench serve` validation without CUDA, real model weights, or real-model performance claims. |
 | `subsystems/frontend/cpu-profiling-baseline.md` | Frontend CPU profiling baseline using `openinfer-sim` with fixed TTFT=5ms/TPOT=12ms: 200 req / concurrency=16 shows ~150ms TTFT overhead (no dominant hotspot), heap allocation ~10%, stream polling ~7.5%, IPC ~1%; reproducible benchmark command and perf evidence documented. |
 | `subsystems/frontend/startup-time.md` | Qwen3-4B warm startup-to-ready 3.25s → ~1.45s: frontend tokenizer load runs concurrently with the engine load (HTTP still binds only after the engine registers), and the source safetensors mmap is kept alive to dodge ~0.4s of munmap stalling the next cudaMalloc. |
-| `subsystems/frontend/prometheus-metrics.md` | `/metrics` for the Qwen3 line: request histograms come free from bridge events/PrefillStats; engine gauges (running/waiting/kv usage) ride the scheduler LoadSnapshot watch as stats-only batches. Which metrics deliberately read zero, and the recipe for wiring another model line. |
+| `subsystems/frontend/prometheus-metrics.md` | `/metrics` request histograms come from bridge events/PrefillStats; scheduler stats ride `LoadSnapshot`. Qwen3 now reports real prefix-cache query/hit counters via monotonic totals differenced by the bridge, plus running/waiting/KV gauges. |
 | `subsystems/frontend/dashboards/README.md` | Grafana 10.4-validated dashboard for OpenInfer's live `/metrics` surface: HTTP traffic, request outcomes, scheduler/KV state, token throughput, and request latency. |
 
 ## subsystems / correctness

docs/subsystems/frontend/prometheus-metrics.md

@@ -1,29 +1,23 @@
-# Prometheus /metrics via the vLLM frontend
+# Prometheus `/metrics` via the vLLM frontend
 
-**TL;DR:** `/metrics` exposes request histograms for every model and engine gauges for schedulers that publish `LoadSnapshot`: Qwen3 uses one engine, GLM5.2 EP8/DP8 uses eight rank-local engines, and GLM5.2 TP8 uses one logical engine. The bridge forwards each partition's stats under the same identity the vLLM frontend uses for least-load routing.
+**TL;DR:** `/metrics` exposes request metrics for every model. Schedulers that publish `LoadSnapshot` also expose load and KV gauges; Qwen3 additionally reports real prefix-cache query and hit token counters.
 
 Last touched: 2026-07
 
-## How the numbers flow
+## Metric sources
 
-Two independent paths feed the upstream Prometheus registry (`vllm-metrics`, served by `vllm-server` at `/metrics` with its HTTP middleware counters):
+- Request metrics come from frontend request events and include latency histograms, prompt/generated token totals, and request outcomes.
+- Scheduler metrics come from `LoadSnapshot`. They are present only for model schedulers that publish a load watch.
+- Each logical scheduler partition has its own `engine` label.
 
-1. **Per-request path (works for every model crate).** The bridge stamps each request's first output with `Queued`/`Scheduled` timestamps and `PrefillStats` (prompt/computed/cached token split). The upstream `RequestMetricsTracker` turns those into `time_to_first_token_seconds`, `inter_token_latency_seconds`, `request_queue_time_seconds`, `prompt_tokens_total`, `generation_tokens_total`, `request_success_total`, `prompt_tokens_by_source_total`, … unconditionally — `disable_log_stats` only gates the periodic *text* logger, not Prometheus.
-2. **Engine-gauge path (needs one `LoadSnapshot` watch per scheduler partition).** The scheduler publishes `LoadSnapshot { kv_used_blocks, kv_total_blocks, num_running_reqs, num_waiting_reqs }` at scheduler boundaries; one bridge identity per partition forwards its snapshot as a stats-only `RequestBatchOutputs`. The enclosing `engine_index` is both the routing identity and the Prometheus `engine` label. Watches coalesce to ≤1 message per scheduler step, and the scheduler's final idle publish settles the gauges back to 0.
+Qwen3 publishes monotonic prefix-cache totals. The frontend converts them to interval deltas before passing them to vLLM's Prometheus collector, so coalesced load-watch updates do not lose or double-count increments.
 
-For a single-partition model, `EngineHandle::with_load_watch` keeps the original one-engine contract. A partitioned scheduler uses `with_load_watches`, and the frontend launch declares the same engine count; a mismatch fails startup. GLM5.2 EP8 therefore registers engines 0–7, each bound to its own pending queue and KV pool. TP8 registers only engine 0 because its eight workers mirror one logical request stream.
+`vllm:prefix_cache_queries_total` counts prompt tokens submitted to an actual first-prefix lookup. `vllm:prefix_cache_hits_total` counts tokens restored from matching full cache blocks. Echo requests, cache-disabled requests, and later chunks of the same prefill do not add queries. Qwen3 cache blocks contain 16 tokens.
 
-Measured cost is noise in both covered configurations:
+## Check prefix-cache counters
 
-- Qwen3 TPOT: 10.6387 ms (main) vs 10.6395 ms (metrics branch) over 828 tokens.
-- GLM5.2 EP8, three-run median at concurrency 64: 1268.58 vs 1264.82 output tok/s (-0.30%); TPOT p50 41.76 vs 41.35 ms.
+Send the same prompt of at least 16 tokens twice, then scrape `/metrics` and inspect `vllm:prefix_cache_queries_total` and `vllm:prefix_cache_hits_total`. Both counters are cumulative. Queries should increase for each request; hits should increase when the repeated prompt reuses cached blocks.
 
-## What deliberately reads zero (state at capture time)
+## Coverage limits
 
-- `prefix_cache_queries/hits` and the by-reason waiting split (`reason="deferred"` is driven by a skipped-request counter we don't report; all waiting shows as `reason="capacity"`).
-- Spec-decode counters, per-GPU FLOPs/bytes estimates, KV-block residency histograms, cudagraph stats — the bridge sends `SchedulerStats::default()` for these fields.
-- Every model crate whose scheduler doesn't publish a `LoadSnapshot` watch (qwen35, deepseek, kimi) gets path 1 only; its engine gauges are absent, not lying-zero — the bridge skips the stats task for that partition when no watch exists.
-
-## Next step
-
-Wire `LoadSnapshot` publishing into the qwen35 scheduler (the other schedulers can follow the same recipe), and report real prefix-cache query/hit counters instead of zeros. A future partitioned model must expose its logical scheduler partitions instead of averaging them behind engine 0.
+Unsupported scheduler fields remain at their upstream defaults, including speculative-decoding counters, GPU FLOP/byte estimates, KV-residency histograms, and CUDA Graph statistics. Models without a `LoadSnapshot` watch expose request metrics but no scheduler gauges.

openinfer-engine/src/engine.rs

@@ -351,6 +351,16 @@ pub struct LoadSnapshot {
     pub num_running_reqs: u64,
     /// Requests admitted but not yet running (KV pressure, prefetch wait).
     pub num_waiting_reqs: u64,
+    /// Prompt tokens queried against the local prefix cache since engine start.
+    ///
+    /// This is a monotonic total because the snapshot rides a coalescing watch
+    /// channel. Consumers that feed delta counters must difference consecutive
+    /// snapshots rather than forwarding this value directly.
+    pub prefix_cache_queries_total: u64,
+    /// Prompt tokens served from the local prefix cache since engine start.
+    /// Monotonic for the same coalescing-safe reason as
+    /// [`Self::prefix_cache_queries_total`].
+    pub prefix_cache_hits_total: u64,
 }
 
 /// One full KV block that just became reusable from this engine's prefix cache.

openinfer-glm52/src/scheduler/load.rs

@@ -35,6 +35,7 @@ pub(super) fn publish_load(
             kv_total_blocks: kv_total_blocks as u64,
             num_running_reqs: slots[rank].iter().flatten().count() as u64,
             num_waiting_reqs: pending[rank].len() as u64,
+            ..LoadSnapshot::default()
         });
     }
 }

openinfer-qwen3/Cargo.toml

@@ -17,7 +17,7 @@ comfy-table = { workspace = true, optional = true }
 crossbeam-channel = { workspace = true }
 cudarc = { workspace = true }
 openinfer-kv-cache = { workspace = true }
-openinfer-kv-offload = { workspace = true }
+openinfer-kv-offload = { workspace = true, optional = true }
 fastrace = { workspace = true }
 half = { workspace = true }
 hex = { workspace = true, optional = true }
@@ -31,6 +31,7 @@ tokio = { workspace = true, features = ["sync"] }
 toml = { workspace = true, optional = true }
 
 [features]
+kv-offload = ["dep:openinfer-kv-offload"]
 kernel-call-trace = ["openinfer-core/kernel-call-trace"]
 test-fixtures = []
 kernel-report = [