Skip to content

metric(qwen3): report prefix-cache query/hit#669

Open
Kokoro2336 wants to merge 4 commits into
openinfer-project:mainfrom
Kokoro2336:feat/issue-603
Open

metric(qwen3): report prefix-cache query/hit#669
Kokoro2336 wants to merge 4 commits into
openinfer-project:mainfrom
Kokoro2336:feat/issue-603

Conversation

@Kokoro2336

@Kokoro2336 Kokoro2336 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

Resolves #603

Report prefix-cache query/hit

Type of Change

  • New feature (non-breaking change which adds functionality)

Checklist

  • My code follows the style guidelines of this project (see docs/conventions/coding-style.md).

  • I have performed a self-review of my own code.

  • I have formatted my commits according to Commitizen conventions.

  • I have run the local test suite and all tests pass (see CLAUDE.md).

@Kokoro2336

Copy link
Copy Markdown
Contributor Author
ba1316c6ee82ac87d726104276f97fc9

I've made a simple test on the feature, sending the same prompt twice and filtering out queries/hits.

@Kokoro2336
Kokoro2336 marked this pull request as ready for review July 17, 2026 06:29
@Kokoro2336
Kokoro2336 marked this pull request as draft July 17, 2026 13:31
@Kokoro2336
Kokoro2336 marked this pull request as ready for review July 17, 2026 15:12

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6bfd50749b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +2184 to +2185
"Qwen3 KV offload is not compiled in; rebuild with `--features qwen3-kv-offload` \
for the server or `--features kv-offload` for openinfer-qwen3"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Expose a server feature for Qwen3 KV offload

When --kv-offload is passed to the server after this change, openinfer-qwen3 is normally compiled via the server's qwen3 feature, which only enables dep:openinfer-qwen3 and does not forward openinfer-qwen3/kv-offload. This fallback therefore rejects the run, and the suggested --features qwen3-kv-offload feature is not defined in openinfer-server/Cargo.toml (I checked the server feature list). Please add/forward a server feature or change the guidance; otherwise Qwen3 offload deployments cannot be built from the server path.

Useful? React with 👍 / 👎.

@xiaguan xiaguan self-assigned this Jul 18, 2026
@xiaguan

xiaguan commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

This PR makes openinfer-kv-offload optional without forwarding the feature from openinfer-server, so the default Qwen3 server rejects --kv-offload while recommending a nonexistent qwen3-kv-offload feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

metrics: report real prefix-cache query/hit counters from the qwen3 scheduler

2 participants