fix: count provider-native media in precomputed token budgets

[insoln]

Summary

Count provider-native type: 'media' blocks in both countFormattedMessageTokens and estimateMediaTokensForMessage, using the existing agents SDK counter and subtracting message framing so it is not counted twice.

Problem

Google/Vertex document encoding emits { type: 'media', mimeType: 'application/pdf', data: ... }. The application-level formatted-message counter handles document and file, but skips media. Its fallback reads block[block.type], which is absent for this shape.

As a result, PDF bytes contribute zero tokens to the precomputed indexTokenCountMap. The runtime trusts those supplied counts, so a PDF-heavy request can bypass the configured context budget and reach the provider, which then rejects it for exceeding its context window. This still reproduces on current main.

Changes

• Reuse SDK media accounting rather than introduce another PDF/audio/video estimation formula.
• Preserve existing text, document, image, tool-call, and Claude correction behavior.
• Cover inline PDF, referenced PDF, image, audio, and video media blocks.
• Add an integration regression with precomputed counts and a 190k budget: an oversized synthetic PDF must be rejected before model invocation.

These remain local estimates, not provider-exact token counts. This fixes omitted media accounting; it does not guarantee that every PDF estimate matches provider usage. No configuration, persisted data, or resendFiles policy changes are included.

Verification

• npx jest src/agents/client.spec.ts --runInBand --coverage=false from packages/api: 35 tests passed.
• Existing estimateMediaTokensForMessage.spec.ts and hitl/answers.spec.ts: 95 tests passed (130 total).
• ESLint on both changed files: passed.
• Prettier and full packages/api TypeScript check: passed (the latter required a 6 GB Node heap).
• Regression check: removing the formatted-counter fix makes the integration test fail because the test model is invoked instead of rejecting the oversized request; restoring it passes.

All regression inputs are synthetic. No user conversations or attachments are included.

[github-actions]

👋 Thanks for the contribution! LibreChat merges all changes into dev first — main only moves at release time — so this pull request's base branch was switched from main to dev automatically.

Nothing is needed from you; your commits, reviews and discussion are unchanged. If the diff now shows files you did not touch, rebase onto dev:

git remote add upstream https://github.com/LibreChat-AI/LibreChat.git
git fetch upstream dev
git rebase upstream/dev
git push --force-with-lease

Maintainers: apply the target: main label and restore the base branch if this one genuinely belongs on main.

[Copilot]

Copilot review overview

🟢 Approval recommended

The focused fix preserves existing accounting behavior, includes regression coverage, and has no unresolved findings.

Review effort: Balanced
Findings: None

What changed in this PR

Fixes omitted provider-native media accounting so precomputed token budgets include attachments.

Changes:

• Reuses SDK media estimates in both counters without double-counting message framing.
• Adds media-format coverage and an oversized-PDF regression test.

File	Description
packages/​api/​src/​agents/​client.ts	Counts native media in both token estimators.
packages/​api/​src/​agents/​client.spec.ts	Tests SDK consistency and oversized-request rejection.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.