Copilot: surface AI credit usage for token-based-billing seats - #2647
Copilot: surface AI credit usage for token-based-billing seats#2647KSEGIT wants to merge 13 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aaeb632f41
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Codex review: needs maintainer review before merge. Reviewed August 16, 2026, 11:45 AM ET / 15:45 UTC. ClawSweeper reviewWhat this changesAdds a per-Copilot-account AI-credit entitlement that turns token-billed seat usage into a menu-card progress row and rewrites cached rows immediately when the entitlement changes. Merge readinessKeep open for maintainer product sign-off. The implementation is coherent, has sufficient live UI proof, and no discrete correctness or security defect was found; the remaining decision is whether the menu-card-only, user-entered credit ceiling is the intended Copilot contract. Priority: P2 Review scores
Verification
How this fits togetherCodexBar turns Copilot API usage snapshots into provider-detail rows shown in the macOS menu card. This change combines the existing credit counter with a user-entered per-seat ceiling to render a ratio without inventing a quota. flowchart LR
A[Copilot usage response] --> B[Usage snapshot]
C[Selected Copilot account] --> D[Seat credit ceiling]
B --> E[Credit detail row]
D --> E
E --> F[Cached snapshot repair]
F --> G[Menu card]
Decision needed
Why: The implementation avoids fabricated API data and does not change auth scope, but it adds a persisted setting and establishes a new provider-display contract that the repository vision reserves for sign-off. Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Approve the narrow menu-card contract if that is the desired product surface, keeping the ceiling explicitly user-entered and leaving broader metric publication for a separately scoped follow-up. Do we have a high-confidence way to reproduce the issue? Not applicable as a bug report. The changed behavior itself has a high-confidence after-fix path: four inspected live screenshots show the real Copilot menu card online and offline at two configured ceilings. Is this the best way to solve the issue? Unclear pending product sign-off. The implementation is a maintainable way to avoid inventing a GitHub quota, but only a maintainer can decide whether menu-card-only visibility is the intended scope. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 355f8443a55c. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (33 earlier review cycles; latest 8 shown)
|
|
Thanks for this, and for the excellent original report in #2593 — you found both the |
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
Re The upstream data is real, not inferredLive from a Copilot Business seat (redacted), via the endpoint CodexBar already calls: "quota_snapshots": {
"premium_interactions": { "unlimited": true, "entitlement": 0, "remaining": 0,
"percent_remaining": 100.0, "credits_used": 31,
"token_based_billing": true, "overage_permitted": true },
"chat": { "unlimited": true, "entitlement": 0, "credits_used": 0 },
"completions": { "unlimited": true, "entitlement": 0, "credits_used": 0 }
},
"organization_login_list": ["<ORG>"],
"copilot_plan": "business", "token_based_billing": true,
"quota_reset_date": "2026-09-01"Note And the org endpoint: { "usageItems": [
{ "product": "Copilot", "sku": "Copilot AI Credits", "model": "Code Review model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 31.13 },
{ "product": "Copilot", "sku": "Copilot Cloud Agent", "model": "Coding Agent model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 49.97 } ] }31.13 + 49.97 = 81.10, which matches the "81 / 6,000 AI credits" numerator on the org billing page exactly. That agreement is the strongest evidence I have that the org lane reads the right thing. What the card actually rendersAsserted on exact strings through the real
The org tests drive What I can't evidence yetNo screenshot. Producing one means building this branch, packaging it, and running it against a real Copilot Business account — which per AGENTS.md is exactly the kind of validation that can raise Keychain prompts, so I didn't do it unasked. Happy to if you want it. Two of three surfaces are untouched. I've added a table to the PR description rather than leave it implied: the menu card shows credits; the menu bar icon, the widget, and So today a Business token-billing user still gets an empty icon and has to open the menu. I can wire the icon up in this PR — publishing credits as a Since the PR was opened
|
clawsweeper
Bot
added
rating: 🦪 silver shellfish
bfe422a to
b7406fb
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 auth-provider
|
Current-head live capture, taken from a build of this branch running against my Copilot Business seat:
What this shows: a real token-billed Business seat rendering Being straight about its limits: this is the end state, not the transition. It does not by itself distinguish the cached text→bar repair from an ordinary successful refresh — the card reads "Updated just now", so a fetch had just landed. The offline sequence that isolates the cached rewrite (clear the entitlement with the network down, watch the row collapse to plain text and come back as a bar while the last-updated timestamp never advances) is still to come, and I'll post it as a follow-up. Posting this now because it does close one gap the last review named: the live bar existing at the current head rather than only at |
|
Current-head behavior evidence, all four frames captured from a build of this branch against my live Copilot Business seat ( 1 — Bar renders at current head, online.
2 — The cached row survives a failed refresh. Network off; the card shows "The Internet connection appears to be offline" and the credit row is still rendered as a bar at
3 — A changed entitlement re-renders the row. Allowance changed
4 — And that survives offline too. Network off again; offline banner present, row still
What these do and don't establishThey establish, on the current head: the bar renders on a real token-billed Business seat; the cached row survives a failed refresh with its denominator intact at two different entitlements; and changing the entitlement re-renders the ratio. They do not isolate the in-place cached rewrite as tightly as I'd like. Frames 2 and 4 are both offline but sit either side of a brief reconnect, so there is no single unbroken offline stretch spanning an entitlement change. I'd rather say that plainly than let the sequence imply more than it shows. Worth noting why the literal "clear it and watch it return to text" capture is awkward on my setup specifically: the settings field resolves as Unit coverage for the exact text↔bar transition is in @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
clawsweeper
Bot
added
proof: sufficient
2d25ce6 to
31281db
Compare
|
@steipete — short status, since my last "ready to merge" note is well out of date and the thread has grown a lot since. The proof blocker is cleared. Review now reads ready for maintainer look with no actionable findings, real behavior verified from current-head captures on my live Business seat, and the earlier release-note issue fixed. What's left is the product question rather than anything mechanical: whether a menu-card-only, per-account user-entered credit ceiling is the contract you want, with the menu-bar icon, widget, and CLI deliberately left untouched. The recommendation on the review is to approve that seat-only scope and treat the other surfaces as follow-up, but that's your call, and I'd rather have it settled than guess. The branch currently conflicts with main — this repo moves fast enough that it drifts within a day, and the conflicts are the mechanical kind (an allowlist line-anchor file and a changelog list that both moved on main). I've deliberately not been force-pushing rebases on a loop while it waits for a look. Say the word, or just leave a review, and I'll rebase and have it green within the hour. No rush from my side, and no more pings from me on this — happy for it to sit until you have time. |
Remove the opt-in organization billing lane pending an approved authorization model for org-wide billing reads with device-flow tokens. Rewrite the cached seat credit row immediately when the entitlement changes so a stale denominator or bar never survives a failed refresh.
Text-only credit rows carried no structured usage, so entering an entitlement during an offline or failed refresh left the cached row without its bar. Stamp usageValue on the row at fetch time and use it as the repair numerator.
Rebases landed the entry under 0.49.6, which shipped on 2026-08-14, so merging would advertise this feature as part of that release. Move it to 0.49.7 Unreleased; 0.49.6 is now byte-identical to main. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016puVootfTssyPQe2TzmK1M
The credit-entitlement additions to MenuCardView, SettingsStore, and UsageFetcher shift 13 allowlisted provider constructs past the gatekeeper's two-line anchor tolerance. Point each entry at its current line; the anchors, provider sets, and fingerprints are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016puVootfTssyPQe2TzmK1M
31281db to
c39ff51
Compare
clawsweeper
Bot
added
rating: 🦞 diamond lobster
2 participants
Closes #2593.
The problem
On a Copilot Business seat with
token_based_billing: true,GET /copilot_internal/userreports every quota snapshot asunlimited: true, entitlement: 0, remaining: 0, percent_remaining: 100. The#1258guards correctly drop those so no misleading "0% used" bar appears — but the result is a Copilot card showing only the plan label and no usage at all.Real consumption is available in
credits_used, already present on each quota snapshot in the response CodexBar already fetches (landed in #2613).GitHub publishes no credit entitlement anywhere
This is the finding that shaped the design. I probed all 8 documented billing endpoints plus
budgets,cost-centers,included_credits,ai_credit/entitlement,copilot/metricsandusage/summary. None expose the included-credit ceiling — the "6,000" that the org billing page displays.discountQuantityreveals only what included credits absorbed, so the ceiling is observable only once exceeded.It is derivable as seats × per-seat allowance, but the per-seat figure is currently a promotional 3,000 against a standard 1,900 — a hardcoded table would silently produce a wrong bar when the promo ends. So the denominator is user-entered and never inferred. A row with no entitlement renders as text, never a bar with a fabricated ceiling.
If you know of an endpoint I missed, that would simplify this considerably.
What this adds
Seat credit bar (builds on #2613). Extends the shared provider-detail row contract with an optional row id, progress ratio, and retained numeric usage (
ProviderDetailSection.Row), so the existing "Credits used" row becomes a "31 / 3000" bar when a seat entitlement is set, and stays plain text otherwise. Created only when it carries real signal (token-billed / unlimited quota / credits > 0 / entitlement set), so metered accounts reportingcredits_used: 0don't gain a permanent empty row. No guard line inCopilotUsageFetcher.swiftis modified — the#1258guard block is byte-identical to main.Per-account entitlements. The seat allowance is stored on the selected
ProviderTokenAccount(following the z.ai per-account fields precedent), resolved incopilotSettingsSnapshotwith the existing global UserDefaults value as fallback — so existing single-account setups keep working unchanged, and two Copilot accounts with different allowances each get the right denominator. The settings field writes to the selected account when one exists.Cached rows repair immediately. Changing or clearing the entitlement rewrites the cached "Credits used" row synchronously, in both the live snapshot and the last-known-reset baseline. The numerator comes from the row's structured progress, falling back to the retained
usageValueon text-only rows — never re-parsed from display text. This is what lets a cached text-only row grow a bar the moment an entitlement is entered even when the follow-up refresh never lands (offline, token lost, 401), and it is why the row carriesusageValueat all.Which surfaces this covers
Worth being explicit, since it's a menu bar app:
codexbar usageCLICredits are detail rows, not a
RateWindow, because a row with no user-entered entitlement has no percentage to show.MenuBarLayout,IconRemainingResolverand the widget all build fromprimary/secondary/tertiary/extraRateWindows, so they see nothing.The consequence: a Business token-billing user still gets an empty menu bar icon and must open the menu to see credits. I'd rather state that than have you find it.
Making the icon work needs a decision I didn't want to make unilaterally: publishing credits as a
NamedRateWindowwhen an entitlement exists would light up the icon, widget and the existing "menu bar secondary metric" picker for free, but Copilot's extra windows are gated behindcopilotBudgetExtrasEnabledin the card renderer, so it risks either double-rendering the row or coupling credits to an unrelated setting. Happy to do it in this PR if you'll say which shape you want.Live behavior on the current head
Captured from a build of this branch against my live Copilot Business seat (token-based billing). Full sequence and caveats in this comment.
2455 / 30002455 / 30002455 / 5000, shorter fillStated plainly: the two offline frames sit either side of a brief reconnect, so they do not form one unbroken offline stretch spanning an entitlement change. The exact text↔bar transition is covered by unit tests rather than by these captures — see
UsageStoreCoverageTests.Testing
Verified on the current head against
e915ca13f:make check— 0 violations, 0 serious across 1,878 files.ProviderArchitectureGatekeeperTests— 38/38. The credit-entitlement additions shift 13 allowlisted provider constructs past the gatekeeper's two-line anchor tolerance, so this branch re-anchors those entries to their current lines; anchors, provider sets and fingerprints are unchanged.One pre-existing failure, unrelated to this PR:
MiniMaxMenuCardBillingTestsexpects"1,234"but gets"1.234"(locale-dependent number grouping; this machine isen_PL). It reproduces identically on a pristineorigin/mainworktree with none of these changes applied. Note that a plainmake testaborts on it rather than reporting and continuing — that is why the evidence above is sharded.All new tests use
ProviderHTTPTransportStubor the existingURLProtocolstub. No live network, no Keychain access, per AGENTS.md.Regression coverage worth calling out: a test asserts the
#1258behaviour still holds — a Business token-billing payload yields credits andprimary == nil, secondary == nil. Per-account tests cover: account override wins over the global fallback, fallback when the account has no value, two accounts producing different snapshot entitlements, and a Codable round-trip through the config store (including legacy JSON without the new keys). Cached-repair tests cover text→bar transition, clear-restores-text-only, no-numeric-usage no-op, and the stale-reset-baseline path.Known limitations
L("AI credits")andL("credits used")have noLocalizable.stringsentries.Scripts/check-app-locales.mjsenforces all 23 catalogs together, so a partial addition failsmake check— left out deliberately rather than done badly. Happy to add all 23 here if you'd prefer.resetsAtis available and a monthly credit budget is a good fit for it (cf. Show ahead/behind pace for all providers and in the menu bar #807, Add Projections to Copilot Tracker #2166).Commits
12 focused commits, each with its own tests, left unsquashed so they can be reviewed in sequence. Squash on merge if you'd rather.
🤖 Generated with Claude Code