fix: treat billing errors as valid key

[smakosh]

Summary

• The Anthropic API returns a 400 error with "credit balance is too low" for valid API keys that have no credits. The validation was treating this as an invalid key.
• Now treats billing/quota/rate-limit errors (402, 429, or messages containing "credit balance", "billing", "quota", "rate limit") as valid — the key is authenticated, just out of credits.
• Frontend toast now shows the actual backend error message instead of a generic one.

Test plan

• Add an Anthropic key with insufficient credits — should succeed
• Add an actually invalid Anthropic key — should still fail with real error message
• Add keys for other providers — no regression

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved error messaging for provider key validation failures, now displaying specific error details instead of generic messages to help users troubleshoot issues more effectively
  • Enhanced billing error detection and classification to properly handle rate-limit and credit-related errors during validation, preventing false validation failures and improving overall validation reliability

[coderabbitai]

Walkthrough

The pull request updates provider key validation to introduce billing-error classification. When validation returns 402, 429, or billing-related error messages, the validation is now marked as successful (valid: true) rather than failed. The UI error handler is concurrently updated to accept and display dynamic error messages from the backend.

Changes

Cohort / File(s)	Summary
UI Error Message Display apps/ui/src/components/provider-keys/create-provider-key-dialog.tsx	Modified the mutation error handler to accept an error parameter and display error.message in the toast description when available, falling back to the prior static message.
Billing Error Classification packages/actions/src/validate-provider-key.ts	Added logic to classify 402, 429 responses and billing-related error messages (containing "credit balance", "billing", "quota", or "rate limit") as billing errors. When classified, validation returns { valid: true } instead of treating the response as invalid, with a debug log entry.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• fix: improve provider key validation error messages and logging #1647 — Also modifies packages/actions/src/validate-provider-key.ts to change how validation errors and results are handled.
• fix(api): treat provider validation errors as 400 instead of 500 #1697 — Changes provider validation error classification to treat billing-related responses (402/429) differently, aligning with this PR's billing error handling.
• feat(UI): improve mutation error handling #1037 — Modifies the same UI error handler in create-provider-key-dialog.tsx to accept error parameters and display dynamic messages.

Suggested labels

codex

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: treat billing errors as valid key' directly and clearly summarizes the main change: reclassifying billing/quota errors to indicate valid keys rather than invalid ones.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/anthropic-key-validation-billing

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (2)

packages/actions/src/validate-provider-key.ts (2)

291-298: Consider logging the error message for debugging purposes.

When a billing error is detected, the errorMessage is not included in the debug log. This information could be useful for troubleshooting why a particular key was classified as having billing issues.

♻️ Suggested improvement

 if (isBillingError) {
     logger.debug("Provider key is valid but has billing/quota issues", {
         provider,
         model: validationModel,
         statusCode: response.status,
+        errorMessage,
     });
     return { valid: true, model: validationModel };
 }

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@packages/actions/src/validate-provider-key.ts` around lines 291 - 298, The
debug log for billing/quota issues currently omits the error details; update the
logger.debug call in the isBillingError branch (where provider, validationModel,
response.status are logged) to also include the errorMessage (or the variable
containing the error text) so the log payload contains errorMessage for
troubleshooting; locate the isBillingError conditional in
validate-provider-key.ts and add errorMessage to the metadata passed to
logger.debug.

---

283-289: Consider narrowing the 429 check to avoid false positives.

Treating 429 unconditionally as a billing error may be too broad. Some providers return 429 for security-related rate limiting (e.g., detecting suspicious validation attempts) regardless of key validity. Consider combining the status code check with the error message check for 429:

 const isBillingError =
     response.status === 402 ||
-    response.status === 429 ||
+    (response.status === 429 &&
+        (errorMessage.toLowerCase().includes("rate limit") ||
+         errorMessage.toLowerCase().includes("quota"))) ||
     errorMessage.toLowerCase().includes("credit balance") ||
     errorMessage.toLowerCase().includes("billing") ||
     errorMessage.toLowerCase().includes("quota") ||
     errorMessage.toLowerCase().includes("rate limit");

This ensures 429 responses that don't explicitly mention rate limits or quotas (which could indicate security throttling) are still treated as validation failures.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@packages/actions/src/validate-provider-key.ts` around lines 283 - 289, The
isBillingError check is too broad because it treats any 429 as billing; update
the logic so 402 remains billing but 429 is considered billing only when the
response/error text actually indicates billing/rate-limit (e.g., errorMessage
contains "rate limit", "quota", "billing", or "credit balance"). Locate the
isBillingError declaration (uses response.status and errorMessage) and change
the 429 clause to require both response.status === 429 and
errorMessage.toLowerCase().includes(...) for the same keywords already used,
leaving the other checks unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/actions/src/validate-provider-key.ts`:
- Around line 291-298: The debug log for billing/quota issues currently omits
the error details; update the logger.debug call in the isBillingError branch
(where provider, validationModel, response.status are logged) to also include
the errorMessage (or the variable containing the error text) so the log payload
contains errorMessage for troubleshooting; locate the isBillingError conditional
in validate-provider-key.ts and add errorMessage to the metadata passed to
logger.debug.
- Around line 283-289: The isBillingError check is too broad because it treats
any 429 as billing; update the logic so 402 remains billing but 429 is
considered billing only when the response/error text actually indicates
billing/rate-limit (e.g., errorMessage contains "rate limit", "quota",
"billing", or "credit balance"). Locate the isBillingError declaration (uses
response.status and errorMessage) and change the 429 clause to require both
response.status === 429 and errorMessage.toLowerCase().includes(...) for the
same keywords already used, leaving the other checks unchanged.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 8a3e7d67-55a2-4cd2-bfc4-24037c0d8d58

📥 Commits

Reviewing files that changed from the base of the PR and between 1a66808 and b2f5b46.

📒 Files selected for processing (2)

• apps/ui/src/components/provider-keys/create-provider-key-dialog.tsx
• packages/actions/src/validate-provider-key.ts