Feature/cloudflare gateway connector

[anassg-lago]

What

• Cloudflare AI Gateway connector: live path (wrap() auto-detects a client pointed at gateway.ai.cloudflare.com, skips billing on cf-aig-cache-status: HIT, auto-primes Workers AI pricing) and backfill path (gateway.adapters.cloudflare_gateway extracts a Logs API entry and bills Cloudflare's own metered cost via emit(usd_cost=..., event_id=...), idempotent across re-runs).
• Model-attribution fix: OpenAI/Anthropic/Gemini adapters now prefer the response's own resolved model over the requested alias — fixes mispriced/misattributed events for aliased model names (e.g. Mistral's -latest, Gemini resolving to a dated snapshot).
• Mistral pricing: resolves -latest aliases against Mistral's own /v1/models (union-find over its mutually-aliasing shape) so OpenRouter price lookups hit the right dated model.
• Lazy pricing warm-up: Cloudflare Workers AI and Mistral pricing are primed reactively on the first wrap()'d call, not eagerly at SDK init.
• Queue reliability: split permanent (4xx) vs. transient send failures; bounded shutdown drain instead of silently dropping stranded events on exit.
• verify_ssl config option for local dev against a self-signed Lago instance.
• README: documented the gateway connector, trimmed redundant sections and roadmap/phase markers.

Testing

• 443 unit tests passing, 23 skipped (require live credentials).
• Live-verified against a real Cloudflare AI Gateway (Workers AI, Anthropic, Mistral passthrough) and a real Lago account — confirmed exact current_usage event counts before/after.

[ancorcruz]

Review notes

Six items, in the order I'd act on them. The first two I'd fix before merge; the next three are cheap hardening I'd do in the same pass; the last is a behaviour change worth double-checking against live data.

None of these are caught by the current suite — CI is green on all 6 jobs. Two of them (the id collision, the streaming Workers AI path) need a scenario the tests don't construct.

[ancorcruz]

prime() force-flags every source stale with no TTL check, and wrap() calls it every time — so pricing_ttl_seconds is effectively ignored.

_openrouter_stale = True unconditionally (and the CF/Mistral flags below it), while maybe_refresh() (line 908) branches purely on those flags and never consults _openrouter_fetched / self._ttl. LagoSDK._auto_prime_pricing_for runs on every single wrap() (sdk.py:160).

A server doing sdk.wrap(OpenAI(...)) per request therefore re-marks every primed source stale continuously, and the queue thread re-downloads OpenRouter's full catalog on every flush_interval tick — indefinitely. maybe_refresh() also runs synchronously at the top of EventQueue._run's tick, ahead of the drain, so each of those fetches delays event delivery.

No wrong bills, just steady unnecessary work — but it's a divergence from what this docstring describes ("zero further network calls until the TTL expires") and from the CHANGELOG's claim that prime() no longer eagerly force-fetches. Gating each flag on table is None or age >= self._ttl restores the documented behaviour.

[anassg-lago]

Confirmed, and I'm deferring this one rather than fixing it now — flagging explicitly so it doesn't read as forgotten.

Your diagnosis matches what I see: prime() sets _openrouter_stale = True unconditionally, maybe_refresh() branches only on those flags without consulting _openrouter_fetched / self._ttl, and _auto_prime_pricing_for runs on every wrap() — so a server wrapping per request re-downloads the catalogue on roughly every flush tick, indefinitely, and each fetch sits ahead of the drain in the queue's tick.

The reason for deferring is your own framing: "No wrong bills, just steady unnecessary work." Everything else outstanding on these two PRs is a money path — the cache-key casing turned out to be a measured 3.96x over-bill, the unit basis was under-reporting an Anthropic cached call by nearly half, and the ~-alias gap billed nothing at all for 11 models. Against those, wasted bandwidth and a delayed drain are real but not urgent, and the fix touches the refresh path that every price-mode customer depends on — I'd rather land it with its own attention than bundle it into a batch of unrelated changes.

It's grouped with the other hardening items from your non-blocking list (the per-source backoff you raised alongside it, the Workers AI pagination cap, send_individually ordering, the eager undici import) for a follow-up PR per repo once the money paths are done. pricing_ttl_seconds being effectively ignored is a documented divergence from both the docstring and the CHANGELOG, so it stays on the list rather than being written off.

If you'd rather it land before merge, say so and I'll pull it forward — it isn't a large change, and gating each flag on table is None or age >= self._ttl is exactly what you described.

[ancorcruz]

Re-review of the fixes (17b3005..5c514fe)

All five fixes verified in the code, and three went further than the reports asked. Streaming model-drop was fixed in all three wrappers, not just OpenAI's; the cache-key lists gained the providers' native spellings; and the pagination fix corrected my suggestion — I proposed keying on result_info.total_count, you measured it reporting 291 against 64 served and used a short page instead. I also checked deoverlapped_token_total agrees with the split path in all four provider/price combinations, and the _tok_/_cost_ regression test does fail without the fix.

Agreed on deferring prime()/TTL, for the reason you gave — it's the only one of the set that doesn't touch money. Leaving that thread open; resolving the other five.

CI green on all 6 jobs, 513 unit tests pass locally.

Four of the eleven items below are regressions introduced by these fixes, which is the main thing worth acting on: the overflow report can deadlock a customer's thread, api_url now accepts "" and silently stops all billing, deoverlapped_token_total over-reports for workers-ai, and narrowing _PERMANENT_STATUSES re-opened head-of-line blocking for 413. Six were reproduced by running them.

Smaller notes not worth their own threads:

• _WORKERS_AI_COMPAT_PREFIX is now defined in both pricing.py:124 and openai_native.py:52, each with a comment asking a human to keep it in sync — and they're load-bearing on each other (recognise the prefix, then strip it to price). pricing.py has no cycle with adapters/, so one definition would do.
• sdk.py:291's no-subscription path now emits logger.error and a logger.warning via _report_error — two lines per drop.
• canonical.py's nonzero_numeric silently drops a negative count, while the same commit adds on_error reporting for the other two drop paths. Defensible since adapters clamp upstream, but it's the odd one out now.
• verify_ssl is now a first-class LagoSDK(...) argument and the docstring recommends verify_ssl=False for local dev — which routes the documented happy path straight through lago_client.py:25's unguarded requests.packages access (on the deferred list). Worth pulling that one forward with the rest of the hardening, since it's now the advertised setup.

[ancorcruz]

Blocker (new): the overflow report runs the customer's callback while holding the lock, so a re-entrant callback deadlocks their thread.

push() takes self._lock at line 123, and this _report_error — which invokes config.on_error — is called inside it. self._lock is a plain threading.Lock (line 83), not an RLock. So a callback that touches the SDK in any way reaching push(), flush(), or stop() blocks on a lock its own thread already holds, and hangs forever. The surrounding try/except can't help: a deadlock isn't an exception.

This is the only one of the four _report_error call sites inside the lock — 198, 251 and 284 are all outside.

It's a plausible trigger rather than an exotic one: overflow happens under sustained load, which is exactly when someone's on_error hook would want to emit a diagnostic or force a flush. And the failure is worse than the unreported drop it was added to fix — an unnoticed dropped event costs one event, a hung producer thread costs the whole application.

Even without re-entry there's a cost: once the buffer is full, every subsequent push runs the callback plus a logger.warning synchronously, on the customer's LLM-call thread, holding the lock that all producers and the drain thread need.

Same shape as should_wake fixes it — set a flag inside the lock, report after releasing.

[ancorcruz]

Blocker (new): api_url="" now overwrites the production default, and the resulting failure retries forever.

The if api_url: → if api_url is not None: change is right for the bug it fixes, but it swapped one silent misroute for another. LagoSDK(api_key=k, api_url=os.environ.get("LAGO_API_URL", "")) with the env var unset previously kept https://api.getlago.com/api/v1; now it writes "".

The downstream behaviour is the bad part. send_batch POSTs to "/events/batch", requests raises MissingSchema, and that isn't a LagoApiError — so _is_permanent_failure returns False, _replay_failed re-prepends the batch, and the queue retries at the 60s ceiling indefinitely. All billing stops, nothing is ever dropped or escalated, and the only symptom is a growing buffer.

if api_url: for this one field keeps your fix intact — the bug you were fixing was a truthy default overwriting config, so falsiness is the right guard here specifically. Or if api_url is not None and api_url != "" if you'd rather keep the shape uniform across the four fields.

[ancorcruz]

Blocker (new): workers-ai is in _INPUT_INCLUDES_CACHE_READ but missing from _OUTPUT_INCLUDES_REASONING, so this over-reports by 73%.

Verified by running it — identical usage, different provider:

CanonicalUsage(input=100, output=1000, reasoning=800)
  provider="openai"      -> 1100
  provider="workers-ai"  -> 1900

workers-ai is reached only through Cloudflare's OpenAI-compatible endpoint — that's the argument _INPUT_INCLUDES_CACHE_READ's own comment (line 81) makes for including it. In that payload shape completion_tokens_details.reasoning_tokens is a subset of completion_tokens, which is precisely why openai is in _OUTPUT_INCLUDES_REASONING. And extract_openai_native fills reasoning from that key with no provider gate (lines 159, 170). So the subset gets counted twice, and the new single-event unit publishes 1900 for a call that consumed 1100. Cloudflare hosts reasoning models (deepseek-r1, qwen, glm), so it's reachable.

compute_cost would double-bill the same tokens; it doesn't today only because _CLOUDFLARE_UNIT_FIELD_MAP happens to carry no reasoning unit. That's an accident, not a guard.

Adding "workers-ai" to _OUTPUT_INCLUDES_REASONING fixes the symptom. Worth considering the structural version too: lines 336-355 restate compute_cost's de-overlap (same .lower(), same PRICED_FIELDS comprehension, same two membership tests), and the stated goal is that the two paths "agree by construction". Two copies can only agree by convention — and this finding is them agreeing with each other while both diverging from what _OUTPUT_INCLUDES_REASONING is for. A token_total on CostBreakdown, set from the counts compute_cost already builds, would make it structural and drop the usage: Any signature.

[ancorcruz]

Blocker (new): excluding 429/408 is right, but the explicit 6-status list makes genuinely-unretryable 4xx transient again.

Rescuing 429 and 408 from the permanent path is a real fix. The cost is that 413, 402, 415, 405 and 410 are now transient, and for those re-sending the same batch provably cannot succeed.

413 Payload Too Large is the concrete one — realistic for the default max_batch_size=100 with sizeable properties dicts, and returnable by any ingress in front of Lago. send_batch raises LagoApiError(413), _is_permanent_failure says transient, _replay_failed re-prepends the identical oversized batch at the head of the FIFO, and it backs off to 60s forever. Every event behind it is blocked until the 10,000-event buffer overflows.

The sharp part: _send_individually would have split that batch into per-event sends and actually delivered them — but it's only called from the permanent branches (lines 247, 282), so the batch that most needs splitting never gets there.

The docstring's rationale ("dropping one that would have been accepted costs revenue") is sound for an unknown 4xx. It just doesn't cover statuses where the batch can't succeed as-is but its events can individually. Adding 413/402/415 to the set routes them to _send_individually, which is the behaviour you want — split, deliver what's deliverable, drop only what genuinely 4xxs alone.

[ancorcruz]

The native-spelling coverage is asymmetric between the three fields.

The premise behind this fix — the gateway can forward a provider's own keys untouched, proven by the reasoningTokens fixture — is now applied unevenly:

• cache_read gets Gemini's native cachedContentTokenCount, but not Anthropic's native cache_read_input_tokens.
• cache_write (line 140) does get Anthropic's native cache_creation_input_tokens.
• reasoning (line 145) gets neither provider's native name — no Gemini thoughtsTokenCount.

So an Anthropic entry forwarding native keys resolves cache_write correctly and cache_read as 0. Anthropic's cache is additive, so those cached prompt tokens drop out of the bill entirely — and nonzero_numeric then emits no llm_cached_input_tokens event at all. That's a straight under-bill, and it's the mirror image of the over-bill this commit fixed.

Adding cache_read_input_tokens to the first list and thoughtsTokenCount to reasoning closes it. The module docstring's "Field mapping" block (lines 8-12) still lists one spelling per field and is now stale.

[ancorcruz]

The -\d{3} arm governs Bedrock and Cloudflare lookups too, and the safety check only covered OpenRouter.

"Zero of the 415 live ids have a model part ending in exactly three digits" is the right check — for OpenRouter. But _strip_version is also called from bedrock_model_key (line 695), _aws_model_keys (701, 705) and lookup_cloudflare_workers_ai (661), and those catalogs weren't part of it. Verified:

_strip_version("text-bison-001")             -> "text-bison"
_strip_version("amazon.titan-text-lite-100") -> "amazon.titan-text-lite"
_strip_version("@cf/baai/bge-base-en-001")   -> "@cf/baai/bge-base-en"
bedrock_model_key("amazon.titan-text-lite-100") -> "titantextlite"

The Bedrock case is the one that can mis-bill rather than just miss: parse_bedrock_offer writes with table.setdefault(key, {})[direction] = price, so two AWS products differing only by a trailing 3-digit marker collapse onto one key and the later price wins — cross-model mispricing rather than a clean miss. In lookup_cloudflare_workers_ai the stripped candidate can match a different catalog entry.

Good news: mistral-small-2501 is safely untouched (4 digits), so the Mistral snapshot ids aren't at risk. Either run the same verification against Cloudflare's catalog and the AWS offer names, or scope the new arm to the OpenRouter lookup that motivated it.

[ancorcruz]

Gemini samples model_version only from the chunk that carries usage; Anthropic's fix in the same commit persists it.

last_with_usage is rebuilt from scratch on every chunk where usage_metadata is truthy, and model_version is read from that same payload. So for chunks [{model_version: "gemini-2.5-flash-002", usage_metadata: {…}}, {usage_metadata: {…}}] — a usage-bearing final chunk that omits the version — the emitted event reverts to model_version=None, resolve_model falls back to the requested gemini-flash-latest, and price mode misses exactly as it did before the fix.

The Anthropic wrapper handles the identical problem correctly, with resolved_model = _merge_stream_usage(...) or resolved_model carried across all events. Same treatment here would do it. The new test can't catch it because FakeModels puts model_version on every chunk, and the async path at line 136 has the same code.

Tiny related thing in wrappers/anthropic.py:107: the docstring says "The caller keeps the first one it sees", but x = f(...) or x keeps the last. Only message_start carries message.model today so it's latent — but the docstring is the only statement of the rule. resolved_model = resolved_model or _merge_stream_usage(...) would match the doc and short-circuit the merge.

[ancorcruz]

Carrying the gateway's echoed model through makes _infer_provider depend on what the gateway echoes — worth confirming with a captured /compat streaming chunk.

Fixing the streaming model drop is right, and belt-and-braces with the lookup_cloudflare_workers_ai prefix strip. But it does change the input _infer_provider sees on the streaming path, and it makes one sentence of that function's new docstring false: "what a streaming call always reports (the synthetic usage payload carries no model, so resolve_model falls back to the requested string verbatim)". After this commit the payload does carry a model, sourced from the gateway.

That matters because _infer_provider only returns workers-ai for @cf/… or workers-ai/@cf/…. If Cloudflare's /compat streaming chunks echo model as anything else — a bare slug, a normalised name — a streamed Workers AI call is stamped openai, looked up against OpenRouter, and degrades to token events. Which is the regression the workers-ai fix closes, reachable again on the streaming path only.

It may well echo the prefixed form, in which case there's nothing to do but correct the docstring. One captured streaming chunk from /compat settles it, and it'd be worth a fixture either way — this is the one path where the two halves of the fix could disagree.

[ancorcruz]

apply_markup now returns "0" on bad input with no report — a silent $0 bill where there used to be a loud failure.

Not raising from a money helper is the right call, and JS parity is a good reason. The gap is that nothing reports it. Verified: apply_markup("abc","1.2"), apply_markup("1.5","abc"), apply_markup("1.5","-2") and apply_markup("1e16","1") all return "0", and in _push_cost_event that flows straight into "value" and money_str_to_cents(...) → precise_total_amount_cents="0". A split cost event is billed at $0.00 with no log line and no on_error.

Worse, it consumes the transaction_id: a corrected re-run of that window is then rejected as a duplicate, so the $0 sticks. That's the same failure shape I flagged on the Databricks PR for negative usage_quantity.

It's a defensive path — parts["cost"] comes from compute_cost, which produces well-formed money strings, so I can't construct a live trigger. But this commit's own theme is "report every dropped event", and this is the one new silent-zero path. A _report_error alongside the zero would make it consistent with the overflow and no-subscription reports. Note _parse_price also rejects ≥1e16, so apply_markup now zeroes some inputs the old code computed correctly.

[ancorcruz]

The 40-page bound still allows ~400s of blocked event delivery per refresh.

_CF_MAX_PAGES = 40 sequential requests.get calls at the default 10s request_timeout_seconds is up to ~400s, all on the lago-queue thread inside maybe_refresh() — which _run calls before _take_batch(). A slow endpoint, or one that ignores page and keeps returning full pages, stalls every queued billable event for that whole walk; and since _cloudflare_stale clears only on success, the walk repeats.

The changed comment says the bound exists so the loop "must not stall event delivery indefinitely" — which is true, it's now bounded. But 400s is a stall, just a finite one.

This is the same family as the prime()/TTL item you're deferring, so it probably belongs in that same follow-up rather than here. Two cheap options for whenever it lands: drain the buffer before maybe_refresh() in _run, or bound the whole walk by wall clock the way the shutdown drain already does with min(self._max_retry_seconds, 10.0), instead of by page count.