Skip to content

TICKET-611: Add openai-remote provider for the ai-telemetry-gateway - #41

Open
Naragod wants to merge 4 commits into
mainfrom
ai-telemetry-gateway-connection
Open

TICKET-611: Add openai-remote provider for the ai-telemetry-gateway#41
Naragod wants to merge 4 commits into
mainfrom
ai-telemetry-gateway-connection

Conversation

@Naragod

@Naragod Naragod commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Cloud OpenAI calls from MarkUs grading must pass through the ai-telemetry-gateway so every call is attributed to a course and checked against budgets. No existing provider can reach it: openai talks straight to api.openai.com, and remote speaks polymouth's custom contract. This adds the provider for the gateway path; MarkUsProject/markus-autotesting#731 is the autotester side.

  • Register openai-remote ->OpenAIRemoteModel, so the autotester selects the gateway path like any other model.
  • Subclass OpenAIModel, changing only the endpoint (--remote_url, the gateway's OpenAI-compatible /v1 root) and auth (LITELLM_API_KEY virtual key as Authorization: Bearer), since the gateway speaks the standard OpenAI contract.
  • Forward the x-litellm-spend-logs-metadata header from LITELLM_SPEND_METADATA (set per invocation by the autotester), failing loud on malformed JSON, because the gateway rejects unattributed calls.
  • Default max_tokens to 1024 (caller values win), because the gateway rejects calls that omit it and the tester config has no field to set one.

Known limitation: scope: image uses process_image, which does not send max_tokens, so the gateway rejects it; code and text feedback are the validated paths.

@donny-wong donny-wong left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Naragod , looks good.

@donny-wong
donny-wong requested a review from david-yz-liu July 14, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants