Skip to content

feat: add sference batch inference provider#68

Merged
vienneraphael merged 2 commits into
vienneraphael:mainfrom
apejcic:feat/sference-provider
Jun 15, 2026
Merged

feat: add sference batch inference provider#68
vienneraphael merged 2 commits into
vienneraphael:mainfrom
apejcic:feat/sference-provider

Conversation

@apejcic

@apejcic apejcic commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add sference as a batch inference provider for European OpenAI-compatible inline batches (POST /v1/batches).
  • Extend OpenAIProvider with sference-specific inline submission, polling, and result decoding (result_json / error_json rows).
  • Batch only /v1/chat/completions — sference's batch executor rejects Responses-shaped bodies at execution time.
  • Include example, generated docs, registry/contract tests, and architecture notes.

Test plan

  • pytest tests/test_provider_registry.py tests/test_provider_contracts.py
  • Live batchify against https://api.sference.com with moonshotai/Kimi-K2.6 (chat completions path)
  • CI green on upstream

Made with Cursor

@apejcic

apejcic commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

hi @vienneraphael. I was playing around with batchling (love it) and validated it against sference (I'm a cofounder).

  1. would love to talk to you 1:1
  2. would be honored to have sference as batch provider in batchling

@vienneraphael

Copy link
Copy Markdown
Owner

hi @vienneraphael. I was playing around with batchling (love it) and validated it against sference (I'm a cofounder).

1. would love to talk to you 1:1

2. would be honored to have sference as batch provider in batchling

Hi @apejcic, thanks for your PR I really appreciate you contributing to batchling through implementing sference!
Sure let's meet, I just sent you a DM on LinkedIn.

PR looks good to me, can you just make sure to run the ruff-check autofix? This should handle the CI issue.
I see some other CI issues not related to the PR, which I'm going to fix on main directly, you should be able to rebase soon after and the CI should go green.

@vienneraphael

Copy link
Copy Markdown
Owner

CI fix was just merged on my end, I'll let you rebase and run ruff-check autofix, the CI should work after that.

- Updated navigation and documentation to include the Sference provider.
- Implemented Sference provider functionality, including inline batch submission and result handling.
- Added tests for Sference provider's batch payload and response handling.

Co-authored-by: Cursor <cursoragent@cursor.com>
@apejcic apejcic force-pushed the feat/sference-provider branch from 53f1c64 to c5182f6 Compare June 13, 2026 07:34
@apejcic

apejcic commented Jun 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for fixing CI. Fixed formatting with ruff-check. Waiting for CI workflow approval.

Match prek ruff-format hook settings so CI passes.

Co-authored-by: Cursor <cursoragent@cursor.com>
@apejcic

apejcic commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Fixed line width format cf17bd27cca2b7c81127bd40b711a2cf0b7ff1a5

@vienneraphael

Copy link
Copy Markdown
Owner

Awesome thank you, merging the PR right now!

@vienneraphael vienneraphael merged commit b85e008 into vienneraphael:main Jun 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants