Skip to content

Add kb-arena to LLMOps#394

Open
xmpuspus wants to merge 1 commit intotensorchord:mainfrom
xmpuspus:add-kb-arena
Open

Add kb-arena to LLMOps#394
xmpuspus wants to merge 1 commit intotensorchord:mainfrom
xmpuspus:add-kb-arena

Conversation

@xmpuspus
Copy link
Copy Markdown

@xmpuspus xmpuspus commented Apr 19, 2026

Adds kb-arena to the LLMOps section, placed alphabetically between Izlo and Keywords AI.

About kb-arena

Repo: https://github.com/xmpuspus/kb-arena

Open-source RAG evaluation arena that benchmarks 7 retrieval strategies on user-provided documents:

  1. Naive vector retrieval
  2. Contextual vector (Anthropic's contextual embeddings)
  3. QnA-pair pre-indexing
  4. Knowledge graph retrieval (Neo4j)
  5. RAPTOR (recursive summarization)
  6. PageIndex (hierarchical)
  7. Hybrid (BM25 + semantic)

Why it fits LLMOps

  • Evaluation and observability — RAGAS metrics (faithfulness, context precision/recall, answer relevancy)
  • A/B testing — ELO-based blind Strategy Arena for head-to-head ranking
  • Cost tracking — per-strategy token/call cost accounting
  • Multi-provider — Anthropic / OpenAI / Ollama support
  • Production-grade — 514 tests, FastAPI backend, Neo4j + ChromaDB, Next.js UI

Adjacent to entries like Evidently, Laminar, Langfuse, and Opik in the same LLMOps section — this fills a specific gap for RAG architecture benchmarking.

Checklist per contributing.md

  • Individual PR for this suggestion
  • Alphabetical placement (kb < Keywords)
  • Format matches existing entries (Project / Details / Repository columns with star badge)
  • Star badge URL follows `img.shields.io/github/stars//.svg?style=flat-square` convention
  • Repo is active (last commit April 2026), not archived, not 404

kb-arena is an open-source RAG evaluation arena that benchmarks 7 retrieval
strategies (naive vector, contextual, QnA pairs, knowledge graph, RAPTOR,
PageIndex, hybrid) with ELO-based blind A/B ranking and RAGAS metrics.
Includes an MCP server for agent integration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant