Verify that an AI agent's on-chain claims are actually true. A claim-level groundedness checker for Solana / memecoin agents.
Grounded multi-domain support-triage agent (HackerRank Orchestrate Hackathon). 8-layer DAG fusing ChromaDB vector + BM25 lexical retrieval, with an adversarial-critic pass that blocks ungrounded answers — 93% groundedness across a 772-document corpus.
Single-agent, evidence-grounded claim verification to catch LLM hallucinations — a pluggable fact-gate for agent-arena and any multi-agent system (CrewAI, AutoGen, LangGraph).
Provider-agnostic enterprise RAG and agent evaluation harness for Azure Foundry, vLLM, Ollama, and local demos.
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
pip install gauntlet-evals · v0.1.0. Merge-blocking evaluation gates for generative AI features: YAML suites run against any HTTP endpoint or Python callable, fail the build on a miss, and emit both a diffable JSON pack and a reviewer document cross-referenced to California's published GenAI risk framework. Aligned to, never approved by.
Measuring RAG Groundedness on Financial Filings
v0.2.0. Fail-closed evaluation harness for government-facing chat systems: reproducible, provenance-stamped audit verdicts, byte-identical across Python 3.11 to 3.14, with no third-party dependencies. A silent or unreadable target scores zero rather than passing by absence. Two public projects of my own pin it by exact commit.
Beta. Reduced-fare policy assistant citing dated corpus passages in English and Spanish; the bilingual-parity gate is currently failing (see EVALS.md). Corpus of eighteen California transit agencies, public 385-case evaluation harness. Deployed demo serves five agencies; published evidence run lags the repository.
In-build reference implementation: an offline-first plant-care assistant and public evaluation harness with cited-corpus answers, calibrated abstention, toxicity guardrails, EN/ES parity, photo plant ID, and local reminders.
RAG evaluation workbench for retrieval recall, citation coverage, groundedness checks, and failure analysis
Your RAG, put to proof — self-correcting answers and CI eval gates that block the merge when hallucination rate crosses your threshold
Deterministic citation and claim-support checks for RAG evaluation datasets.
Measure whether a RAG system is actually grounded, and fail the build when it stops being: retrieval metrics, claim-level groundedness, hallucination rate and citation accuracy.
Use Jev instead of an LLM for RAG decisions: relevance grading, query routing, hallucination checks and LLM-as-a-judge evals. Faster, cheaper, measured first.
A local groundedness judge for RAG: QLoRA-distilled to match a frontier judge 100% at $0/call. Ships only if its own evals beat baseline.
How wrong are LLMs about emission factors — and how often do they name the wrong source for a number? 467 questions, five models, scored against sourced ground truth.
An AI-powered RAG groundedness / hallucination checker - segments an LLM answer into atomic claims and flags those NOT supported by the retrieved context, via an LLM judge. Provider-agnostic, BYO-key, offline --dry-run. Python CLI.
Scores whether an AI agent's answer is actually supported by the tool results it had access to. An LLM-as-judge harness for catching ungrounded or fabricated claims in agent outputs.
What the four-box RAG diagram leaves out hybrid retrieval, chunking, security trimming, and groundedness evaluation, as runnable Azure AI Search samples.
To associate your repository with the groundedness topic, visit your repo's landing page and select "manage topics."