Skip to content

About

Read-only mongosh checks that measure how many agents in a LibreChat deployment would fail a delegation policy review: capability held without stated bounds, reach beyond the author, revisions nobody approved. Reports audit coverage too -- lifecycle events that happened against records kept of them. Aggregate output only. Submissions welcome.

Topics

Resources

Contributing

Stars

4 stars

Watchers

0 watching

Forks

Repository files navigation

Agent Governance Check

Checks agents against the Delegation Charter Specification, an open standard for AI agent accountability.

Looking to diagnose your own deployment? Use librechat-charter-diagnostic: a per-agent report against the full charter, with a draft charter for every agent. Version 1.1 only adds fields; every v1.0 definition is unchanged, so results submitted here remain comparable across deployments.

Paper: Agents Outrun Review: A Read-Only Audit and Multi-Site Protocol for Delegated AI Agents (SSRN 7599098).

A read-only check that measures how much of a LibreChat deployment's agent population would fail a delegation policy review, and how much of it was never recorded in the first place.

One script, no dependencies beyond mongosh, aggregate output only.

mongosh "mongodb://localhost:27017/LibreChat" --quiet \
  --file scripts/agent_governance_check.js > submission.json

Why

Agent creation in self-hosted deployments is typically unrestricted and unlogged. Users build agents, attach tools, share them by link, revise them, and delete them, and no record of any of it survives. Operators discover the resulting exposure by going and looking — after the fact, if at all.

This repository makes that measurable. It does not judge whether agents behaved well. It measures whether the delegation was specified well enough that anyone could tell.


What it measures

Checks derive from a delegated-task specification: a delegation is governed when it names an owner, bounds the task, states what the agent must not do, defines an escalation path, and remains reviewable after it changes.

Hard tier — capability held without stated bounds, or behaviour changed while others could use it:

Check Fails when
no_instructions The agent has no instructions at all
capability_without_scope Holds tools with under 200 characters of scope
external_reach_unbounded Reaches outside the tenant, no stated prohibitions
code_exec_unbounded Executes code, no stated prohibitions
drift_while_shared Revised after being made reachable by others

Structural tier (v1.1, the cross-site headline) — no_instructions, capability_without_scope and drift_while_shared. None of them reads the words of an agent's instructions, so the rate means the same for agents instructed in German, French or English. The keyword-based checks are reported alongside it as an English-only measure.

Context tier — reported but not counted as failures, because they are often near-universal: no_escalation, no_prohibitions, no_role_statement, no_named_contact, reachable_beyond_author, revised_without_record, author_has_persistent_memory, author_holds_api_key.

Audit coverage — lifecycle events that occurred, against records the platform kept of them. This is the control question, and on some deployments the denominator is thousands and the numerator is zero.

Unreviewable traffic — conversations referencing agents that no longer exist. These cannot be checked at all: the configuration is gone.

Full definitions, including the regular expressions, are in CODEBOOK.md.


Output

Aggregate counts only. The script reads no message bodies and prints no instruction text, names, email addresses, identifiers or hostnames. Nothing it emits can be traced to a user or an agent. Read scripts/agent_governance_check.js before running it — it is about 330 lines and every query is visible.

Results are reported over three populations, so that failures among working agents can be distinguished from abandoned drafts:

  • all agents
  • agents used at least once
  • agents used five or more times

Contributing results

A single deployment is a case study. Several are a measurement.

If you run LibreChat or a fork, run the script and email the output to walll@cua.edu. See CONTRIBUTING.md for how to run and check it, PROTOCOL.md for how submissions are stored, published and kept anonymous, and DATA_DICTIONARY.md for every field the script produces. Nothing is posted publicly without your written consent.


Limitations

It measures what agents declare, not how they behave. A well-behaved agent with no written boundaries fails. A badly-behaved one with careful instructions passes. That is the intended reading: without a record, nobody can tell which is which.

Language checks are English-only. Escalation, prohibition and scope are detected by keyword patterns. Agents governed in other languages read as failing, which biases the rate upward. Structural checks — tools held, sharing state, revision history, deletion — are exact and unaffected.

Small counts are suppressed. From v1.1, every count from 1 to 4 is reported as "<5", and tools held by fewer than five agents are grouped, so that a small deployment's results cannot single out an individual. Set AGC_SUPPRESS_SMALL=0 only for internal use.

Stock deployments share agents through the access-control layer. v1.0 counted sharing only through fork-specific fields, which undercounts sharing and drift on current upstream LibreChat. v1.1 adds access-control grants in a separate v1_1 block, leaving every v1.0 field unchanged.

Fork-specific collections are optional. sharedagents, agentapikeys, auditlogs and memoryentries do not exist on stock LibreChat. The script detects and skips them, reporting null rather than failing. Submissions note which were present.


Repository layout

scripts/agent_governance_check.js the portable check (run this) scripts/as-run/ the original three-pass scripts, for provenance schema/submission.schema.json structure of a valid results file CODEBOOK.md check definitions and patterns CONTRIBUTING.md how to submit results PROTOCOL.md multi-site study protocol: storage, publication, conflict of interest DATA_DICTIONARY.md every output field, with definitions test/ mock stock and fork databases; checks v1.1 against v1.0 results/ aggregates posted with the site's written consent


Citation

The design of this check and the multi-site protocol are described in:

Wall, Lucas E. 2026. "Agents Outrun Review: A Read-Only Audit and Multi-Site Protocol for Delegated AI Agents." SSRN Working Paper 7599098. https://papers.ssrn.com/abstract=7599098

@techreport{wall2026agents,
  author      = {Wall, Lucas E.},
  title       = {Agents Outrun Review: A Read-Only Audit and Multi-Site Protocol for Delegated AI Agents},
  institution = {SSRN},
  type        = {Working Paper},
  number      = {7599098},
  year        = {2026},
  url         = {https://papers.ssrn.com/abstract=7599098}
}

License

MIT. See LICENSE.

About

Read-only mongosh checks that measure how many agents in a LibreChat deployment would fail a delegation policy review: capability held without stated bounds, reach beyond the author, revisions nobody approved. Reports audit coverage too -- lifecycle events that happened against records kept of them. Aggregate output only. Submissions welcome.

Topics

Resources

Contributing

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages