FEAT add AgentThreatRulesScorer with no new dependency - #2624
FEAT add AgentThreatRulesScorer with no new dependency#2624Adam Lin (eeee2345) wants to merge 4 commits into
Conversation
Re-proposes the scorer reverted in microsoft#2410. That revert asked for a version without the `[atr]` extra, and this one has none: `pyproject.toml` is untouched and nothing imports `pyatr`. The scorer subclasses RegexScorer and loads a precompiled digest that ATR's CI publishes at data/pyrit-digest.json. Every pattern in that digest is plain Python `re` syntax, so RegexScorer compiles it exactly as it does any other pattern set. The digest is fetched from a pinned commit by default and cached under DB_DATA_PATH — the same path the ATR seed dataset from microsoft#1715 already uses. Callers who want to track ATR's main branch pass ref="main" and accept that their results move when ATR does. Two design points worth review: Field selection. ATR rules are written against specific agent surfaces (content, tool_response, tool_args, ...). A scorer receives one piece of text with no surface label, so only conditions written against the digest's declared default_fields are loaded by default — 1,349 patterns of the 3,303 in the current digest. `fields=` widens or narrows that when the caller knows which surface the text came from. Fail loud, not quiet. Every pattern is compiled at construction, so an ATR-side regression surfaces as a construction error naming the offending condition rather than a scorer that silently matches less than it reports. An unsupported digest schema is rejected for the same reason. The exclusions ATR could not express as a pure regex OR are carried in the digest as an explicit versioned list rather than dropped with a warning. The docstring states plainly that ATR's own precision figures are measured on corpora its rules were partly mined from and do not transfer to this setting; this is a fast local pre-filter, not a calibrated detector. 18 unit tests, all offline — the digest is served from a fixture, so no test touches the network.
The pre-commit hook pins ruff v0.16.6, which flags unsorted-imports where the locally installed 0.15.22 did not.
PyRIT renders docstrings through MyST, not Sphinx, so ``:class:`Foo``` appears as literal text in the built site. build_scripts/check_no_rest_roles enforces this; gen_api_md.py auto-links known symbols at render time.
tests/unit/docs/test_scorer_documentation.py requires every scorer the registry discovers to be named in doc/code/scoring. The example is shown as a fenced block rather than an executed cell because the scorer fetches the ATR digest over the network on first use.
Adam Lin (eeee2345)
changed the title
|
Adrian Gavrila (@adrian-gavrila) — this is the re-proposal you asked for in #2410, out of draft now. No new optional dependency: I verified it on a fork run of Two points worth your eye rather than mine: Field selection. ATR rules are written against specific agent surfaces, and a scorer gets text with no surface label, so only the digest's declared The docstring says plainly that ATR's published precision figures come from corpora its rules were partly mined from and do not transfer here. I would rather ship that caveat in the source than have someone infer a calibrated detector. |
1 participant
Description
Re-proposes the scorer reverted in #2410, without the dependency that revert objected to.
#2410 said "going to re-propose without a new optional dependency." This version has none:
pyproject.tomlis untouched, the[atr]extra stays deleted, and nothing importspyatr.The scorer subclasses
RegexScorerand loads a precompiled digest that ATR's CI publishes atdata/pyrit-digest.json. Every pattern in that digest is plain Pythonresyntax, soRegexScorercompiles it exactly as it does any other pattern set. The digest is fetched from a pinned commit by default and cached underDB_DATA_PATH— the same path the ATR seed dataset from #1715 already uses. Callers who want to track ATR's main branch passref="main"and accept that their results move when ATR does.Two design points worth review:
Field selection. ATR rules are written against specific agent surfaces (
content,tool_response,tool_args, ...). A scorer receives one piece of text with no surface label, so only conditions written against the digest's declareddefault_fieldsare loaded by default — 1,349 patterns of the 3,303 in the current digest.fields=widens or narrows that when the caller knows which surface the text came from.Fail loud, not quiet. Every pattern is compiled at construction, so an ATR-side regression surfaces as a construction error naming the offending condition rather than a scorer that silently matches less than it reports. An unsupported digest schema is rejected for the same reason. The patterns ATR could not express as a pure regex OR are carried in the digest as an explicit versioned exclusion list rather than dropped with a warning.
One thing stated plainly in the docstring rather than left for someone to discover: ATR's own precision figures are measured on corpora its rules were partly mined from, so they do not transfer to this setting. This is a fast local pre-filter, not a calibrated detector.
Tests and Documentation
18 unit tests in
tests/unit/score/test_agent_threat_rules_scorer.py, all offline — the digest is served from a fixture, so no test touches the network. They cover field selection, digest schema rejection, a pattern that fails to compile, cache hit, corrupt-cache fallback, cache write, and that a branch name cannot escape the cache directory via path traversal.Full
tests/unit/score/suite passes (1,783 tests).ruff format --checkandruff checkare clean on all four changed files.Verified against the real published digest: 1,349 patterns load for the default fields and 2,303 with
fieldswidened, all compiling under Pythonre.Marked draft until CI reports.