FEAT: Add ANSI escape output scorers and Garak AnsiEscape scenario

[v0ropaev]

Part of #511

AnsiAttackConverter (#577 → #597) covers the prompt side of garak's ansiescape probe. The other half discussed on #577 — a scorer that checks whether the model actually emitted terminal control sequences — was the first item under "Next Steps" in #597 but was never followed up. There is also no scenario for garak's two active probes, ansiescape.AnsiRaw (tier OF_CONCERN) and ansiescape.AnsiEscaped (INFORMATIONAL).

Right now PyRIT can send ANSI prompts but can't tell whether the target produced something a terminal, log viewer or renderer downstream would act on.

Proposed scope

Two PRs, scorers first since they're useful on their own:

1. Scorers in pyrit/score/true_false/regex/, following XSSOutputScorer:

   • AnsiEscapeOutputScorer — raw sequences: ESC [ (CSI), ESC ] (OSC) and the C1 code points U+009B / U+009D. Parity with garak's detectors.ansiescape.Raw.
   • EscapedAnsiOutputScorer — escaped text forms that go live once something unescapes them: \x1b, \033, \u001b, \27, \e followed by [ or ], plus \x9b / \x9d. Parity with detectors.ansiescape.Escaped.

   Two classes rather than one with a mode, so raw and escaped findings stay distinguishable in memory and metrics — same split garak makes with its tiers.

2. Scenario garak.AnsiEscape with Raw / Escaped techniques on PromptSendingAttack, prompts from a local garak_ansi_escape dataset (garak's asks × tasks, repeat stubs, unescape stubs), mirroring WebInjection / Doctor. Plus exports, tests, the synchronized scanner docs and third_party/garak-provenance.json entries.

AnsiAttackConverter and the Foundry ansi_attack technique stay as they are.

One upstream detail

garak's RAW_COMMANDS contains "\27", which in a Python string literal is octal U+0017, not ESC, so that entry can never match what it's meant to. I'd leave it out of the raw scorer. The decimal form is still covered on the escaped side, where \27[ is literal text.

One question before I start

Is the two-PR split fine, or would you rather see scorers and scenario together the way #2064 shipped PolicyPuppetryConverter with Doctor?

References

• Parent: Parent: LLM Vulnerability Scanner Feature Parity with Garak, Giskard, CyberSecEval #511; converter: FEAT: Add ANSI Escape Code Attack #577, FEAT: Ansi Escape Code Converter #597
• Upstream (pinned to the garak revision in third_party/garak-provenance.json): probes/ansiescape.py, detectors/ansiescape.py, resources/ansi.py
• LLM Output Can Take Over Your Computer

[v0ropaev]

I would like to take this. I can't self-assign (no repo permissions) — could a maintainer assign me?

[v0ropaev]

Flagging an overlap so a maintainer doesn't have to diff two branches: this issue now has two PRs, #2688 (mine, the scorer half) and #2689, opened a few minutes later.

They are substantially the same thing — same detection surface, case-insensitive escaped forms, same decision to leave garak's octal "\27" raw command out, same provenance entries. The differences I can see:

• FEAT: Add ANSI escape output scorers #2688 keeps one named pattern per escape form (Hex Escaped ESC Sequence, Octal Escaped ESC Sequence, ...), so the score rationale says which form matched; FEAT: add ANSI escape output scorers #2689 folds them into one alternation (Escaped ESC Introducer).
• FEAT: Add ANSI escape output scorers #2688's tests run garak's LIVE_PAYLOADS / ESCAPED_PAYLOADS through both scorers and assert the cross-negatives (escaped text must not trip the raw scorer and vice versa), plus the \27 and caret-notation negatives.

Happy either way — pick whichever shape you prefer, or tell me what to change in #2688. If you take #2689, I'll close mine.

The scenario half from the description above isn't in either PR yet; I'll send it separately once the scorer shape is settled.