FEAT GUI: Expose configured objective-scorer presets

[romanlutz]

Is your feature request related to a problem? Please describe.

The custom-scenario flow needs a simple scorer choice. PyRIT already has ScorerRegistry and ScorerInitializer presets, but the GUI currently has no read API for choosing configured objective-scorer instances.

Work item 12 of 17 in #2744. This is now available for contributions as an independent API track, in parallel with the main explorer sequence (currently #2748). It uses the existing scorer registry and initializer and does not depend on dataset browsing, the explorer UI, or provider loading. Integration into the scenario flow remains a later step in #2757.

Describe the solution you'd like

Expose a small typed API for initialized, compatible objective-scorer presets from ScorerRegistry.instances.

• Return stable registry names, useful descriptions, scorer/result family, relevant tags, compatibility/prerequisite information, and the compatible configured default_objective_scorer selection.
• Offer ready instances, not just buildable class names. Listing must not instantiate scorers, invoke targets, or manufacture credential-backed configuration.
• Reuse existing presets/composites where available, such as inverted_refusal, task-achieved variants, acs_with_refusal, and scale_and_refusal. Availability depends on initialized targets and ScorerInitializer.
• Preserve objective-result meaning. Raw refusal=true means refusal and must not be exposed as attack success; use an appropriate existing inverter/composite or clearly validated objective preset.
• Report an empty/unusable registry clearly so the GUI can guide setup. Do not silently choose an incompatible default or expose secrets through identifiers/configuration metadata.

Acceptance criteria:

• The API distinguishes configured instances from merely discoverable classes.
• Compatible default tags and non-default presets are represented correctly.
• Missing dependencies, empty registries, incompatible result families, and refusal polarity are covered.
• The response contains no credential values and listing performs no scoring/model calls.
• Tests use fake registered scorers and require no paid services.

Describe alternatives you've considered, if relevant

Do not create a hardcoded frontend scorer list, a general scorer builder, a new scorer registry, or an unscored scenario mode. One selected objective scorer may already be a composite; arbitrary multi-scorer aggregation is out of scope.

Additional context

Start with pyrit\registry\components\scorer_registry.py, pyrit\setup\initializers\scorers.py, and Scenario._get_default_objective_scorer(). The existing /scores routes concern score operations, not a configured-scorer catalog. Follow model/Python/scorer/test instructions and keep registry responsibility separate from service DTOs.

[feiiiiii5]

I would like to take this.

[romanlutz]

As described in #2834 we should follow something closer to #2834 (review)

[feiiiiii5]

Thanks for pointing me to the #2834 review; I read it. I understand the objective-preset endpoint in #2755 would be premature until the named Scorer API and scenario-run selection share one contract. I’ll pause that specialized endpoint and would like to work on the general Scorer API first.

My starting shape is:

• GET /api/scorers/types: registered type names and constructor-parameter schemas.
• GET /api/scorers: paginated safe summaries of configured instances.
• GET /api/scorers/{name}: one safe instance summary.
• POST /api/scorers: create through ScorerRegistry from a registered type and parameters, returning the same summary shape.

For summaries, I propose only a stable public name, scorer type, and safe display description—no prompts, nested target configuration, credentials, or raw scorer identifiers. Objective-scorer selection would remain out of scope until the run API consumes the same names.

Does that contract match what you have in mind, especially the summary fields and create request, or would you prefer a narrower first slice? I’ll hold implementation until we agree on the response shape.

[romanlutz]

To be honest, I'm not sure if Richard Lundeen (@richlundeen) is taking this up as part of his scoring refactoring work so I'd hold off on that for a minute.