Skip to content

test(redteam): cover RedTeamCase metadata sync and _build_system_prompt assembly - #301

Open
zeroshotmind wants to merge 1 commit into
strands-agents:mainfrom
zeroshotmind:test/redteam-case-and-common-coverage
Open

test(redteam): cover RedTeamCase metadata sync and _build_system_prompt assembly#301
zeroshotmind wants to merge 1 commit into
strands-agents:mainfrom
zeroshotmind:test/redteam-case-and-common-coverage

Conversation

@zeroshotmind

@zeroshotmind zeroshotmind commented Jul 7, 2026

Copy link
Copy Markdown

Description

Two small pieces of experimental/redteam logic had no direct unit tests — only incidental exercise as fixture setup inside other strategy test files, which never asserted their actual behavior:

  • RedTeamCase._sync_metadata_from_config (case.py): the model_validator that merges AttackGoal fields into metadata. Untested branches: metadata=None gets fully populated from attack_goal; caller-provided keys are preserved (setdefault semantics — config never overwrites an existing key); partial metadata is backfilled for missing keys only; and an explicit empty dict (falsy but not None) still takes the "existing metadata" branch rather than the "populate" branch.
  • _build_system_prompt (strategies/_common.py): the helper that assembles the six shared prompt blocks (ROLE_BLOCK, PROFILE_BLOCK, playbook, RULES_BLOCK, EXIT_BLOCK, FORMAT_BLOCK) used by every prompt-based strategy. Untested: that all blocks are present, in the correct order (block order is load-bearing for the {actor_profile}/{max_turns} placeholders filled in later via .format()), joined with a blank line, and that those placeholders survive assembly unconsumed.

This is a test-only PR — no production code changes. Adds tests/strands_evals/experimental/redteam/test_case.py and tests/strands_evals/experimental/redteam/test_common.py.

Related Issues

None — found via a review of test coverage gaps in experimental/redteam/, not filed as an issue first since it's test-only and additive.

Documentation PR

N/A

Type of Change

Tests only

Testing

  • Ran python -m pytest tests/strands_evals/experimental/redteam/ — 326 passed (previously 319; +7 new)
  • ruff format --check and ruff check clean on the new files
  • mypy clean on the new files

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

@github-actions github-actions Bot added chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation labels Jul 7, 2026
…pt assembly

RedTeamCase._sync_metadata_from_config (case.py) and _build_system_prompt
(strategies/_common.py) had no direct unit tests — only incidental exercise
via fixture setup in other strategy test files. Neither behavior was
actually asserted:

- RedTeamCase: metadata=None population, setdefault-style preservation of
  caller-provided keys over attack_goal values, partial backfill, and the
  empty-dict (falsy but not None) branch.
- _build_system_prompt: that it includes and correctly orders all six
  shared prompt blocks, joins them with a blank line, and leaves the
  {actor_profile}/{max_turns} placeholders for callers to fill in later.
@zeroshotmind
zeroshotmind force-pushed the test/redteam-case-and-common-coverage branch from bc2a2b6 to 0a53c80 Compare July 7, 2026 21:49

@pgrayy pgrayy Jul 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, I would recommend placing this under the directory redteam/strategies/. This also helps directly map the test module to its src module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-redteam Red teaming: adversarial generation, attack strategies, attack success evaluation chore Maintenance tasks, dependency updates, CI changes, refactoring with no user-facing impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants