FIX: keep template conditions until their parameters are rendered

[Jaafer Rahmani (bIackr0se)]

Description

Crescendo and TAP without a prepended conversation tell the adversarial chat that a prior conversation exists, and give it the text None:

###Prior Conversation Context###
The following conversation has already occurred with the target LLM. Continue the jailbreak sequence from this established context:
None

The template only shows this section {% if conversation_context %}. The if is lost when the YAML is loaded: SeedPrompt renders trusted templates once with the path variables only, and the placeholder for the missing conversation_context is truthy, so the block is kept and its tags are dropped. The later render with conversation_context=None has nothing left to decide. All nine shipped templates with this guard are affected (Crescendo variants 1 to 5, escalation_crisis, image_generation, split_payload, and the TAP adversarial system prompt).

The fix applies to the load-time render only. SeedPrompt and SeedObjective construction now render trusted templates with a separate placeholder: when a missing parameter would decide something (if/elif, inline x if c else y, iteration, ==/!=, a test such as is defined, or the default filter), it raises a private signal instead of answering. The load render then keeps the template as-is, as render_template_value_silent already does for a missing {% for %} collection, and writes each parameter it was given (the dataset paths) that the template uses in front of it as a {% set %}, so later renders and stored copies keep the value. render_template_value_silent itself is unchanged, so TextJailBreak extra kwargs and the adversarial first message render as on main.

CrescendoAttack._setup_async, system prompt sent to the adversarial chat:

	main	this PR
no prepended conversation	section present, reads None	section absent
two prepended turns	section with the turns	section with the turns

Loading all 907 YAML and .prompt files under pyrit/datasets gives the same values as on main, except these nine templates.

Every shipped jailbreak template (650) gives the same output on main and on this branch through TextJailBreak construction and get_jailbreak.

Tests and Documentation

• test_seed_prompt_keeps_template_whose_missing_parameter_decides_a_branch: if/else with True, False, None, elif, inline if, a set alias, is defined, is filter, is test, default, ==, a loop filter, a loop inside a macro.
• test_loaded_template_renders_like_its_source: each of the nine YAML templates, loaded and rendered with and without context, equals a direct Jinja render of its source.
• test_render_template_value_silent_renders_condition_once_its_parameters_are_provided and ..._renders_if_guard_on_loop_variable.
• test_seed_prompt_keeps_path_resolved_at_load_when_its_condition_is_deferred: {{ datasets_path }} next to an unresolved guard still resolves at the final render, with and without context, also after a rebuild from the stored value as memory does.
• test_get_jailbreak_keeps_extra_kwargs_of_a_template_with_a_prompt_guard and test_render_template_value_silent_decides_conditions_as_before: a value supplied in an earlier pass survives to the final render.
• test_setup_sets_adversarial_chat_system_prompt now checks that the section is absent.
• 23 of the new cases fail on main; the nine with-context cases and two loop cases pass on both. The five chain cases fail on df3331e and pass now. Ruff and ty check pyrit/models/seeds (locked ty 0.0.84) pass; the unit suite passes except 11 OpenAI URL warning and backend compatibility tests, which fail the same way on main here.

[Roman Lutz (romanlutz)]

Must Fix: This assignment fails the required type check with the locked dependencies:

uv run --frozen --no-sync ty check pyrit\models\seeds\seed.py
error[invalid-assignment] at seed.py:268

_deferring returns Callable[..., Any], which does not preserve the signatures of the functions in Jinja's inferred env.tests dictionary. The new dictionary is therefore not assignable to env.tests. The repository's ty-check pre-commit hook checks all of pyrit, so this blocks that gate even though the runtime tests pass.

Please preserve the wrapped callable's argument and return types, then rerun the type-check hook with the locked environment.

[Jaafer Rahmani (bIackr0se)]

Fixed in e97c4a0. _deferring is now typed (function: T) -> T with T bound to Callable[..., Any], so each wrapped test and the default filter keep their own type and env.tests accepts the new table. With the locked ty 0.0.84, ty check pyrit/models/seeds passes, and ty check pyrit reports the same diagnostics as main here (only imports of optional extras I do not have installed).

[Roman Lutz (romanlutz)]

Must Fix: Returning the original template here loses values supplied in earlier rendering passes. This breaks the existing TextJailBreak API:

from pyrit.datasets import TextJailBreak

template = TextJailBreak(
    string_template="Style: {{ style }}. {% if prompt %}{{ prompt }}{% endif %}",
    style="brief",
)
template.get_jailbreak("Explain rainbows")

This returns Style: brief. Explain rainbows on both current main and v1.1.0, but raises 'style' is undefined with this change. _apply_extra_kwargs supplies style once, and get_jailbreak supplies only prompt. A trusted SeedPrompt combining {{ datasets_path }} with an unresolved guard also loses the path resolved during construction and fails on its final render.

Please preserve the already supplied values while deferring the unresolved condition, or retain and forward the bound context throughout these callers. Add regression tests for the construction/partial/final rendering chain, not just a final render that supplies everything again.

[Jaafer Rahmani (bIackr0se)]

You are right, thanks. In e97c4a0 the deferral applies only to the load-time render of trusted templates (SeedPrompt and SeedObjective construction), through a separate placeholder class. render_template_value_silent is back to the behavior on main, so TextJailBreak extra kwargs and the adversarial first message render as before: your example returns Style: brief. Explain rainbows again.

When the load render defers, each parameter it was given (the dataset paths) that the template uses is written in front of the template as a {% set %}. So {{ datasets_path }} next to an unresolved guard still resolves at the final render, also after memory rebuilds the prompt from its stored value. No shipped template gets such a line: loading all YAML and .prompt files still differs from main only in the nine guarded Crescendo and TAP templates.

New tests for the chains:

• test_get_jailbreak_keeps_extra_kwargs_of_a_template_with_a_prompt_guard: your example, construction with style and then get_jailbreak.
• test_seed_prompt_keeps_path_resolved_at_load_when_its_condition_is_deferred: construction, then the final render with and without context, on the original and on a copy rebuilt from its value.
• test_render_template_value_silent_decides_conditions_as_before, and test_render_template_value_silent_renders_condition_once_its_parameters_are_provided now checks the path bound at load.

These cases fail on df3331e and pass now. I also ran every shipped jailbreak template (650) through construction and get_jailbreak on main and on this branch, and the output is identical.