Repository navigation
FIX: Preserve word-selection parameters in converter identifiers - #3000
Zhibo Lin (LE0-Lin) wants to merge 7 commits into
Conversation
Signed-off-by: Zhibo Lin <147509942+LE0-Lin@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zhibo Lin <147509942+LE0-Lin@users.noreply.github.com>
|
I included the other five converter overrides and kept the legacy default hashes. The related 321 tests and pre-commit checks pass locally. This is now ready for review, Roman Lutz (@romanlutz). Thanks! |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
| def test_identifier_normalizes_equivalent_indices(converter_type: type[WordLevelConverter]) -> None: | ||
| first = converter_type(word_selection_strategy=WordIndexSelectionStrategy(indices=[0, 1])) | ||
| second = converter_type(word_selection_strategy=WordIndexSelectionStrategy(indices=[1, 0])) | ||
|
|
There was a problem hiding this comment.
🔴 Must Fix: reordered indices still collide for seeded converters. These index lists are not equivalent for all of the converters in this test. With CharSwapConverter(max_iterations=1, seed=123) and prompt "alpha beta", [0, 1] produces "alpah btea", while [1, 0] produces "alhpa btea". Both results are repeatable, but the hashes and default registry names are identical, so registering the second still raises ValueError.
WordIndexSelectionStrategy.get_identifier_params() sorts the indices, while select_words() returns them in their original order. CharSwap consumes one shared random stream in that order. I also reproduced this with seeded Zalgo and nondeterministic Leetspeak under a fixed root seed.
Could we add actual conversion assertions for these stochastic cases and preserve behavior-changing order in the identity, or explicitly make execution order canonical? Hash equality alone hides the remaining collision.
There was a problem hiding this comment.
You're right, I missed the effect of execution order on the random stream. I now keep that order in the identity and added conversion and registration tests for all three cases. The outputs are unchanged by the fix.
| ids=lambda converter: converter.__name__, | ||
| ) | ||
| def converter_type(request: pytest.FixtureRequest) -> type[WordLevelConverter]: | ||
| return request.param |
There was a problem hiding this comment.
🟡 Should Fix: this fixture fails the configured test type check. uv run --frozen ty check tests\unit\converter\test_word_selection_identifiers.py reports unsound-return-statement here because request.param is Any, not a verified type[WordLevelConverter]. The pre-commit hook checks only pyrit, so it does not catch this test-file error.
Could we narrow the value before returning it? This version passes the same type-check rules:
converter = request.param
assert isinstance(converter, type)
assert issubclass(converter, WordLevelConverter)
return converterThere was a problem hiding this comment.
Added the narrowing assertions you suggested. ty now passes for all five changed files, including the tests.
Description
Fixes #2995. Different word selections can produce different text but the same converter identifier, causing the second default registry registration to fail.
I extended the BinAscii fix to CharSwap, FirstLetter, Leetspeak, UnicodeReplacement, and Zalgo. They now include the parent word-selection parameters. Existing default identities are preserved, including CharSwap's default 20% selection and the other converters' all-words configurations. Non-default selections intentionally receive corrected hashes; previously saved hashes for those configurations can differ.
WordIndexSelectionStrategyalso retains the supplied index order in its identity. Reordering indices can change which random values are used for each word. The fix preserves existing conversion order and outputs rather than sorting execution.This follows the maintainer invitation to continue in this PR. Constructor APIs and conversion methods are unchanged.
Tests and Documentation
uv run --frozen --all-extras ty checkon the five files changed for review: passed, including the test files. The fixture now narrowsrequest.parambefore returning it.pre-commit run --fileson those five files: passed, including full-treety check pyrit.git diff --checkpassed.The WordIndex selection docstrings now describe conversion order. Full repository tests and real-model integrations were not run locally.
AI assistance was used.