FEAT: Add PinyinConverter for Chinese Pinyin-mix transformations - #2649
Open
Nantha kumar (Nanduu24) wants to merge 1 commit into
Open
FEAT: Add PinyinConverter for Chinese Pinyin-mix transformations#2649Nantha kumar (Nanduu24) wants to merge 1 commit into
Nantha kumar (Nanduu24) wants to merge 1 commit into
Conversation
Adds a deterministic (no-LLM) text converter that rewrites Chinese (Hanzi) characters as their Pinyin romanization, a Chinese-specific adversarial pattern described in recent Chinese LLM safety work (e.g. CSSBench). Pinyin mixing can bypass keyword/token-level safety filters that match on Hanzi rather than on romanized readings. Supports three rendering modes (full reading, first-letter initial, and a per-character mix), a configurable proportion of Hanzi to convert (values below 1.0 leave the rest as Hanzi, producing mixed Hanzi/Pinyin text), an optional syllable separator, and a seed for reproducible selection. Non-Hanzi characters always pass through unchanged. pypinyin is added as an optional dependency (pip install pyrit[pinyin]) and imported lazily, following the pattern used by other optional-dependency converters. Tests that exercise conversion are skipped when pypinyin is absent. Closes microsoft#2647
Nantha kumar (Nanduu24)
force-pushed
the
feat/pinyin-converter
branch
from
September 13, 2026 23:45
3578241 to
9974b05
Compare
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a deterministic (no-LLM) text-to-text converter,
PinyinConverter, that rewrites Chinese (Hanzi) characters as their Pinyin romanization. Pinyin mixing is a Chinese-specific adversarial text transformation described in recent Chinese LLM safety work such as CSSBench: Hanzi characters or spans are rewritten as full or abbreviated Pinyin while the text stays understandable to a Chinese-reading model. This can bypass keyword- and token-level safety filters that match on Hanzi rather than on romanized readings. I could not find an existing PyRIT converter that performs a Hanzi→Pinyin transformation (translation changes the language; character-noise and Unicode converters don't model romanization).Behavior
mode: how a converted Hanzi is renderedfull— full toneless reading (中→zhong)initial— first letter only (中→z)mixed— chooses full or initial independently per characterproportion([0.0, 1.0]): fraction of Hanzi to convert. Values below1.0leave the rest as Hanzi, producing mixed Hanzi/Pinyin text. Selection count isround(proportion * n_hanzi), positions chosen via the converter's seeded RNG.separator: optional string inserted between converted syllables (e.g." "→zhong xin); no trailing separator is left.seed: reproducible selection and, inmixedmode, reproducible per-character rendering.Example:
Dependency
pypinyin(MIT, pure-Python) is added as an optional dependency (pip install pyrit[pinyin]) and imported lazily inside the converter, following the pattern used by other optional-dependency converters. It is also added to theallextra. Conversion tests areskipif-gated whenpypinyinis not installed, so they run in thedev_allCI job and skip cleanly in thedevjob.Testing
tests/unit/converter/test_pinyin_converter.pycovering each mode, proportion/mixing, separator handling, passthrough, reproducibility, and validation.ruff check,ruff format --check, andty checkall pass.Closes #2647