Skip to content

fix(citations): escape substring quotes before fuzzy span matching#2430

Open
RohitMudili wants to merge 1 commit into
567-labs:mainfrom
RohitMudili:fix/citation-regex-escape
Open

fix(citations): escape substring quotes before fuzzy span matching#2430
RohitMudili wants to merge 1 commit into
567-labs:mainfrom
RohitMudili:fix/citation-regex-escape

Conversation

@RohitMudili

@RohitMudili RohitMudili commented Jul 11, 2026

Copy link
Copy Markdown

Describe your changes

CitationMixin._get_span interpolated the LLM-generated quote directly into a
regex fuzzy-match pattern:

minor = quote
s = regex.search(f"({minor}){{e<={errs_}}}", major)

Because the quote is never escaped, any regex metacharacter in the model's
substring_quotes — unbalanced parentheses/brackets, *, +, ?, \, etc.,
all common in real prose — is interpreted as part of the pattern. This either
raises regex.error (crashing validation) or silently matches the wrong span,
instead of the documented behavior of finding the span or dropping the quote.

Reproduction:

context = "The margin is 50% (approx) this quarter."
Answer.model_validate(
    {"substring_quotes": ["50% (approx"]},   # unbalanced paren from the LLM
    context={"context": context},
)
# regex.error: missing ) at position 19

Fix

Escape the quote with regex.escape() before building the pattern so it is
matched literally. Fuzzy (edit-distance) matching is unaffected — a quote with a
one-character typo still resolves within the default edit budget.

Changes

  • instructor/v2/dsl/citation.py: regex.escape(quote) before the fuzzy search.
  • tests/v2/test_citation.py: new tests — metacharacter quotes resolve instead of crashing, non-matching quotes are dropped, fuzzy matching still works, and no-context input is untouched. (These fail on main and pass with the fix.)
  • CHANGELOG.md: entry under [Unreleased].

Issue ticket number and link

Closes #2431#2431

Checklist before requesting a review

  • I have performed a self-review of my code
  • If it is a core feature, I have added thorough tests.
  • If it is a core feature, I have added documentation.

CitationMixin._get_span interpolated the LLM-generated quote directly
into a regex fuzzy pattern (f"({minor}){{e<={errs_}}}") without escaping.
Quotes containing regex metacharacters (e.g. unbalanced parentheses or
brackets, common in real text) crashed validation with regex.error
instead of resolving the span or being dropped.

Escape the quote with regex.escape() so it is matched literally. Fuzzy
(edit-distance) matching is unaffected.
@RohitMudili

Copy link
Copy Markdown
Author

Hey @jxnl @ivanleomk — this fixes the regex.error crash in CitationMixin reported in #2431. When the model returns a substring_quotes value containing regex metacharacters (unbalanced parentheses/brackets, *, \, etc. — common in ordinary prose), validation blows up with regex.error instead of resolving the span or dropping the quote. Would appreciate a review when you get a chance!

TL;DR: _get_span() interpolated the raw LLM quote straight into a fuzzy pattern f"({minor}){{e<={errs_}}}" with no escaping. Wrapped it in regex.escape() so the quote is matched literally; fuzzy (edit-distance) matching is unaffected. Includes regression tests that fail on main and pass with the fix.

@ErenAta16

Copy link
Copy Markdown

Applied this locally and ran the new test file against current main: all 8 pass, including test_fuzzy_matching_still_works_after_escaping, which was the thing I wanted to check, regex.escape() on the quote before wrapping it in the {e<=N} fuzzy syntax doesn't interfere with the edit-distance matching itself, since escaping only affects how literal characters are interpreted, not the fuzzy-match operator appended outside the escaped substring.

Only blocker is the DIRTY merge state, looks like just a CHANGELOG.md conflict from newer entries landing since this was opened, the actual fix and tests applied cleanly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CitationMixin crashes with regex.error when a quote contains regex metacharacters

2 participants