Skip to content

The linter and the PostToolUse hook apply English rules to non-English Markdown #41

Open

Description

@mkjung

Problem

evals/ste_lint.py does not identify the language of a sentence. The PostToolUse hook in src/hooks/lint_hook.py runs this linter on each .md file that the model writes or edits. When the linter finds hits, the hook exits with code 2 and tells the model to fix them. For a Korean document, the model then gets English rules for Korean sentences. The model can then change correct Korean text only to remove these hits.

Reproduction

I used main a6fcb4fde0 on Linux with Python 3.14.4. Save this synthetic Korean text as notes.md:

# 백업 점검

백업 작업은 매일 새벽에 돌고, 결과는 저장소의 점검 기록과 운영 문서에 함께 남으며, 실패하면 다음 날 아침에 담당자가 원인을 확인하고 다시 실행하는 절차를 따른다 — 이 절차는 지난달에 정했다.

복구 시험은 한 달에 한 번 한다; 결과는 같은 기록에 남긴다.
python3 evals/ste_lint.py --type descriptive notes.md

The linter reports 3 hits: sentence_over_limit 1, em_dash 1, and semicolon 1. It counts 40 words, and the longest sentence has 29 words.

Next, I gave the hook a PostToolUse event for a Write of this file, in the form that src/hooks/test_lint_hook.py uses. I also used the environment values of that test: CLAUDE_CONFIG_DIR=/nonexistent/claude-config and an empty SIMPLE_ENGLISH_LINT_EXCLUDE. The hook exited with code 2. Its first line was simple-english: notes.md has 3 STE violations. and its last line was Fix these hits in the file you just wrote, then continue.

Scale

I ran the linter on 25 Korean Markdown documents from a real project. It reported 1,187 hits. Of these hits, 656 were on lines that contain Hangul. Most of them were em-dashes and long sentences. The other 531 hits were on English-only lines, and the rules apply to those lines.

Related work

PR #40 adds a translation procedure to SKILL.md. It does not change the linter or the hook. PR #39 split sentences at CJK full stops, and its author closed it without a merge. That change kept the English rules active on non-English text. Its description notes that a Chinese sentence without spaces counts as one word. Korean text has spaces between words, so the 25-word limit applies to Korean sentences in full.

Question

Before I open a PR, I want to know which direction you prefer. One option is to skip each sentence or line in which most letters are not Latin letters. The report then gives the count of skipped sentences. The other option is for the hook to skip a file in which the share of Latin letters is low. In both cases, I will add tests with Korean text and keep the results for English text the same.

I used Claude Code (Claude Opus 5.5) to investigate this issue and to write this text. The results come from actual runs of the commands above.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions