Repository navigation
feat(i18n): CJK readiness — regional UI font fallbacks, grapheme-safe cuts, IME checks - #885
Merged
Merged
Conversation
Phase 3 of docs/i18n.md. Han characters share codepoints across China, Taiwan, Hong Kong, Japan, and Korea, so the platform cascade draws them in whichever region's shapes it finds first. The sidebar, tab, and interface faces now name each platform's CJK families after the icon fonts: PingFang and Hiragino on macOS, Microsoft YaHei/JhengHei UI and Yu Gothic UI on Windows, Noto Sans CJK (or Source Han Sans, WenQuanYi, Droid) on Linux, one installed family per script. - herdr_i18n::CjkScript reads which glyphs a language wants from its script and region (zh-Hans, zh-TW, zh-HK, yue, ja, ko). - The script of the first CJK language in the `language` setting, else in the system's languages, goes first; the setting counts before any translation in that language ships. - The terminal keeps the icon cascade alone, and an explicit `fallback` still opts a face out.
The app's own ellipses cut text after a count of `char`s, which can split what reads as one character: a letter and its combining accent, an emoji joined by zero-width joiners, a flag, or a Hangul syllable spelled as jamo. truncate.rs cuts between grapheme clusters instead. - Terminal and browser notes, review notes, sidebar search excerpts, and the issue report's logs count grapheme clusters. - A browser tab's title and an orchestrator check pill have a width budget, so they count columns: a CJK title is cut about as wide as a Latin one, not twice as wide. GPUI's width-based text_ellipsis still cuts between chars inside GPUI; docs/i18n.md records that.
…elds Search fields moved the caret and deleted by `char`, so Backspace after an accented letter, an emoji sequence, or a jamo-spelled Hangul syllable left part of it behind. They now move by grapheme cluster, as dialog fields already did. Headless tests compose Pinyin, a Japanese clause, and Zhuyin past the field's edge, and check where the candidate window is asked to open. docs/i18n.md records what Phase 3 built.
Owner
Author
 [Medium impact] The PR appears safe to merge, with a non-blocking issue in blank-comment checks. Findings
SummaryThis PR adds regional CJK font fallbacks and cuts text without splitting visible characters.
Reviews (2) · Last reviewed commit: "fix: keep cut notes bounded and the care..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Follow-up to review on #885. - truncate::cut_chars cuts at the last whole grapheme cluster within a character budget. A quoted terminal line uses it, so a letter carrying thousands of combining marks is capped at 300 characters again. - The three identical one-line note helpers (terminal, browser, review) become truncate::one_line. Its safety cut now ends on a whole cluster and marks the cut, so a run of emoji sequences no longer ends in half of one. - Search and dialog fields find caret boundaries in the whole text, and an edit whose text joins the cluster after it (a letter typed before an accent) leaves the caret after that cluster, so Delete and Backspace never split it.
Owner
Author
|
@greptileai review |
one_line marked its safety cut even when nothing but whitespace was left, so a blank comment longer than the budget became "…" and was kept as a note. Text that is all breaks and controls now stays empty and is refused; text whose first character alone exceeds the budget still shows the ellipsis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 of
docs/i18n.md: CJK readiness. Phase 2's remaining string migration is not part of this.Regional font fallbacks for the interface faces. Han characters share codepoints across China, Taiwan, Hong Kong, Japan, and Korea, so the platform cascade draws them in whichever region's shapes it finds first. When a face's
fallbackis unset, the sidebar, tab, and interface faces now search the installed icon fonts, then one installed CJK family per script:herdr_i18n::CjkScriptreads which glyphs a language wants from its script and region (zh-Hans,zh-TW,zh-HK,yue,ja,ko).languagesetting, or else in the system's languages, comes first. The setting counts even before a translation in that language ships, because the glyphs matter either way.fallback, even[], still opts a face out.Grapheme-safe cuts.
truncate.rscuts between grapheme clusters, so a cut never splits an accented letter, an emoji sequence, a flag, or a Hangul syllable spelled as jamo. It is used for terminal, browser, and review notes, sidebar search excerpts, and the issue report's logs. A browser tab title and an orchestrator check pill have a width budget, so they count columns, and a CJK title is cut about as wide as a Latin one instead of twice as wide. Search fields now move the caret and delete by grapheme, as dialog fields already did.IME. I audited the three input paths:
SearchInput(palette, Settings, sidebar and theme search),DialogInput(rename and other dialogs), and the terminal. Composition, commit, and placement tests already covered dialogs, the terminal painter, the Settings fields, and the menus.SearchInputhad no candidate-placement test, so this PR adds headless tests for Pinyin, a Japanese clause, and Zhuyin composed past a scrolled field's edge.Known limitation: GPUI's own width-based
text_ellipsiscuts betweenchars inside GPUI. CJK text is unaffected, but a combining mark or emoji sequence at that cut can lose its tail. This is recorded indocs/i18n.md.Screenshots
Captured from the release build on macOS against an isolated daemon (system languages en-US, fr-FR), with a workspace named
直角骨 写次 中文工作区. These characters take visibly different shapes by region.language = "ja"language = "zh-TW"language = "zh-CN""Before" is this build with
fallback = [], which gives the same cascade asmain. On this machine it draws Simplified shapes whatever thelanguagesetting says.Native Pinyin composition in the command palette, and the candidate list it produced:
QA
just format: cleanjust ci: passed on 78562ab (macOS arm64)just check-commits: OKjust lint-linuxandjust lint-windows: passed (run on the same tree before it was split into commits)herdr_i18n::tests::cjk(script by language),config::tests::cjk_fonts(per-platform selection, order, dedup, terminal kept separate),truncate::tests(clusters, CJK, column widths),search_input::tests::composition(Pinyin, Japanese clause, Zhuyin past the edge, grapheme deletion)language(ja, zh-TW, zh-CN) on a fresh launch and on a live config reloadSearchInputanswering the OS's placement queries with the field's real bounds. To check: enable Pinyin, open the command palette, typezhongwen, and confirm the candidates open under the field. Repeat in Settings search, sidebar search, a rename dialog (Rename Workspace), and a terminal pane.language = "ja"vs"zh-CN"; fcitx or ibus candidate placement