Peel the #671 credential words off a glued name, except 部長; name_usage answers ends-with for CJK

AGENTS.md

@@ -253,7 +253,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_
 
 **Unknown-key attribute access on `TupleManager`/`RegexTupleManager` warns (1.4, #256) — 2.0 note: the warning became a hard `AttributeError` naming the known keys (shim `TupleManager`); the paragraph below describes the deleted v1 machinery and applies only when reading v1 history.** — a key not currently in the dict emits `DeprecationWarning` naming the miss and the known keys (`_warn_unknown_key`), before falling back to the same `None`/`EMPTY_REGEX` default as before; `.get()` stays silent. Dunder probes (`__deepcopy__`) still raise `AttributeError` outright, and single-underscore probes (`_repr_html_`, IPython's `_ipython_canary_method_should_not_exist_`, etc.) are excluded from the warning too — no real config key starts with `_`, so both guards just keep protocol/introspection probes from misfiring as "typo" warnings. This means internal parser code that reads `self.C.regexes.<key>` unconditionally (e.g. `squash_bidi`'s `bidi`) now warns if a caller's custom `regexes` dict omits that key — a previously-silent partial-override pattern is on the same deprecation path as an actual typo.
 
-**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PARTICLES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PARTICLES` ∩ `BOUND_GIVEN_NAMES` (position-dependent: leading token → given-name join, mid-name → family-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `BOUND_GIVEN_NAMES`). Then check the word against real names, not memory: `uv run python tools/name_usage.py <word>` counts its bearers as a given name and as a family name (US Census surnames, SSA given names, Wikidata with countries of citizenship). Pass a name as it is written (`McDonald`): Wikidata labels match exactly, and no rule rebuilds an internal capital from `mcdonald`. It reports and does not decide — read the counts against the position the set's claim acts on. And check the predicates that test a member's SPELLING rather than its membership: they encode what the set's members happened to look like when they were written. `_pieces.starts_a_credential_run` implemented rules.md#S2's "Latin" as `isascii()`, exact until #661 added `mlađi`, which would have read `mlađi Jones` differently from `stariji Jones`; the design-docs review caught it, after a first draft had credited the test to the wrong clause of S2 and proposed deleting it on a mutation that moved nothing because nothing pinned it. (#679 then dropped the script test on purpose, after measuring what it protected.) Before adding the first member of a new script, or the first accented one, grep the set's consumers for `isascii`, `isalpha` and script tests, and read the rule each one implements. The same goes for the first member holding PUNCTUATION: #700 added `d'` and `az-`, the parse read them right and every snapshot move was an intended one, and `capitalized()` still wrote `D' Ivoi`, because case repair splits a token at its apostrophe or hyphen and asks the vocabulary of the stem. `capitalized()` is invisible to the snapshot (`initials()` is recorded there), so parse such a member's names through `capitalized()` and `capitalized(force=True)` in each case before calling it done.
+**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PARTICLES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PARTICLES` ∩ `BOUND_GIVEN_NAMES` (position-dependent: leading token → given-name join, mid-name → family-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `BOUND_GIVEN_NAMES`). Then check the word against real names, not memory: `uv run python tools/name_usage.py <word>` counts its bearers as a given name and as a family name (US Census surnames, SSA given names, Wikidata with countries of citizenship). Pass a name as it is written (`McDonald`): Wikidata labels match exactly, and no rule rebuilds an internal capital from `mcdonald`. It reports and does not decide — read the counts against the position the set's claim acts on. A word that would peel GLUED (`GLUED_HONORIFICS`) asks a different question, whether any name ENDS in it, and for a CJK word the tool answers that too: an ends-with row from a cached index of Wikidata's CJK name labels (built on first use; 33 minutes on 2026-10-17), plus the names the word ends across the family/given join (阿部 + 長 for 部長). Read the tool's hits by item before counting them, because labels include transcriptions and mislabels: #707's one hit for 二世 was 厄德二世, the given name Odo with a regnal numeral in its Chinese label, borne by 109 Western people named Odo. Then ask a third question the tool cannot: does a common longer word end in it? The peel matches the end of a token, spaced or glued, so a listed tail splits every unlisted compound that ends in it — #707 vetted 議員, 社長 and 医学博士 against names and its review found 衆議院議員, 副社長 and 獣医学博士 cut at the tail (`田中 副社長` → given 副); no corpus held a compound office word until #707's rules.md examples added two, so the snapshot was silent about the class. List the common compounds whole or record the split as a cost. And check the predicates that test a member's SPELLING rather than its membership: they encode what the set's members happened to look like when they were written. `_pieces.starts_a_credential_run` implemented rules.md#S2's "Latin" as `isascii()`, exact until #661 added `mlađi`, which would have read `mlađi Jones` differently from `stariji Jones`; the design-docs review caught it, after a first draft had credited the test to the wrong clause of S2 and proposed deleting it on a mutation that moved nothing because nothing pinned it. (#679 then dropped the script test on purpose, after measuring what it protected.) Before adding the first member of a new script, or the first accented one, grep the set's consumers for `isascii`, `isalpha` and script tests, and read the rule each one implements. The same goes for the first member holding PUNCTUATION: #700 added `d'` and `az-`, the parse read them right and every snapshot move was an intended one, and `capitalized()` still wrote `D' Ivoi`, because case repair splits a token at its apostrophe or hyphen and asks the vocabulary of the stem. `capitalized()` is invisible to the snapshot (`initials()` is recorded there), so parse such a member's names through `capitalized()` and `capitalized(force=True)` in each case before calling it done.
 
 **Before adding a short/common word to `PARTICLES` globally**, test it mid-string against realistic 3-token names, not just check for English-word collisions: Korean/Vietnamese given names put a short syllable in the middle slot (`Park In Hwan`, `Nguyen To Nga`), and Western names put a bare initial there (`John V. Smith`). A word that looks safe ("nobody is named 'to'") can still swallow a real middle name/initial into the last name once it's a global prefix — confirmed regressions for `to`/`in`/`an`/`ten`/`then` and bare `v` this way (PR #191).
 

docs/design/decisions.md

@@ -658,7 +658,8 @@ Declined:
 - 2026-08 #308 — an entry peels only where it can never end a name: 씨/님/さん/様/先生 peel; 양/군/氏/博士/殿 stay spaced-only; 君 is in neither set while its kana spelling くん peels. The vocabulary carries the license, so no structural or per-script gate stands over the peel.
 - 2026-08 #312 — the peel crosses the family comma and the 间隔号: both answer where a name DIVIDES into surname and given, a question the peel never asks. Provenance matters here: neither gate was argued for the peel specifically — both came with the placement (#312's own framing) — so the crossing was a repair of inherited gates, not a designed-in property.
 - 2026-08 #319 — a wholly suffix-shaped second run is declined as a peel site (the "田中さん, V." shape), but only when the name's own run offers a site, since a glued honorific is itself part of what makes a run read as suffix-shaped.
-- 2026-10-17 #671 — SPACED AND AFTER-COMMA ONLY, no glued peel (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世 and the job titles 社長/部長/課長 and the simplified 社长/部长/课长 (Derek, 2026-10-17: the characters are shared and the offices are real in both languages; no borne-name data either way, #707) join `SUFFIX_WORDS`; none joins `GLUED_HONORIFICS`. The spaced bar is only that a word standing alone is no name, so they ship; the glued bar ("can it END a name?") needs real-name counts that this change did not get — the Wikidata suffix query timed out and the native-script Wikidata corpus holds none of the words — and 部長 fails it on argument alone (長 is a borne given name, 阿部長). A follow-up can vet 医学博士, 議員 and 太平紳士 one at a time. Consequence measured on `parser_for(locales.ZH)`: after a listed surname each word divides off, `王二世` family 王, suffix 二世 (it read given 二世) and `李部長` family 李, suffix 部長, because the segmenter's division is not a peel and the split-off 二世 is then a whole-token suffix word, the way 王博士 already reads; `陳怡君二世` still reads given 怡君二世 (glued, nothing divides it). ジュニア was already shipped by #661, and #679 declined 二世 as CLDR's generation word ("regnal, and glued"); here it ships spaced-only as the Chinese/Japanese numeral, which does not reverse that. Excluded as the issue stated: 小 (a prefix and a common given-name character) and 冯/馮 for "von" (a top-30 surname).
+- 2026-10-17 #671 — SPACED AND AFTER-COMMA ONLY, no glued peel (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世 and the job titles 社長/部長/課長 and the simplified 社长/部长/课长 (Derek, 2026-10-17: the characters are shared and the offices are real in both languages; no borne-name data either way, #707) join `SUFFIX_WORDS`; none joins `GLUED_HONORIFICS`. The spaced bar is only that a word standing alone is no name, so they ship; the glued bar ("can it END a name?") needs real-name counts that this change did not get — the Wikidata suffix query timed out and the native-script Wikidata corpus holds none of the words — and 部長 fails it on argument alone (長 is a borne given name, 阿部長). A follow-up can vet 医学博士, 議員 and 太平紳士 one at a time. Consequence measured on `parser_for(locales.ZH)`: after a listed surname each word divides off, `王二世` family 王, suffix 二世 (it read given 二世) and `李部長` family 李, suffix 部長, because the segmenter's division is not a peel and the split-off 二世 is then a whole-token suffix word, the way 王博士 already reads; `陳怡君二世` still reads given 怡君二世 (glued, nothing divides it; superseded for the glued question by the #707 entry below). ジュニア was already shipped by #661, and #679 declined 二世 as CLDR's generation word ("regnal, and glued"); here it ships spaced-only as the Chinese/Japanese numeral, which does not reverse that. Excluded as the issue stated: 小 (a prefix and a common given-name character) and 冯/馮 for "von" (a top-30 surname).
+- 2026-10-17 #707 — the #671 words meet the glued bar, all but 部長/部长 (Derek, 2026-10-17). 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世, 社長/課長 and 社长/课长 join `GLUED_HONORIFICS`; 部長/部长 stay spaced-only on the #671 argument, since 長 is a borne given name (Wikidata holds it as a Chinese given name, Q96240389; the Japanese reading rests on the argument) and 阿部長 can be 阿部 + 長 — the measurement below found no such bearer, and a zero that is a lower bound does not refute the argument. The measurement is `tools/name_usage.py`'s ends-with row, new in #707: a local index of every CJK label on eleven name-item classes (family name, given name and their Japanese, Chinese, Korean, male, female, unisex and undifferentiated-kanji subclasses; 215,798 index entries on 128,695 items, 215,696 distinct item-label pairs, fetched 2026-10-17 by the first index, whose character class stopped at the Basic Multilingual Plane; the rebuilt index that takes supplementary-plane Han too (215,820 entries, same day) gave the same answer for every word, 殿 and the boundary pairs below), matched locally, with the matching items' bearers counted live. Two questions per word: does any name item's label END in it, and does any person bear a family name ending in the word's head and a given name equal to its tail (the 阿部 + 長 shape)? Neither found anything for any spelling, with one exception that is a data error: the only item ending in 二世 is Q18339792, the given name Odo, whose Chinese label carries a regnal numeral (厄德二世), and its 109 bearers are Western people named Odo. Of the boundary shapes, only 二+世, 社+長, 社+长, 部+長 and 部+长 had a family name and a given-name item to pair at all, and each counted 0 people. What makes the zeros evidence: the index agreed exactly with a live `STRENDS` query on all 20 class × word cells it was cross-checked on (five subclasses × 殿, 子, 郎, 田, the largest 403 items); 子 found 439 items and 2,436 given-name bearers, 2,081 of them Japanese; and 田 + 太郎 counted 22 people across the boundary. What limits them: Wikidata's coverage, not the index's — 殿 ends 13 name items where a surname dictionary lists about 90, and Japanese family-name items number 4,919. Two shortcuts were measured and rejected before the index: `STRENDS` over every label of a name class timed out at WDQS's 60s with no person join, and WDQS full-text search found 4 family names ending in 殿, matching only where its tokenizer split the character out. Consequence: the snapshot moved four CLDR renderings, each losing its glued word to `suffix` (`王雅婷婷婷二世議員`, both spellings of `…小惠儀太平紳士`, `馮·陳·怡君·達印·二世醫學博士`), and `陳怡君二世`, which the case table and rules.md W2 pin; cldr_agreement records one suffix field newly agreeing (`二世 醫學博士`, where the peel takes 醫學博士 and the 二世 left behind is a whole-token suffix word) and none regressing. The peel still takes one word: `王雅婷婷婷二世議員` peels 議員 and keeps 二世 in the name. Recompute: `uv run python tools/name_usage.py 議員 议员 国会議員 國會議員 太平紳士 太平绅士 医学博士 醫學博士 二世 社長 部長 課長 社长 部长 课长`; the first run builds the index (33 minutes on 2026-10-17, resumable), and `--refresh-index` rebuilds it. ACCEPTED COST, found by the design-docs review and kept (Derek, 2026-10-17): the vetting asked whether a NAME ends in each word and never whether a longer, unlisted OFFICE word does, and common ones do — 衆議院議員 and 参議院議員, 副社長 and 代表取締役社長, 元議員 and 市议员, 副課長, 獣医学博士. The peel matches at the end of the token, spaced or glued, so each splits at the listed tail: `岸田文雄衆議院議員` reads family 岸田文雄衆議院, suffix 議員, and `田中 副社長` given 副, suffix 社長, where before the word stayed whole in the name. Neither reading is right; the compound's head is what the peel leaves behind. No corpus held a compound office word when the review found this, which is why the snapshot was silent; the two W2 examples now put `岸田文雄衆議院議員` and `田中 副社長` in corpus_rules.jsonl. Older entries carry the same cost unrecorded until now (`김민준의학박사` reads given 민준의학, suffix 박사). Listing a compound whole would peel it whole, longest first.
 
 Excluded (the never-given / ambiguous particle line, nameparser/config/particles.py — #360 owns the vocabulary question):
 
@@ -961,6 +962,7 @@ Excluded (Lexicon.honorific_tails — a glued tail peels only if it could never
 - 博士 — 田中博士 is Tanaka Hiroshi as readily as Doctor Tanaka.
 - 殿 — Japanese surnames end in it (鵜殿, 真殿, four-figure populations); peeling it would cut a real family name in two.
 - 君 — 王君 is a complete Chinese name; its kana spelling くん does peel.
+- 部長, 部长 — 長 alone is a borne given name, so 阿部長 can be 阿部 + 長 (#707). The measurement found no such bearer, as it found none for the words that shipped; the exclusion rests on the argument, which a lower-bound zero does not refute, so a re-measure that again comes back zero is no reason to add them.
 - जी, and with it the whole spaced trailing set the 2026-09-06 Indic bundle shipped — जी, साहब, साहिब, साहेब, महाराज, সাহেব, বাবু, মহারাজ and Latin rinpoche. None is a honorific_tails entry and none may become one. The evidence is जी's and it carries the set: Banerjee, Mukherjee and Chatterjee end in the substring (बनर्जी/मुखर्जी/चटर्जी), and glued peeling strands a fragment on a bare virama (बनर् + जी). The 殿 criterion, in a non-CJK script — which also shows the criterion is not CJK-specific. The accepted cost is that गांधीजी stays unpeeled, and it is the right trade: an unrecognized honorific costs one field, a strand cuts a real family name in two. Both shapes are pinned in tests/v2/test_locales.py. See #indic-honorifics.
 
 Excluded (TITLES):