Peel the #671 credential words off a glued name, except 部長; name_usage answers ends-with for CJK

[derek73]

Closes #707.

What changes

tools/name_usage.py can now answer "does any name END in this word?" for a CJK word. That is the question the glued-honorific peel rests on (rules.md#W2). The Census and SSA rows now say they do not apply to a CJK word, and a new row, Wikidata people, name ends with, gives:

• the bearers of every Wikidata name item whose CJK label ends in the word, with countries of citizenship, and each matched item's Q-id so a transcription or mislabel can be traced;
• people whose family name ends in the word's head and whose given name is its tail: the 阿部 + 長 shape Can 陳怡君二世 and 田中医学博士 lose their glued credential word? name_usage cannot answer for CJK words #707 raised against 部長.

The #671 words peel off a glued name, except 部長/部长. 議員/议员, 国会議員/國會議員, 太平紳士/太平绅士, 医学博士/醫學博士, 二世, 社長/課長 and 社长/课长 join GLUED_HONORIFICS. 部長/部长 stay spaced-only on the 阿部 + 長 argument.

Input	2.4.2	Now
田中社長	family 田中社長	family 田中, suffix 社長
陳怡君二世 (locales.ZH)	family 陳, given 怡君二世	family 陳, given 怡君, suffix 二世
エリザベス二世	family エリザベス二世	given エリザベス, suffix 二世
田中部長	family 田中部長	unchanged

How the tool answers it

Live SPARQL can't: STRENDS over every label of a name class timed out at WDQS's 60s limit even with no person join, and WDQS full-text search found 4 family names ending in 殿, matching only where its tokenizer split the character out. So:

1. Each of eleven name-item classes is listed once, as CSV. The listing is held to a COUNT taken just before it, because 73MB of JSON was cut off mid-stream and CSV has no closing bracket to show a cut. Lexemes, which also carry P31 family name, are skipped.
2. Labels are fetched by VALUES batch of 1,000 items, keeping CJK-script labels only. Slicing a class by Q-id suffix inside one query doesn't work, because it scans the whole class every time. A batch that overruns is halved. Finished batches are cached, so a stopped build resumes; 429 responses wait out Retry-After.
3. Matching is local; only the matching items' bearers are counted live.

The first build took 33 minutes: 215,798 labels on 128,695 items, cached outside the repo. Later runs take seconds; --refresh-index rebuilds.

Measurements (2026-10-17)

• The index is complete relative to Wikidata: it matched live STRENDS exactly on 20 class × word cells (five subclasses × 殿, 子, 郎, 田; the largest cell had 403 items).
• Positive controls: 子 matches 439 items and 2,436 given-name bearers, 2,081 of them in Japan; the boundary shape 田 + 太郎 counts 22 people.
• The Can 陳怡君二世 and 田中医学博士 lose their glued credential word? name_usage cannot answer for CJK words #707 words: no name item ends in any of the 15 spellings, except one mislabel: the given name Odo (Q18339792), whose Chinese label is 厄德二世, with 109 Western bearers. Every boundary pair that exists (二+世, 社+長, 社+长, 部+長, 部+长) counts 0 people.
• The limit is Wikidata's coverage, not the index: 殿 ends 13 name items, where a surname dictionary lists about 90.
• Corpus moves: the snapshot moved four CLDR renderings and 陳怡君二世, each losing the glued word to the suffix. cldr_agreement gains one agreeing suffix field (二世 醫學博士) and loses none. The JA locale view, run with the ja extra, doesn't move.

Docs

• rules.md W2: examples updated, Accepted clause narrowed to 部長/部长
• decisions.md#W2: new entry with the counts and a recompute command
• usage.rst: the glued-honorific section and its doctest
• release log: the Chinese and Japanese credential words aren't recognized: 王雅婷，議員 reads given 議員, 陳怡君二世 reads given 怡君二世 #671 bullet is in the same Unreleased block, so it is merged with this change
• AGENTS.md: how to vet a glued word, and reading ends-with hits by item

Commits

1. 5faae975 the tool (its tests pass on their own)
2. e70b8342 the vocabulary change, case rows, records and docs
3. 41e0727a merge of master (Ivoi, Paul d' and Zaʿīm, Ḥusnī az-: a particle written after the given name isn't joined to the family #700); AGENTS.md conflict resolved keeping both additions
4. 05087fe8, 567b9452 the review fixes above
5. d6531ef7, 45e24510 the PR-review fixes below

PR review (code, tests, silent failures, comments, types)

Fixed in d6531ef7:

• CI on 3.14/3.15: Python 3.14 warns when an HTTPError is left unclosed, and the 429 retry discarded errors without closing them. The retry loop and the overrun path now close each error. The tool tests pass under 3.14.7 and 3.15.0.
• False zeros closed:
  • an empty class listing is refused, rather than cached as [];
  • corrupt cache files raise a ValueError that names the file and --refresh-index, rather than a KeyError that took every source's answer down;
  • each batch records the first and last item it covers and is re-fetched when the listing no longer matches;
  • a CSV listing cut mid-line is an error;
  • the character class takes supplementary-plane Han and 〇, and words are NFC-normalized;
  • the cache is versioned, and the old one was rebuilt.
• Pairing counts: a failed pairing count no longer discards the row's item counts; it fails only that pairing, and the exit status is still 2. Pairings are counted in chunks over distinct people.
• Role check: NAME_CLASSES roles are checked at import.
• Tests: added for each fix above, and for the reviewers' coverage gaps (two classes in one build, every cut of a longer word, both edges of the count's slack, 429 exhaustion).
• Docs: comment and doc corrections. One sentence was made false by this PR's own examples: "the corpora hold no compound office word".

The rebuilt index (215,820 entries) gives the same answers for 殿, every #707 word and the boundary pairs.

Design-docs review

The eleven-axis review ran on 41e0727a and found four doc problems, all fixed:

• Compound office words split at the new tail (kept as an accepted cost). The vetting asked whether a name ends in each word, never whether a longer unlisted office word does. So 田中 副社長 reads given 副, suffix 社長, and 岸田文雄衆議院議員 reads family 岸田文雄衆議院, suffix 議員; 獣医学博士 and 副課長 split the same way. Before, the whole word stayed in the name, which was no more right. Recorded in rules.md W2 (a second Accepted clause), the Can 陳怡君二世 and 田中医学博士 lose their glued credential word? name_usage cannot answer for CJK words #707 decisions entry and suffixes.py. AGENTS.md's vetting recipe gains the question "does a common longer word end in it?". (567b9452)
• 部長/部长 added to decisions.md's honorific_tails Excluded block. (05087fe8)
• The release bullet now says 部長 is read after a comma and under locales.ZH after a listed surname. (05087fe8)
• usage.rst no longer says no name in Wikidata ends in these words (厄德二世 does). (05087fe8)

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.03%. Comparing base (8cd815c) to head (45e2451).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #712   +/-   ##
=======================================
  Coverage   99.03%   99.03%           
=======================================
  Files          46       46           
  Lines        4557     4557           
=======================================
  Hits         4513     4513           
  Misses         44       44           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.