russ.fugl.dev / kiri
Multi-runtime Japanese morphological tokenizer in TypeScript, inspired by Sudachi. Performs Viterbi tokenization with NFKC normalization.
Star
Fork 0
Watch

Remove dead get_text calls from normalize loop (saves O(M×N)) master

The `text` parameter was threaded through do_rewrite/normalize_char but never read — iteration is driven by the original charlist, not the rebuilt string. For M replacements on N-character text, this eliminates M × O(N) wasted work from InputText.get_text rebuilding the array. Also alias Kiri.Dictionary.Converter in kiri.convert mix task (Credo). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author Russ T Fugal Co-author Claude Opus 4.6 Date Commit f651d97b Parent 8f2873fb
+15 -20
2 changed files