Multi-runtime Japanese morphological tokenizer in TypeScript, inspired by Sudachi. Performs Viterbi tokenization with NFKC normalization.
kiri bench benchmarks.html
29 kB · 982 lines
HTML
at main