Something went wrong. Try again.
Jam Programming Language jamlang.org
compiler
Something went wrong. Try again.
add byteOffset to Token (Zig Ast.TokenList shape) master
Match Zig's persistent per-token storage: { tag, start } only. The lexer captures tokenStart at the top of each scan-loop iteration before the first advance(), and addToken stamps it onto every emitted Token. Tokens still carry lexeme + line for compat with existing parser / init-analysis consumers; those can drop later once consumers migrate to byte-offset-only access. End offsets are deliberately not stored — Zig recomputes them via tokenSlice() re-tokenizing from start (short-circuit on tag.lexeme() for keywords/operators).
Author Raphael Amorim Date (May 12, 2026, 8:12 AM UTC) Commit e28ca9a2 e28ca9a2157a17e0ecb626e40ead99a8f10180a5 Parent 92612970 92612970b32f5fb711a4588b9a96ab465a359142