Jam Programming Language jamlang.org
compiler

add byteOffset to Token (Zig Ast.TokenList shape) master

Match Zig's persistent per-token storage: { tag, start } only. The lexer captures tokenStart at the top of each scan-loop iteration before the first advance(), and addToken stamps it onto every emitted Token. Tokens still carry lexeme + line for compat with existing parser / init-analysis consumers; those can drop later once consumers migrate to byte-offset-only access. End offsets are deliberately not stored — Zig recomputes them via tokenSlice() re-tokenizing from start (short-circuit on tag.lexeme() for keywords/operators).


+30 -4
3 changed files