src · qxmx · Tangled

A single-model (Bonsai-27B) inference engine for Intel Arc GPUs

qxmx src

40 files

Chris Lee · engine: abort with diagnostics on null USM allocations · 2mo ago

gguf: support Q2_0 g128 format alongside g64

gguf: support Q2_0 g128 format alongside g64

qxmx_chat_template.cpp

M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle

qxmx_chat_template.h

M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle

M2.5: multi-slot token-interleaved scheduler (closes M2)

M2.5: multi-slot token-interleaved scheduler (closes M2)

qxmx_deltanet_common.cpp

Refactor: extract DeltaNet recurrence to per-impl files + meson toggle

qxmx_deltanet_naive.cpp

prefill: batch the tail in one M=16 call (fixes 1.58x serve slowdown)

qxmx_deltanet_wyut.cpp

engine: abort with diagnostics on null USM allocations

qxmx_deltanet.h

prefill: batch the tail in one M=16 call (fixes 1.58x serve slowdown)

qxmx_device.cpp

engine: abort with diagnostics on null USM allocations

engine: abort with diagnostics on null USM allocations

qxmx_engine.cpp

M2.2: sampling (temperature/top-k/top-p/seed) on engine_i

serve: correct prefill tok/s in the request log (cache-aware)

qxmx_fa_fused.cpp

engine: abort with diagnostics on null USM allocations

M1.7c: fused FlashAttention via sycl::joint_matrix -- 741 tok/s (+20%)

M1.7c: fused FlashAttention via sycl::joint_matrix -- 741 tok/s (+20%)

Reorganize source tree: src/ + tools/ + tests/

qxmx_gemm_tf32.cpp

Reorganize source tree: src/ + tools/ + tests/

qxmx_gemm_tf32.h

Reorganize source tree: src/ + tools/ + tests/

hoist loop-invariant address math in gemv + FFN GEMM kernels (decode +4.5%, prefill +6.7%)

M1.7b.9+10: FFN gate+up fusion (gemm_gpu_v4_dual) + GEMM GPP=16 tile tune -- 500 tok/s

engine: abort with diagnostics on null USM allocations

engine: abort with diagnostics on null USM allocations

gguf: support Q2_0 g128 format alongside g64

M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle

gguf: support Q2_0 g128 format alongside g64

gguf: support Q2_0 g128 format alongside g64

qxmx_prefix_cache.cpp

M2.6.2+2.6.3: engine restore path + inline SSM snapshotting

qxmx_prefix_cache.h

Remove dead code and targets

gguf: support Q2_0 g128 format alongside g64

Reorganize source tree: src/ + tools/ + tests/

qxmx_serve: fix max_tokens

qxmx_scheduler.cpp

M2.5: multi-slot token-interleaved scheduler (closes M2)

qxmx_scheduler.h

M2.5: multi-slot token-interleaved scheduler (closes M2)

M2.3: stop conditions (EOS ids, stop strings, max_tokens, ctx exhaustion)

qxmx_tokenizer.cpp

M2.4: incremental UTF-8-safe detokenizer for streaming

qxmx_tokenizer.h

M2.4: incremental UTF-8-safe detokenizer for streaming

qxmx_tool_call.h

M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle

gguf: support Q2_0 g128 format alongside g64