clee.sh / qxmx
A single-model (Bonsai-27B) inference engine for Intel Arc GPUs
Star
Fork 1
Watch

Commits on extra-debug

engine: abort with diagnostics on null USM allocations extra-debug
Chris Lee committed
prefill: fix DEVICE_LOST on short prompts (< 16 tokens) main
Chris Lee committed
prefill: batch the tail in one M=16 call (fixes 1.58x serve slowdown)
Chris Lee committed
prefill_cached: fast path when no snapshot group fires + --driver-cpu pin
Chris Lee committed
coarsen prefix-cache snapshot stride + real device banner in CLI tools
Chris Lee committed
update AGENTS.md and README.md
Chris Lee committed
hoist loop-invariant address math in gemv + FFN GEMM kernels (decode +4.5%, prefill +6.7%)
Chris Lee committed
diff: relabel 'B70 eng' -> 'GPU eng' (Arc Battlemage, not B70-specific)
Chris Lee committed
decode gemv: tune split-K target to 512 WGs, add QXMX_GEMV_* knobs (ffn 16.9->16.0)
Chris Lee committed
decode gemv: raise split-K S cap to hide latency (ffn 19.7->16.9 ms)
Chris Lee committed
gguf: support Q2_0 g128 format alongside g64
Chris Lee committed
attn: v3 FD-in-WG decode scan (16.4 tok/s at 63k ctx, beats llama.cpp scan)
Chris Lee committed
attn: GQA-folded decode scan + micro-fixes (2.3x attn, 1.7x TG at 63k ctx)
Chris Lee committed
bench: add qxmx_bench (llama-bench-style pp/tg matrix)
Chris Lee committed
attn: native::exp in decode softmax (6.7% attn at 33k ctx)
Chris Lee committed
serve: add llama.cpp-compatible GET /props endpoint
Chris Lee committed
serve: emit chunked-body terminator on stream completion (llama-benchy fix)
Chris Lee committed
M4.1: multi-slot batched FFN spike (1.78x aggregate at N=16)
Chris Lee committed
attn: T13 deferred WHT in decode (12% attn, 3% decode)
Chris Lee committed
profile: QXMX_PROFILE=2 reports device-busy vs dispatch split
Chris Lee committed
serve: correct prefill tok/s in the request log (cache-aware)
Chris Lee committed
M4: FlashDecoding split-KV decode attention (7.4x attn, 2.8x decode)
Chris Lee committed
qxmx_serve: fix max_tokens
Chris Lee committed
Remove dead code and targets
Chris Lee committed
Add README.md and AGENTS.md
Chris Lee committed
MVP #7: Dockerfile for the post-reorg layout; server validated in-container
Chris Lee committed
docs: M3 complete -- ROADMAP checkmarks, AGENTS state, plan records
Chris Lee committed
M3.2-M3.6: OpenAI endpoint, ChatML template, tool calling, lifecycle
Chris Lee committed
Add plan.md: M3 execution plan + M2.6 cache + M3.3 chat-template notes
Chris Lee committed
M2.6.2+2.6.3: engine restore path + inline SSM snapshotting
Chris Lee committed
M2.6.1: hash-chained KV block cache + SSM checkpoint store
Chris Lee committed
M3.1 follow-up: fix per-slot KV over-allocation, bump default ctx to 64K
Chris Lee committed
M3.1: vendor cpp-httplib + qxmx_serve skeleton
Chris Lee committed
M2.5: multi-slot token-interleaved scheduler (closes M2)
Chris Lee committed
M2.4: incremental UTF-8-safe detokenizer for streaming
Chris Lee committed
M2.3: stop conditions (EOS ids, stop strings, max_tokens, ctx exhaustion)
Chris Lee committed
M2.2: sampling (temperature/top-k/top-p/seed) on engine_i
Chris Lee committed
M2.1: slot-based engine API (open/close/reset/prefill/forward/sample)
Chris Lee committed
Add AGENTS.md (project guide), delete plan.md (M1 working spec, done)
Chris Lee committed
M1 closeout: refresh ROADMAP, trim plan.md to M1-done record (−708 lines)
Chris Lee committed
M1.7c: fused FlashAttention via sycl::joint_matrix -- 741 tok/s (+20%)
Chris Lee committed
FFN GEMM tile tune + naive DeltaNet default -- 613 tok/s (+22%)
Chris Lee committed
Update plan.md + ROADMAP.md: M1.7b done, 500 gate cleared, naive 598 tok/s
Chris Lee committed
ssm_timing_test: add naive (gpu_deltanet_batched) timing line
Chris Lee committed
M1.7b.13: naive DeltaNet barrier reduction -- 599 tok/s (+1.3% over M1.7b.12)
Chris Lee committed
Add runtime chunk size knob (QXMX_CHUNK) + chunk-size sweep
Chris Lee committed
M1.7b.12: naive register-recurrence DeltaNet -- 591 tok/s (+18% over WY/UT)
Chris Lee committed
Refactor: extract DeltaNet recurrence to per-impl files + meson toggle
Chris Lee committed
Dead code cleanup: remove 873 lines of evolutionary dead ends
Chris Lee committed
M1.7b.9+10: FFN gate+up fusion (gemm_gpu_v4_dual) + GEMM GPP=16 tile tune -- 500 tok/s
Chris Lee committed
M1.7b.8: WY/UT UT-solve opt -- precompute+fuse G into KK, 481 tok/s
Chris Lee committed
M1.7b.7: 3-kernel FA split (DPAS QK + softmax + DPAS PV) -- 439 tok/s
Chris Lee committed
M1.7b.6: plain-SYCL scalar FA kernel -- 230 tok/s (was 176 M1.7b.4)
Chris Lee committed
Reorganize source tree: src/ + tools/ + tests/
Chris Lee committed
Remove accidental build artifact + gitignore it
Chris Lee committed
M1.7b.4: ESIMD-vectorized FA kernel (no DPAS) -- 2.35x faster attn
Chris Lee committed
M1.7b.3: wire fused FlashAttention into attn_forward_b
Chris Lee committed
M1.7b.1: fused FlashAttention-2 kernel (correct, no DPAS)
Chris Lee committed
M1.7b.0: T13 rotated-domain V accumulator CPU harness (fa_t13_test)
Chris Lee committed
M1.7a: wire GEMM WY/UT into engine (ssm_forward_b), 2x faster DeltaNet
Chris Lee committed