Writing
Notes on LLM architectures, inference engines and serving systems.
More throughput from fewer GPUs: profiling and fixing GLM-5.3 prefill on NVIDIA B300
Oct 8, 2026 · 23 minHow a sharded KV store, a FlashInfer kernel fix and per-rank Valkey shards got GLM-5.3 on B300 65% more throughput on 5 nodes instead of 11.
vLLM v0.11.0: Technical Highlights & Optimizations
Nov 12, 2025 · 2 minEngine Architecture: Full Migration to V1 V0 engine has been fully removed. This clears away LLMEngine, AsyncLLMEngine, and all old attention backends. V1 is now the sole, unified…
DeepSeek-V2: MLA Explained
Sep 24, 2025 · 2 minMLA is a novel attention mechanism designed for Transformer-based Large Language Models, aiming to boost inference efficiency by massively compressing how Key-Value (KV) cache is s…