Lossless, But Not Free: The Lossless, But Not Free — When Speculative Decoding Actually Pays Off (and When It Doesn't)

One of the hottest topics in LLM inference acceleration right now is Speculative Decoding. DSpark claims 60%–85% single-user speedup at the same throughput. Google has published a stream of research on it — SpecTr, block verification, SpecRouter, and more. Sounds great, right? A small model (draft m