airhaohan 的博客

This article explores the use of KVCache in LLM inference acceleration. By caching and incrementally updating the Key (K) and Value (V) matrices, we can significantly reduce computation during inference. The concept of "prefix invariance" ensures that most