KV cache compression for LLM inference: 37× smaller KV cache at full-cache accuracy. Per-token precision on transform coding, question-time refinement, and attention over the compressed cache. PyTorch + Hugging Face.
deep-learning gpu transformers pca attention llama quantization memory-efficient modelcompression inference-optimization huggingface kv-cache llm long-context llm-inference memory-efficient-compression kv-cache-compression
-
Updated
Oct 26, 2026 - Python