SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference
Paper
•
2502.18137
•
Published
•
58
Low-rank attention
-
-
XAttention: Block Sparse Attention with Antidiagonal Scoring
Paper • 2503.16428 • Published • 15 -
On the Benefits of Rank in Attention Layers
Paper • 2407.16153 • Published -
Towards Understanding the Nature of Attention with Low-Rank Sparse Decomposition
Paper • 2504.20938 • Published -
Loki: Low-Rank Keys for Efficient Sparse Attention
Paper • 2406.02542 • Published -
Learning to Compress: Local Rank and Information Compression in Deep Neural Networks
Paper • 2410.07687 • Published • 1 -
LoRA Learns Less and Forgets Less
Paper • 2405.09673 • Published • 89