What is Sparse Attention? Why long context and inference cost issues always talk about it
Sparse attention can be simply understood as: instead of having each token look at all tokens, selectively look at only a part of them. This term come...
AI Encyclopedia • Admin •
83
