LongCat-Flash-Lite Explained: How N-gram Embedding Rewrites the Efficiency Curve of Sparse MoE

LongCat-Flash-Lite Interpretation: A New Efficiency Path for Sparse MoE with N-gram Embeddings

1. Abstract LongCat-Flash-Lite is an open-source large model targeting high-sparsity MoE scenarios: the total parameters are 68.5B, but only about 2.9...