Found 4 related articles
Mixture of Experts (MoE) is a model architecture that "doesn't put the whole model together every time". Its most important feature is that some layer...
Diffusion LLM can be understood as transferring some core ideas of the "diffusion model" to the language model, generating text in a gradual denoising...
Sparse attention can be simply understood as: instead of having each token look at all tokens, selectively look at only a part of them. This term come...
A Transformer is a neural network architecture. It's important not because of the name, but because it does a good job of "parallel processing" and "c...