MiMo-V2-Flash sliding window and global attention mixing mechanism

MiMo-V2-Flash Open Source Interpretation: 309B MoE, 15B Activation Parameters and 256K Long Context

1. Abstract MiMo-V2-Flash is an open-sourced hybrid expert (MoE) large language model from the Xiaomi MiMo team, with a total parameter of about 309B ...