Alibaba Tongyi Qianwen released Qwen3.8-Flash, placing low-cost inference and next-generation architecture preview in the same update. This model adopts a multimodal MoE approach, with the main model having 125B parameters, and each token activates only 6B; Its Next version introduces the Qwen4 architecture design ahead of time, targeting long-context efficiency and unit inference cost.
The 125B model only activates 6B each time
Qwen3.8-Flash-Next consists of a 125B parameter main model and an additional 51B N-gram Embedding. N-gram Embedding can look up tables based on local context, increasing model capacity while reducing additional computational load to a lower level.
Each token activates only 6B parameters, which is a key metric for this MoE model . Officially, its training cost is about one-ninth that of Qwen3.7-Plus, while improving capabilities in programming and office tasks, model competition has shifted from "larger scale" to "more efficient per call."
The Qwen4 architecture has been revealed early
Qwen3.8-Flash-Next is not the full Qwen4, but an early preview of the next-generation architecture. Attention uses a hybrid design of GDN and QSA: GDN compresses historical information, while QSA filters more important contexts to reduce long-sequence computation and KV cache access pressure.
The residual structure has also been upgraded to Gated Residual, with N-gram Embedding that can be offloaded to Host Memory. The model natively supports 262,144 token contexts and can be extended to 1M tokens via YaRN. Long context efficiency has become the core goal of this architectural adjustment.
Flash targets large-scale AI applications
The production version of Qwen3.8-Flash is designed for QwenCloud and supports 1M context by default. This product positioning is better suited for AI programming, document processing, agents, and high-frequency API calls, as these scenarios require long contexts and are highly sensitive to inference cost and throughput.
The Qwen team chose to publicly release the Next architecture before the full Qwen4 family, essentially giving the open-source ecosystem a period of adaptation. Inference frameworks, quantization tools, and deployment plans can be adjusted in advance around the new structure, reducing migration costs when the next-generation model is officially released.
Qwen 3.8-Flash clearly reveals a clear direction: the next round of large model competition will not focus solely on parameters and benchmarks. Whoever can handle longer contexts with fewer activation parameters and lower memory overhead will be more likely to enter truly high-frequency AI applications.
