Alibaba’s Qwen team has released Qwen4-Max, the first model of its Qwen4 generation. Qwen4-Max is a proprietary sparse MoE model with a 2.4T-parameter backbone and 71B parameters active per token. It accepts text, images and video, and supports a 1M-token context window. Alibaba also cut API prices and reports a 62% drop in average cost per task.
TL;DR
- Size: 2.4T backbone parameters, 71B active per token, plus 328B N-gram Embedding parameters. 1M context, 393K max output tokens.
- Runs on: Hosted only: QwenCloud API, Token Plan, Qwen Studio, QwenWork and Qoder. No open weights, so no self-hosting or quantized builds.
- Performance: A large jump over Qwen3.8-Max. Competitive with frontier models on agentic work, behind on hard coding and vision.
- Best: HLAB-Vals-1.1 (legal agent work): 21.3, versus 11.7 for Kimi K3.1, the next-best model.
What is Qwen4-Max?
Qwen4-Max is a multimodal MoE language model from Alibaba, built for long-horizon professional work. A Token Plan-only preview opened this month only.
How does the Qwen4-Max architecture work?
Qwen4 upgrades the sparse MoE design in 3 places: attention, residual connections and embeddings.
- GDN + Qwen Sparse Attention (QSA): Each QSA layer pairs with 3 Gated DeltaNet layers. These compress history into fixed-size states. QSA then uses a lightweight indexer to pick relevant context at micro-block granularity. This lowers attention and indexing cost on long sequences.
- Gated Residual (GR): 4 parallel residual branches use dynamic read and write gates. The aim is stronger information flow and better training stability.
- N-gram Embedding: 328B extra parameters provide lookups of local token patterns at layer 2. This adds capacity with little extra compute.
Alibaba reports no loss spikes across the 2.4T-scale pre-training run. On in-house tests, Qwen4-Max-Base shows 5 to 13% lower bits per byte than Qwen3.8-Max-Base. It does this while activating 25% fewer parameters.
How does Qwen4-Max perform on benchmarks?
Alibaba’s comparison chart covers 16 benchmarks. Rivals are Qwen3.8-Max-0902, Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra and Kimi K3.1. Several scores are cited from third-party leaderboards. None were independently reproduced at launch.
Qwen4-Max ranks first on 4 of 16 benchmarks:
- HLAB-Vals-1.1 (legal agents): 21.3, versus 11.7 for Kimi K3.1 and 9.2 for Opus 5.5.
- Agents’ Last Exam v1.1: 46.7, versus 43.4 for Opus 5.5 and Kimi K3.1.
- BioMysteryBench: 85.0, versus 84.4 for GPT-6.1 Sol.
- DeepSWE v1.1: 75.7, versus 75.5 for Kimi K3.1.
Claude Opus 5.5 leads 7 of the 16 charts. Qwen4-Max trails most clearly on coding and vision:
- ProgramBench: 31.5 versus 65.0 for Opus 5.5.
- Terminal-Bench 4.0: 48.5 versus 67.2 for Kimi K3.1.
- Chartography: 54.2 versus 71.1 for GPT-6 Astra.
- Humanity’s Last Exam: 51.7 versus 59.8 for Opus 5.5.
On Alibaba’s own Work360 benchmark, Qwen4-Max scores 52.69, up from 47.74 for Qwen3.8-Max. Work360 spans 26 top-level domains and 721 specialized fields.
