Alibaba Releases Qwen4-Max: 2.4T MoE, 71B Active, 1M Context





Alibaba’s Qwen team has released Qwen4-Max, the first model of its Qwen4 generation. Qwen4-Max is a proprietary sparse MoE model with a 2.4T-parameter backbone and 71B parameters active per token. It accepts text, images and video, and supports a 1M-token context window. Alibaba also cut API prices and reports a 62% drop in average cost per task.

TL;DR

  • Size: 2.4T backbone parameters, 71B active per token, plus 328B N-gram Embedding parameters. 1M context, 393K max output tokens.
  • Runs on: Hosted only: QwenCloud API, Token Plan, Qwen Studio, QwenWork and Qoder. No open weights, so no self-hosting or quantized builds.
  • Performance: A large jump over Qwen3.8-Max. Competitive with frontier models on agentic work, behind on hard coding and vision.
  • Best: HLAB-Vals-1.1 (legal agent work): 21.3, versus 11.7 for Kimi K3.1, the next-best model.

What is Qwen4-Max?

Qwen4-Max is a multimodal MoE language model from Alibaba, built for long-horizon professional work. A Token Plan-only preview opened this month only.

How does the Qwen4-Max architecture work?

Qwen4 upgrades the sparse MoE design in 3 places: attention, residual connections and embeddings.

  • GDN + Qwen Sparse Attention (QSA): Each QSA layer pairs with 3 Gated DeltaNet layers. These compress history into fixed-size states. QSA then uses a lightweight indexer to pick relevant context at micro-block granularity. This lowers attention and indexing cost on long sequences.
  • Gated Residual (GR): 4 parallel residual branches use dynamic read and write gates. The aim is stronger information flow and better training stability.
  • N-gram Embedding: 328B extra parameters provide lookups of local token patterns at layer 2. This adds capacity with little extra compute.

Alibaba reports no loss spikes across the 2.4T-scale pre-training run. On in-house tests, Qwen4-Max-Base shows 5 to 13% lower bits per byte than Qwen3.8-Max-Base. It does this while activating 25% fewer parameters.

How does Qwen4-Max perform on benchmarks?

Alibaba’s comparison chart covers 16 benchmarks. Rivals are Qwen3.8-Max-0902, Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra and Kimi K3.1. Several scores are cited from third-party leaderboards. None were independently reproduced at launch.

Qwen4-Max ranks first on 4 of 16 benchmarks:

  • HLAB-Vals-1.1 (legal agents): 21.3, versus 11.7 for Kimi K3.1 and 9.2 for Opus 5.5.
  • Agents’ Last Exam v1.1: 46.7, versus 43.4 for Opus 5.5 and Kimi K3.1.
  • BioMysteryBench: 85.0, versus 84.4 for GPT-6.1 Sol.
  • DeepSWE v1.1: 75.7, versus 75.5 for Kimi K3.1.

Claude Opus 5.5 leads 7 of the 16 charts. Qwen4-Max trails most clearly on coding and vision:

  • ProgramBench: 31.5 versus 65.0 for Opus 5.5.
  • Terminal-Bench 4.0: 48.5 versus 67.2 for Kimi K3.1.
  • Chartography: 54.2 versus 71.1 for GPT-6 Astra.
  • Humanity’s Last Exam: 51.7 versus 59.8 for Opus 5.5.

On Alibaba’s own Work360 benchmark, Qwen4-Max scores 52.69, up from 47.74 for Qwen3.8-Max. Work360 spans 26 top-level domains and 721 specialized fields.

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *