Loading...
Loading...
Efficiency-optimized 284B MoE (13B active) with hybrid attention, 1M context, and high/xhigh reasoning, built for fast, high-throughput coding and agent workflows
Take DeepSeek V4 Flash 0423 (preview) apart, layer by layer. The 3D teardown below shows its 43 transformer layers (0 dense + 43 MoE), MLA attention, and expert routing, with values from the public model config.
DeepSeek is a Chinese AI lab founded in July 2023 in Hangzhou by Liang Wenfeng. It was funded solely by High-Flyer, the quantitative hedge fund Liang co-founded, until a $7 billion Series A in May 2026, and grew out of GPU capacity originally stockpiled for trading research. The lab releases its V, R and V4 model lines as open weights under the MIT licence, with the V4 Pro and Flash tiers succeeded by V4.1 Flash in September 2026.
more about DeepSeek →4096-dim vectors over a 129,280-token vocabulary, same as V4 Pro.
18 fewer layers than Pro, with mHC residuals. The first 3 layers use static Hash-MoE routing; the rest use learned MoE routing.
Same Compressed Sparse + Heavily Compressed hybrid as Pro, with a 128-token sliding window and 64 heads. The sparse indexer (64 heads, 512 top-k) keeps the 1M context cheap.
256 routed experts, 6 active per token, 1 shared. Flash activates only 13B of 284B parameters per token, roughly a quarter of Pro's active budget.
Per-head RMSNorm then projection to 129,280 logits. Flash is the cost-optimised V4 tier for high-volume inference where Pro's 49B active is more than needed.