DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Technical Report 👁️

Introduction

We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively.

Architecture. DeepSeek-V4.1-Flash adopts a Causal Encoder-Decoder (CED) architecture: a 40-layer Transformer organized as a 20-layer causal encoder followed by a 20-layer decoder. With CED, the decoder's global KV cache is projected from the final encoder hidden states rather than derived from each decoder layer's own hidden states. This allows the model to activate only 8B parameters per token during prefill and 16B during decode, substantially improving cost efficiency for input-heavy agentic workloads. SWA Bounded Replay reconstructs missing SWA KV states by replaying only the most recent n_win tokens, avoiding the need to persist SWA KV to SSD and reducing the persistent KV cache footprint to roughly 1/8 of that of DeepSeek-V4-Flash.

Compressed Sparse Attention 2 (CSA2). DeepSeek-V4.1-Flash uses CSA2, which assigns each attention layer one of three static modes — Full, Reindex, or Reuse — to share main KV and indexer K across layers and reuse Top-K sparse-attention indices. In the decoder, a Hierarchical Sparse Indexer further restricts later indexing layers to a candidate pool constructed by the first Full Mode layer, bounding deeper indexer cost independently of context length. Combined with FP4 main KV caching (E2M1 format, one E4M3 scale per 16 channels), these designs reduce the global KV cache footprint to 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash.

Additional architectural components include Single-Pass mHC (revised residual-stream mixing with an efficient Mega-mHC kernel), Engram conditional memory (196B parameters, sparsely accessed via token-based lookup), and DSpark speculative decoding (semi-autoregressive draft generation with confidence-scheduled verification). The model uses 1 shared expert and 384 routed experts per MoE layer, activating 6 routed experts per token.

Multimodal architecture. A vision encoder (DeepSeek-ViT, trained from scratch with 2D-RoPE and 3×3 pixel-unshuffle downsampling) and a two-layer MLP projector convert images into visual embeddings, processed jointly with text embeddings from the start of language-model pre-training.

Pre-training. DeepSeek-V4.1-Flash is trained from scratch on a multimodal corpus comprising 45T tokens, with sparse attention trained at a sequence length of 64K and context extended to 1M tokens at 34T tokens.

Post-training. The post-training recipe follows the standard SFT → RL → on-policy distillation (OPD) paradigm without algorithmic modifications. All substantive changes lie instead in the data pipeline: large-scale automated synthesis of agent tasks and environments with progressive scaling of data, tasks, and rollouts. The model supports a continuously controllable reasoning effort setting (integer 1–100) that trades inference cost for accuracy.

Figure 1. (a) Performance of DeepSeek-V4.1-Flash and counterparts on agentic benchmarks. (b) Global KV cache size per token (bytes) across generations of DeepSeek models. DeepSeek-V4.1-Flash achieves approximately 4-fold and 437-fold reductions relative to DeepSeek-V4-Flash and DeepSeek-V1, respectively.

Evaluation Results

Base Model

All base models are evaluated in our internal framework under the same evaluation settings. Scores within 0.3 of each other are considered equivalent.

Instruct Model

DeepSeek-V4.1-Flash supports a continuously controllable reasoning effort from 1 to 100. All instruct results below use the maximum effort setting (reasoning_effort=100). Evaluations use temperature=1.0, top_p=0.95.

For code agent benchmarks (Terminal-Bench 2.1/3.0/4.0, DeepSWE v1.1, NL2Repo-Bench, ProgramBench), the model is evaluated with the Minimal mode of DeepSeek Harness and a 1M-token context window. To align with official setup requirements, the mini-SWE harness is used for DeepSWE v1.1, and the Claude Code harness for SEC-Bench Pro. Visual agent benchmarks (Chartography, BabyVision, ZeroBench) use the Claude Code harness with a 512k-token context window. Agent's Last Exam and AutomationBench use their official scaffolds. All agentic evaluations use temperature=1.0, top_p=0.95.

Comparison with frontier models (Max reasoning effort)

† Text-only subset of HLE.

Performance across agent scaffolds (DeepSWE v1.1 and Terminal-Bench 2.1, Max reasoning effort)

All scaffolds use N=8 samples per task on DeepSWE v1.1 and N=3 on Terminal-Bench 2.1, with Linux containers, temperature=1.0, top_p=0.95, a 1M-token context limit, and max_steps=500 per agent. Terminal-Bench 2.1 is evaluated without network access.

Prompt Encoding

This release does not include a Jinja-format chat template. The encoding folder contains a self-contained Python reference implementation (encoding.py) with test cases for multi-turn conversations, tool calling, thinking mode, numeric reasoning effort, mid-conversation system messages, and interleaved image content.

For production use, we additionally release deepseek-recipe, a set of Rust libraries with Python bindings that provides the same prompt format as a maintained, protocol-aware toolkit. It converts Messages, Chat Completions, and Responses API requests into the Conversation format, encodes them into DeepSeek V4 and V4.1 prompts or token IDs, and parses model output back into complete or streamed responses — covering thinking, tool calls, images, and generation settings. Model inference, tool execution, and HTTP transport are left to the caller.

Minimal Inference

Please refer to the inference folder for instructions on weight conversion and running inference locally.

Recommended sampling parameters:

Reproducing DeepSWE Benchmark Results

The evaluation folder contains step-by-step instructions for reproducing the DeepSWE v1.1 benchmark results, covering both the dsh-minimal agent and the official mini-swe-agent. The patch required to integrate dsh-minimal with Pier is also included there.

W8A8 INT8 Quantization

This directory is a W8A8 INT8 per-channel quantized derivative of DeepSeek-V4.1-Flash, All non-quantized files (tokenizer, inference/encoding code, docs) are inherited from the source checkpoint.

Quantization Method

• Path: FP8 (E4M3, 32x32 block, E8M0 scale) / FP4 (E2M1 packed, per-32 E8M0 scale) -> dequantize to BF16 (lossless) -> INT8 symmetric per-channel quantization.
• Per-tensor format (sglang w8a8_int8 compatible):
  • <layer>.weight: int8, shape (out, in), values clamped to [-127, 127], row-wise amax == 127.
  • <layer>.weight_scale: float32, shape (out, 1), = amax(|W|, dim=in) / 127.
• Activations: dynamic quantization (activation_scheme: dynamic), no calibration data used.

Quantized Weights (47,457 tensors)

Kept in BF16 (registered in quantization_config.ignore)

KV-cache-related and precision-sensitive projections stay in BF16 (dequantized from FP8 where applicable):

• attn.wq_a, attn.wkv, attn.wo_a (43 x 3 = 129 tensors)
• attn.compressor.wkv, attn.compressor.wgate (kv-source layers 2/8/14/20)
• attn.indexer.weights_proj, attn.indexer.wk
• engram.wkv
• vision.* and aligner.* (entire vision tower + projector, natively BF16)

Kept As-Is

• engram.embed.weight (+.scale): the two giant n-gram embedding tables (384M x 256, ~98 GB each) remain in their native FP8 E4M3 + E8M0 scale format. They are lookup tables, not GEMM linears; converting them to BF16 would add ~196 GB with no benefit.
• embed.weight, head.weight, markov_head.*, ffn.gate.* (router weight/bias/bias_vl), all *norm*, attn_sink, hc_*, image_*, engram.q_weight/k_weight.

config.json Changes

"quantization_config": {
  "quant_method": "w8a8_int8",
  "activation_scheme": "dynamic",
  "is_dynamic": true,
  "is_checkpoint_int8_serialized": true,
  "expert_dtype": "int8",
  "ignore": ["vision.", "aligner.", "attn.compressor.wkv", "attn.compressor.wgate",
             "attn.indexer.weights_proj", "attn.indexer.wk",
             "attn.wq_a", "attn.wkv", "attn.wo_a", "engram.wkv"]
}

Layout Audit (all passed)

Verified by static safetensors-header audit (verify_v41_int8_layout.py):

1. Shards 1..48 form a continuous closed set; key diff vs source: 0 missing / 0 extra.
2. All 47,457 INT8 weights are paired with F32 (out, 1) weight_scale.
3. All 346 ignore-matched linear weights are BF16.
4. engram.embed tensors are byte-identical FP8 copies of the source.
5. No un-quantized BF16 linear weights outside the ignore/blacklist rules.
6. dtype totals: I8 47,457 / F32 47,844 (47,457 new scales + 387 native) / BF16 650 / F8 4.

Notes

• Product size: 722 GiB (source: 476 GiB; FP4 experts expand 2x when unpacked to INT8, while the FP8 engram tables are copied unchanged).
• Serving requires an inference stack with deepseek_v41 model support and the w8a8_int8 quant method (e.g. sglang with the corresponding model class).

License

This repository and the model weights are licensed under the MIT License.

Citation

@misc{deepseekai2026deepseekv41flash,
      title={DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression},
      author={DeepSeek-AI},
      year={2026},
}

Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.