DeepSeek-V4.1-Flash-1017 — Rapid-MLX mixed-precision (256 GB Macs)
A mixed-precision MLX build of deepseek-ai/DeepSeek-V4.1-Flash-1017 sized to run on a single 256 GB Apple Silicon Mac with Rapid-MLX.
- No experts removed. All 40 × 384 routed experts are kept.
- Bits go where they were measured to matter. The routed experts hold 96% of the parameters. They get 2/3/4-bit widths per layer and per projection. Every other weight stays at high precision.
- ~206.5 GB resident (weights the model keeps in RAM), plus 122.9 GB of Engram tables that stay on SSD. The download is 329 GB in total.
Quick start
The
deepseek-v41-flash-1017alias ships in the next Rapid-MLX release. Until then this repo is weights-only.
pip install -U rapid-mlx
rapid-mlx serve deepseek-v41-flash-1017
This starts an OpenAI-compatible server on http://127.0.0.1:8000/v1. The
first run downloads the pinned revision and verifies its file hashes.
Requirements and current limits
| Item | Value |
|---|---|
| Unified memory | 256 GB Mac (the server refuses to start below the guard) |
| Free disk | ~330 GB (Engram tables are read from SSD on demand) |
| Decoding | greedy only (temperature 0); seed is rejected |
| Input | text only, ≤ 8,192 prompt tokens |
| Tools / function calling | not yet supported by this serving lane |
| Speculative decoding | built-in DSpark draft (dspark/, 4-bit experts) |
Sampling, tool calling and longer inputs are planned follow-ups for the serving lane. They are not limits of the weights.
What is in this build
| Component | Precision |
|---|---|
| Routed experts, layers 0–14, 16–19, 32–34 | gate/up/down 2/2/2-bit |
| Routed experts, layer 15 | 3/3/3-bit |
| Routed experts, layers 20–31 | 2/2/3-bit (down_proj 3-bit) |
| Routed experts, layers 35–38 | 3/3/3-bit |
| Routed experts, layer 39 | 4/4/3-bit |
| Attention, shared experts, indexer/compressor | 8-bit |
| Embeddings, LM head, router | BF16 |
| Engram (layers 1, 14) | 4-bit, group 32, offloaded to SSD |
| DSpark draft head | 8-bit dense, 4-bit experts |
How the widths were chosen
- Reference. The source FP4/FP8 checkpoint was streamed one layer at a time to produce teacher logits on 38 held-out text slices. These cover chat, code, math, tool-call transcripts, agentic terminal sessions, law, medicine, finance, science, and 20 languages including low-resource ones. A 2-bit teacher was not used.
- Pruning was evaluated and rejected for this size. Contrastive hot/cold expert scoring on document-disjoint calibration halves gave a very stable "important" set (Jaccard 0.98), but unstable "unimportant" candidates (Jaccard 0.26–0.35). Removing experts on that evidence was not safe, so none are removed.
- Measured layer sensitivity. Mixed-bit variants were built and scored by
token-level KL against the reference. The late layers (35–39) and the
middle-to-late band (20–34) dominate quality. Layers 0–14 barely matter.
Within the middle band,
down_projalone recovers most of the gain at a third of the memory.
Quality
Teacher-forced next-token KL divergence vs. the source-precision reference. There are 38 held-out slices of 512 tokens each; lower is better.
| Build | Resident | Mean KL (nats) | Worst slice | Mean top-1 flip |
|---|---|---|---|---|
| Uniform 2-bit experts | 180 GB | 2.02 | 5.93 | 34% |
| This build | 206.5 GB | 0.439 | 0.749 | 21% |
Per-domain KL for this build:
| Domain | KL |
|---|---|
| code | 0.128 |
| agentic terminal | 0.156 |
| math | 0.179 |
| tool-call transcripts | 0.187 |
| general chat (EN) | 0.251 |
| general chat (ZH) | 0.246 |
| hardest slice (Chinese folk-customs text) | 0.749 |
Non-English and long-tail knowledge text drifts more than code, math and agentic text.
Task benchmarks
Greedy, non-thinking mode, fixed sample IDs, through the Rapid-MLX server.
| Benchmark | Items | Score | Notes |
|---|---|---|---|
| GSM8K (test) | 250 | 90.8% | exact numeric match, zero-shot chat |
| HumanEval | 164 (all) | 89.6% pass@1 | executed in a no-network sandbox |
| MMLU-Pro | 300 (stratified) | 66.7% | zero-shot letter answer; 26 items hit the 512-token cap |
These are small, fixed subsets with a 512-token generation cap and no thinking mode. They are not leaderboard protocols, so compare them across builds, not against published numbers.
Speed and memory (Mac Studio M3 Ultra, 256 GB)
| Prompt | Decode (tok/s) | Prefill (tok/s) |
|---|---|---|
| 1k tokens | 19.9 | ~167 |
| 8k tokens | 18.8 | ~170 |
Greedy decoding with the built-in DSpark draft. Prefill is a client-side estimate.
Memory. The weights are loaded into RAM up front: ~206.5 GB resident, plus working buffers. The measured peak process footprint is 228 GB (8k-token prompt, otherwise idle Mac Studio). On a 256 GB Mac, quit other memory-heavy apps before starting the server. If macOS runs out of memory and swap during loading, it terminates the server.
Reproducing
The build and evaluation tooling lives in the Rapid-MLX repository:
scripts/deepseek_v41_rd_build.py/scripts/repack_deepseek_v41_native.pyscripts/evaluate_deepseek_v41_rd.pyscripts/eval_deepseek_v41_tasks.py
The per-layer bit allocation and byte breakdown ship as rapid-mlx-allocation.json.
License
MIT, same as the base model. See LICENSE.