Hugging Peers

DeepSeek-V4-Flash-0731-bf16

☆ Save
by PrimeIntellect text-generation transformers mit
9.8K downloads 0 likes on Hugging Face updated

Model card

DeepSeek-V4-Flash-0731-bf16

A bfloat16 re-export of deepseek-ai/DeepSeek-V4-Flash-0731. The weights are the published ones, dequantized: the dense linears from fp8 (e4m3 with ue8m0 scales over 128x128 blocks) and the routed experts from MXFP4, both to plain bfloat16. 43 layers, 114 safetensors shards.

Why this exists

The upstream config.json carries quantization_config and expert_dtype = "fp4". An inference engine reading those fields allocates fp8 linear parameters and MXFP4 expert parameters, which bfloat16 weights cannot fill. The routed experts at least fail loudly, in vLLM's _load_w13, where the packed MXFP4 shape does not match. The fp8 linears fail silently: a bfloat16 tensor has the same shape as the fp8 parameter it is loaded into, so copy_ casts it elementwise and no dequantization scale is ever applied. Serving bfloat16 weights therefore requires a checkpoint whose config carries neither field, which is what this repo is.

Differences from upstream

  • quantization_config and expert_dtype removed from config.json.
  • num_nextn_predict_layers, dspark_block_size, dspark_markov_rank, dspark_noise_token_id and dspark_target_layer_ids removed from config.json.
  • No mtp.* weights (upstream ships 4,705 of them; this export has none), so speculative decoding, including DSpark, is not available here. Do not pass --speculative-config or --speculative-algorithm DSPARK.
  • Not everything is bfloat16: the mHC, attention-sink and bias tensors are kept in fp32, and the router hash table in int64. The export holds 33,814 bf16 tensors, 406 fp32 tensors and 3 int64 tensors.

Prompting

There is no Jinja chat template, by upstream design. Use the bundled encoding/ directory, which is copied verbatim from upstream and is the reference implementation for turning OpenAI-format messages into an input string and for parsing completions back.

prime-rl users: MODEL_RENDERER_MAP is keyed on the upstream repo id (deepseek-ai/DeepSeek-V4-Flash-0731), so this repo id does not resolve automatically and falls back to the default renderer. Set the renderer explicitly:

[orchestrator.renderer]
name = "deepseek-v4"

Provenance

Exported from the conversion cache that prime-rl's trainer writes the first time it loads the published checkpoint, which is where the fp8 and MXFP4 weights are dequantized to bfloat16. The metadata.total_size in model.safetensors.index.json is byte-identical to that cache's (568,753,728,604 bytes), so the cache-to-HF round trip drops nothing.

License

MIT, inherited from upstream. The upstream LICENSE is included verbatim.