Hy4-preview-SF-WFP4AFP8
Quant Diff
CrossingBits PTQ of Tencent Hunyuan Hy4-preview (hy_v4), from the official MXFP8 release
Tencent-Hunyuan/Hy4-preview-FP8 (dequantized layer by layer, then quantized).
Same layout as siliconflow/DeepSeek-V4.1-Flash-SF-WFP4AFP8: only routed experts are FP4.
Changed from the previous build (tag 20260910, Hopper layout): every non-expert FP8 linear moved from
block-128 FP8 with FP32 scales to MXFP8 (group 32, UE8M0 scales), and self_attn.linear_gate went from FP8 back to BF16.
Calibration KLD dropped from 0.281 to 0.078. FP4 routed experts are unchanged in format (FP4 E2M1 g32 + FP8 E4M3 g32 activations).
This build targets Blackwell serving; the shard layout changed (model-common-*-of-00009).
| Recipe | weight_fp4_activation_fp8, group 32, ue8m0 / e4m3 |
Routed experts (mlp.experts.*) |
FP4 E2M1 weights + FP8 E4M3 activations (g32) |
MLA q_a / kv_a / q_b / o, indexer wk / wq_b, dense layer-0 FFN, mlp.shared_experts.{gate,up,down}_proj |
MXFP8 W8A8 g32 |
| BF16 | router (mlp.gate), MLA absorb (kv_b / k_b / v_b), indexer.weights_proj, self_attn.linear_gate, MTP eh_proj, embed, lm_head |
| MTP | enabled (mtp_layers.0); shares the target embed_tokens / lm_head as in the provider checkpoint |
Full quantization metadata is in quantize_config.json.
Validation
| Check | Result |
|---|---|
Calibration KLD (quantize_config.json) |
0.078 (previous build 0.281) |
| Weight dequant vs source | MXFP8 layers identical to the MXFP8 source (cos 1.0000); FP4 experts cos ≥ 0.991 |
| Unquantized tensors | identical to source (BF16/FP32) |
| Long-form text decode (chat template, default reasoning) | coherent; exits think and answers (plans ~500 tokens before answering) |
| MTP top-1 / top-5 acceptance (crossingbits eval) | 0.602 / 0.746 |
Not yet validated:
- Serving engine. FP4 GEMMs need Blackwell; this checkpoint was validated only with crossingbits reference inference, not yet served on an engine.
The upstream model card (Chinese) is kept as README_CN.md; see LICENSE for the Apache-2.0 license.