📢 Note: This repository was renamed from GLM-5-MXFP4-FP8 on 2026-08-21. The weights are unchanged; only the model name has been updated.

Model Overview

Model Architecture: GlmMoeDsaForCausalLM
- Input: Text
- Output: Text
Inference Engine: SGLang
Quantization:
- dense (layers 0-2): up_proj, gate_proj, down_proj
  - Weight quantization: FP8, Static, Per-Channel
  - Activation quantization: FP8, Dynamic, Per-Token
- moe: up_proj, gate_proj, down_proj
  - Weight quantization: OCP MXFP4, Static
  - Activation quantization: OCP MXFP4, Dynamic
- shared_experts: up_proj, gate_proj, down_proj
  - Weight quantization: FP8, Static, Per-Channel
  - Activation quantization: FP8, Dynamic, Per-Token
- self_attn: q_a_proj, q_b_proj, kv_a_proj_with_mqa, kv_b_proj, o_proj, wq_b, wk
  - Weight quantization: FP8, Static, Per-Channel
  - Activation quantization: FP8, Dynamic, Per-Token

This model was built with GLM-5 model by applying MXFP4 and FP8 quantization.

Deployment

Use with SGLang

This model can be deployed efficiently using the SGLang backend.

Evaluation

The model was evaluated on IFEval GSM8K C-EVAL benchmarks.

Accuracy

Benchmark	GLM-5	GLM-5-MXFP4-FP8 (this model)	Recovery
IFEval	92.42	92.8	100.41%
GSM8K	97.88	97.5	99.61%
C-Eval	92.57	92.2	99.6%

Reproduction

The IFEval GSM8K C-EVAL results were obtained using the EvalScope and SGLang framework.

Install T-head SGLang (Version: 0.5.10) and EvalScope (Version: 1.5.0) in container first.

Launching server

MODEL_PATH=<Path to GLM-5-MXFP4-FP8>
SGLANG_USE_MODELSCOPE=true python3 -m sglang.launch_server --model-path $MODEL_PATH \
  --tp-size 8 \
  --mem-fraction-static 0.85 \
  --disable-radix-cache \
  --trust-remote-code \
  --decode-attention-backend flashmla --prefill-attention-backend fa3 \
  --cuda-graph-max-bs 16 \
  --disable-shared-experts-fusion \
  --watchdog-timeout 6000 --dist-timeout 6000 \
  --served-model-name GLM-5

Evaluating model in a new terminal

evalscope eval \
  --model GLM-5 --api-url http://127.0.0.1:8000/v1  --api-key EMPTY \
  --eval-type openai_api --datasets gsm8k \
  --dataset-args '{"gsm8k": {"few_shot_num": 0, "filters": {"remove_until": "</think>"}}}' \
  --eval-batch-size 128 \
  --generation-config '{"max_tokens": 32768, "temperature": 0.6, "top_p": 0.95, "n": 1, "top_k": 20}' \
  --timeout 60000 --stream --work-dir outputs

GLM-5-MXFP4-FP8

Model card