mAPEX Quantized Models

Source Data

The source data for quantization was taken from deepgrove/maple-preview.

Original source file: model-00001-of-00009.safetensors

Quantization Method

All quants were produced using the modified APEX quantization scheme.

mAPEX (modified Automated Precision EXpert allocation) assigns per-layer, per-tensor precision for MoE models using llama.cpp's --tensor-type-file.

For more information, see: https://github.com/DrMoriarty/apex-quant/

Quantized Models

Note: TierN-s quants are typically slightly larger than their regular counterparts, but provide roughly 10–20% faster inference (highly dependent on the model and GPU).