Hugging Peers

WizardCoder-Python-7B-V1.0-ct2-int8_float16

☆ Save
by bgsach transformers llama2
18 downloads 0 likes on Hugging Face updated

Model card

license: llama2
tags:

  • code

This is a int8_float16 quantized version of WizardLM/WizardCoder-Python-7B-V1.0, quantized using ctranslate2 (see inference instructions there).

The license/caveats/intended usage is the same as the original model.
The quality of its output may have been negatively affected by the quantization process.

The command run to quantize the model was:

ct2-transformers-converter --model ./models-hf/WizardLM/WizardCoder-Python-7B-V1.0 --quantization int8_float16 --output_dir ./models-ct/WizardLM/WizardCoder-Python-7B-V1.0-ct2-int8_float16

The quantization was run on a 'high-mem', CPU only (8 core, 51GB) colab instance and took approximately 10 minutes.