metadata
license: llama2
tags:
- code
This is a quantized version of WizardLM/WizardCoder-Python-13B-V1.0, quantized using ctranslate2 (see inference instructions there).
The license/caveats/intended usage is the same as the original model.
The quality of its output may have
been negatively affected by the quantization process.
The command run to quantize the model was:
ct2-transformers-converter --model ./models-hf/WizardLM/WizardCoder-Python-13B-V1.0 --quantization int8_float16 --output_dir ./models-ct/WizardLM/WizardCoder-Python-13B-V1.0-ct2-int8_float16
The quantization was run on a 'high-mem', CPU only (8 core, 51GB) colab instance and took approximately 10 minutes.