| --- |
| license: cc-by-nc-nd-4.0 |
| datasets: |
| - ajibawa-2023/Python-Code-23k-ShareGPT |
| language: |
| - en |
| tags: |
| - code |
| quantized_by: bartowski |
| --- |
| |
| ## Exllama v2 Quantizations of Python-Code-13B |
|
|
| Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.8">turboderp's ExLlamaV2 v0.0.8</a> for quantization. |
|
|
| Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions. |
|
|
| Conversion was done using Evol-Instruct-Code-80k-v1.parquet as calibration dataset. |
|
|
| Original model: https://huggingface.co/ajibawa-2023/Python-Code-13B |
| |
| <a href="https://huggingface.co/bartowski/Python-Code-13B-exl2/tree/3.75">3.75 bits per weight</a> |
| |
| <a href="https://huggingface.co/bartowski/Python-Code-13B-exl2/tree/4.0">4.0 bits per weight</a> |
| |
| <a href="https://huggingface.co/bartowski/Python-Code-13B-exl2/tree/6.0">6.0 bits per weight</a> |
| |
| <a href="https://huggingface.co/bartowski/Python-Code-13B-exl2/tree/8.0">8.0 bits per weight</a> |
|
|
| ## Download instructions |
|
|
| With git: |
|
|
| ```shell |
| git clone --single-branch --branch 4.0 https://huggingface.co/bartowski/Python-Code-13B-exl2 |
| ``` |
|
|
| With huggingface hub (credit to TheBloke for instructions): |
|
|
| ```shell |
| pip3 install huggingface-hub |
| ``` |
|
|
| To download the `main` (only useful if you only care about measurement.json) branch to a folder called `Python-Code-13B-exl2`: |
|
|
| ```shell |
| mkdir Python-Code-13B-exl2 |
| huggingface-cli download bartowski/Python-Code-13B-exl2 --local-dir Python-Code-13B-exl2 --local-dir-use-symlinks False |
| ``` |
|
|
| To download from a different branch, add the `--revision` parameter: |
|
|
| ```shell |
| mkdir Python-Code-13B-exl2 |
| huggingface-cli download bartowski/Python-Code-13B-exl2 --revision 4.0 --local-dir Python-Code-13B-exl2 --local-dir-use-symlinks False |
| ``` |
|
|
