IST Austria Distributed Algorithms and Systems Lab

university

https://ist.ac.at/en/research/alistarh-group/

https://github.com/IST-DASLab

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

kwanhee updated a model about 20 hours ago

ISTA-DASLab/Qwen3.5-397B-A17B-P48NVFP4-MoESQ

kwanhee updated a model about 20 hours ago

ISTA-DASLab/Qwen3-30B-A3B-P48NVFP4-MoESQ

kwanhee updated a model about 20 hours ago

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

View all activity

Papers

WUSH-KV: KV Cache Quantization with Data-Adaptive Transforms

Disaggregated Quantization: Specializing LLM Prefill and Decode

View all Papers

ISTA-DASLab 's collections 14

MoESQ

Paired-4:8 + NVFP4 W4A4 expert compressed MoE models for NVIDIA Blackwell Sparse Tensor Cores.

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

Text Generation • 646B • Updated about 20 hours ago • 192
ISTA-DASLab/Qwen3-30B-A3B-P48NVFP4-MoESQ

Text Generation • 20B • Updated about 20 hours ago • 275 • 2
ISTA-DASLab/Qwen3.5-397B-A17B-P48NVFP4-MoESQ

Image-Text-to-Text • 258B • Updated about 20 hours ago • 46
ISTA-DASLab/Qwen3.8-Flash-Next-P48NVFP4-MoESQ

Text Generation • 135B • Updated about 20 hours ago • 304 • 6

GSQ

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling, https://huggingface.co/papers/2604.18556

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Paper • 2604.18556 • Published Apr 20 • 20
ISTA-DASLab/Kimi-K2.6-2Bit-GSQ

Image-Text-to-Text • 84B • Updated Jun 17 • 242 • 6
ISTA-DASLab/Kimi-K2.5-2Bit-GSQ

Image-Text-to-Text • 84B • Updated Jun 17 • 474 • 1
ISTA-DASLab/Llama-3.1-70B-Instruct-2Bit-GSQ

Text Generation • 7B • Updated May 18 • 117 • 2

Optimizers

DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Paper • 2602.02016 • Published Feb 2 • 13

MatGPTQ

MatGPTQ quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MatGPTQ

8B • Updated 5 days ago • 104
ISTA-DASLab/Qwen3-8B-MatGPTQ

8B • Updated 5 days ago • 33
MatGPTQ: Accurate and Efficient Post-Training Matryoshka Quantization

Paper • 2602.03537 • Published Feb 3 • 5

GGUF

DASLab support for GGUF

ISTA-DASLab/Llama-3.1-8B-Instruct-GGUF

8B • Updated Sep 11, 2025 • 524
ISTA-DASLab/Llama-3.2-1B-Instruct-GGUF

1B • Updated Sep 11, 2025 • 486

QuEST

https://arxiv.org/abs/2502.05003

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations

Paper • 2502.05003 • Published Feb 7, 2025 • 44
ISTA-DASLab/QuEST-800M-INT1

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-sparse-INT4

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-INT4

Text Generation • Updated Feb 19, 2025

AQLM+PV

Official AQLM quantizations for "PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression": https://arxiv.org/abs/2405.14852

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Paper • 2405.14852 • Published May 23, 2024 • 2
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Text Generation • 11B • Updated Sep 17, 2024 • 53 • 45
ISTA-DASLab/Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf

3B • Updated Aug 16, 2024 • 11 • 3
ISTA-DASLab/Meta-Llama-3.1-8B-Instruct-AQLM-PV-2Bit-1x16-hf

Text Generation • 2B • Updated Aug 28, 2024 • 36 • 8

GSQ-RCO-GGUF

Non-uniform GGUF quantizations via GSQ + RCO: per-tensor mixed precision in standard GGUF form

ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Image-Text-to-Text • 27B • Updated Sep 2 • 1.64M • 1.96k
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Image-Text-to-Text • 177B • Updated 6 days ago • 1.89M • 556
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Image-Text-to-Text • 117B • Updated 6 days ago • 351k • 260

RCO

11:04 AM RCO: Model Compression with Exact Budget Constraints via Riemannian Manifolds, https://huggingface.co/papers/2605.00649

ISTA-DASLab/Qwen3-Coder-Next-RCO-pruned

Updated May 8 • 2

FP-Quant QAT

High-quality QAT FP4 models to use with the fp_quant vLLM/Transformers integration on Blackwell NVIDIA GPUs. See https://arxiv.org/abs/2509.23202

ISTA-DASLab/Llama-3.2-1B-Instruct-FPQuant-QAT-NVFP4

0.8B • Updated Oct 27, 2025 • 22
ISTA-DASLab/Llama-3.2-1B-Instruct-FPQuant-QAT-MXFP4

0.8B • Updated Oct 27, 2025 • 25
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4

2B • Updated Oct 27, 2025 • 53 • 1
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-MXFP4

2B • Updated Oct 27, 2025 • 16

MR-GPTQ

MXFP4 and NVFP4 quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-nvfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 27
ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-mxfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 27

Gemma3-GPTQ

GPTQ-quantized Gemma3 models

ISTA-DASLab/gemma-3-4b-it-GPTQ-4b-128g

Image-Text-to-Text • 4B • Updated Apr 13, 2025 • 309 • 7
ISTA-DASLab/gemma-3-12b-it-GPTQ-4b-128g

Image-Text-to-Text • 12B • Updated Apr 13, 2025 • 30.3k • 7
ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g

Image-Text-to-Text • 28B • Updated Mar 20, 2025 • 712k • 45

HIGGS

Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Paper • 2411.17525 • Published Nov 26, 2024 • 6
ISTA-DASLab/Llama-3.3-70B-Instruct-HIGGS-GPTQ-4bit

19B • Updated Dec 12, 2024 • 21 • 7
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 28
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-3bit

Text Generation • 2B • Updated Dec 10, 2024 • 25

AQLM

AQLM quantized LLMs

Extreme Compression of Large Language Models via Additive Quantization

Paper • 2401.06118 • Published Jan 11, 2024 • 15
ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 13, 2024 • 79 • 20
ISTA-DASLab/Meta-Llama-3-70B-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 4, 2024 • 93 • 14
ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Text Generation • 2B • Updated Nov 8, 2024 • 77 • 12

MoESQ

Paired-4:8 + NVFP4 W4A4 expert compressed MoE models for NVIDIA Blackwell Sparse Tensor Cores.

ISTA-DASLab/Kimi-K2.5-P48NVFP4-MoESQ

Text Generation • 646B • Updated about 20 hours ago • 192
ISTA-DASLab/Qwen3-30B-A3B-P48NVFP4-MoESQ

Text Generation • 20B • Updated about 20 hours ago • 275 • 2
ISTA-DASLab/Qwen3.5-397B-A17B-P48NVFP4-MoESQ

Image-Text-to-Text • 258B • Updated about 20 hours ago • 46
ISTA-DASLab/Qwen3.8-Flash-Next-P48NVFP4-MoESQ

Text Generation • 135B • Updated about 20 hours ago • 304 • 6

GSQ-RCO-GGUF

Non-uniform GGUF quantizations via GSQ + RCO: per-tensor mixed precision in standard GGUF form

ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Image-Text-to-Text • 27B • Updated Sep 2 • 1.64M • 1.96k
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Image-Text-to-Text • 177B • Updated 6 days ago • 1.89M • 556
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF

Image-Text-to-Text • 117B • Updated 6 days ago • 351k • 260

GSQ

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling, https://huggingface.co/papers/2604.18556

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

Paper • 2604.18556 • Published Apr 20 • 20
ISTA-DASLab/Kimi-K2.6-2Bit-GSQ

Image-Text-to-Text • 84B • Updated Jun 17 • 242 • 6
ISTA-DASLab/Kimi-K2.5-2Bit-GSQ

Image-Text-to-Text • 84B • Updated Jun 17 • 474 • 1
ISTA-DASLab/Llama-3.1-70B-Instruct-2Bit-GSQ

Text Generation • 7B • Updated May 18 • 117 • 2

RCO

11:04 AM RCO: Model Compression with Exact Budget Constraints via Riemannian Manifolds, https://huggingface.co/papers/2605.00649

ISTA-DASLab/Qwen3-Coder-Next-RCO-pruned

Updated May 8 • 2

Optimizers

DASH: Faster Shampoo via Batched Block Preconditioning and Efficient Inverse-Root Solvers

Paper • 2602.02016 • Published Feb 2 • 13

FP-Quant QAT

High-quality QAT FP4 models to use with the fp_quant vLLM/Transformers integration on Blackwell NVIDIA GPUs. See https://arxiv.org/abs/2509.23202

ISTA-DASLab/Llama-3.2-1B-Instruct-FPQuant-QAT-NVFP4

0.8B • Updated Oct 27, 2025 • 22
ISTA-DASLab/Llama-3.2-1B-Instruct-FPQuant-QAT-MXFP4

0.8B • Updated Oct 27, 2025 • 25
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-NVFP4

2B • Updated Oct 27, 2025 • 53 • 1
ISTA-DASLab/Llama-3.2-3B-Instruct-FPQuant-QAT-MXFP4

2B • Updated Oct 27, 2025 • 16

MatGPTQ

MatGPTQ quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MatGPTQ

8B • Updated 5 days ago • 104
ISTA-DASLab/Qwen3-8B-MatGPTQ

8B • Updated 5 days ago • 33
MatGPTQ: Accurate and Efficient Post-Training Matryoshka Quantization

Paper • 2602.03537 • Published Feb 3 • 5

MR-GPTQ

MXFP4 and NVFP4 quantized models

ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-nvfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 27
ISTA-DASLab/Llama-3.1-8B-Instruct-MR-GPTQ-mxfp

Image-Text-to-Text • 5B • Updated Oct 3, 2025 • 27

GGUF

DASLab support for GGUF

ISTA-DASLab/Llama-3.1-8B-Instruct-GGUF

8B • Updated Sep 11, 2025 • 524
ISTA-DASLab/Llama-3.2-1B-Instruct-GGUF

1B • Updated Sep 11, 2025 • 486

Gemma3-GPTQ

GPTQ-quantized Gemma3 models

ISTA-DASLab/gemma-3-4b-it-GPTQ-4b-128g

Image-Text-to-Text • 4B • Updated Apr 13, 2025 • 309 • 7
ISTA-DASLab/gemma-3-12b-it-GPTQ-4b-128g

Image-Text-to-Text • 12B • Updated Apr 13, 2025 • 30.3k • 7
ISTA-DASLab/gemma-3-27b-it-GPTQ-4b-128g

Image-Text-to-Text • 28B • Updated Mar 20, 2025 • 712k • 45

QuEST

https://arxiv.org/abs/2502.05003

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations

Paper • 2502.05003 • Published Feb 7, 2025 • 44
ISTA-DASLab/QuEST-800M-INT1

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-sparse-INT4

Updated Feb 18, 2025
ISTA-DASLab/QuEST-800M-INT4

Text Generation • Updated Feb 19, 2025

HIGGS

Models prequantized with [HIGGS](https://arxiv.org/abs/2411.17525) zero-shot quantization. Requires the latest `transformers` to run.

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Paper • 2411.17525 • Published Nov 26, 2024 • 6
ISTA-DASLab/Llama-3.3-70B-Instruct-HIGGS-GPTQ-4bit

19B • Updated Dec 12, 2024 • 21 • 7
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-4bit

Text Generation • 3B • Updated Dec 10, 2024 • 28
ISTA-DASLab/Llama-3.1-8B-Instruct-HIGGS-GPTQ-3bit

Text Generation • 2B • Updated Dec 10, 2024 • 25

AQLM+PV

Official AQLM quantizations for "PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression": https://arxiv.org/abs/2405.14852

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Paper • 2405.14852 • Published May 23, 2024 • 2
ISTA-DASLab/Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16

Text Generation • 11B • Updated Sep 17, 2024 • 53 • 45
ISTA-DASLab/Mistral-Nemo-Instruct-2407-AQLM-PV-2Bit-1x16-hf

3B • Updated Aug 16, 2024 • 11 • 3
ISTA-DASLab/Meta-Llama-3.1-8B-Instruct-AQLM-PV-2Bit-1x16-hf

Text Generation • 2B • Updated Aug 28, 2024 • 36 • 8

AQLM

AQLM quantized LLMs

Extreme Compression of Large Language Models via Additive Quantization

Paper • 2401.06118 • Published Jan 11, 2024 • 15
ISTA-DASLab/Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 13, 2024 • 79 • 20
ISTA-DASLab/Meta-Llama-3-70B-AQLM-2Bit-1x16

Text Generation • 11B • Updated May 4, 2024 • 93 • 14
ISTA-DASLab/Meta-Llama-3-8B-Instruct-AQLM-2Bit-1x16

Text Generation • 2B • Updated Nov 8, 2024 • 77 • 12

AI & ML interests

Recent Activity

Papers

Team members 19

ISTA-DASLab 's collections 14