GPTQModel. Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

github.com/liguodongiot/GPTQModel

Problem

Audience

Updates

No founder updates yet. Check back soon.