Download
AutoGPTQ is an implementation of GPTQ (Quantized GPT) that optimizes large language models (LLMs) for faster inference by reducing their computational footprint while maintaining accuracy.
Features
- Efficient quantization for large language models
- Reduces memory usage without major performance loss
- Supports various precision levels (e.g., 4-bit, 8-bit)
- Compatible with Hugging Face Transformers
- Accelerates inference on GPUs and CPUs
- Helps deploy LLMs on resource-constrained hardware
Project Samples
License
MIT LicenseFollow AutoGPTQ
Other Useful Business Software
Respond 100x faster, more accurately, and improve your documentation
Slash response times for questionnaires, audits, and RFPs by up to 90%. OptiValue.ai automates the heavy lifting, freeing your team to focus on strategic priorities with intuitive tools for seamless review and validation.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of AutoGPTQ!
