jllllll

bitsandbytes-windows-webui. Windows compile of bitsandbytes for use in text-generation-webui.

llama-cpp-python-cuBLAS-wheels. Wheels for llama-cpp-python compiled with cuBLAS support

exllama. A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

one-click-installers. Simplified installers for oobabooga/text-generation-webui.

flash-attention. Fast and memory-efficient exact attention - Windows wheels

bitsandbytes. 8-bit CUDA functions for PyTorch

GPTQ-for-LLaMa-CUDA. A combination of Oobabooga's fork and the main cuda branch of GPTQ-for-LLaMa in a package format.

GPTQ-for-LLaMa-Wheels. Precompiled Wheels for GPTQ-for-LLaMa

ctransformers-cuBLAS-wheels. ctransformers wheels with pre-built CUDA binaries for additional CUDA and AVX versions.

windows-venv-installers. Standalone, dependency-less scripts for automatically setting up a virtual environment for easy project installation on Windows.

AutoGPTQ. An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

exllamav2. A fast inference library for running LLMs locally on modern consumer-class GPUs

text-generation-webui. A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA.

h2ogpt. Private Q&A and summarization of documents+images or chat with local GPT, 100% private, no data leaks, Apache 2.0. Demo: https://gpt.h2o.ai/