Andrei Panferov

looped-moe. code for "Tying the Loop - Tied Expert Layers in Mixture-of-Experts Language Models"

slides. Vue

OneCompression. Python package for LLM compression

Quartet-II. Quartet II Official Code

FP-Quant. Python

QuickRecap. An iOS app that integrates a Large Language Model (LLM) to process audio recordings for transcription and summarization.

qutlass. QuTLASS: CUTLASS-Powered Quantized BLAS for Deep Learning

Quartet. Jupyter Notebook

circle-guard-bench. First-of-its-kind AI benchmark for evaluating the protection capabilities of large language model (LLM) guard systems (guardrails and safeguards)

cline. Autonomous coding agent as an SDK, IDE extension, or CLI assistant.

Mathador-LM. Code for the EMNLP 2024 paper "Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on LLMs".

qtip. Python

QQQ. QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.

CPP_from_1998_to_2020. Technical Note: From C++98 to C++2x

QUICK. QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

marlin. FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

QuIP-for-all. QuIP quantization

AQLM. Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

adhd_helper. Go

FriendsDontLetFriends. Friends don't let friends make certain types of data visualization - What are they and why are they bad.

quip-sharp. Python

gptq. Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

qmoe. Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".

meditron. Meditron is a suite of open-source medical Large Language Models (LLMs).

LookaheadDecoding. [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

PFL-DocVQA-Competition. Python

QUIK. Repository for the QUIK project, enabling the use of 4bit kernels for generative inference - EMNLP 2024

optim. 💎A site, that contains systematic optimization methods and theory review

Megatron-LLM. distributed trainer for LLMs

candle. Minimalist ML framework for Rust

EDEN-Distributed-Mean-Estimation. This repository is the official implementation of 'EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning' (ICML 2022).

spellah. A nasty project for the 2014's Microsoft Research Summer School.

llama2.c. Inference Llama 2 in one file of pure C

CocktailSGD. Python

GPU-Puzzles. Solve puzzles. Learn CUDA.

SpQR. Python

plusplus. Enables increment operators in Python using a bytecode hack

g-helper. Lightweight Armoury Crate alternative for Asus laptops with nearly the same functionality. Works with ROG Zephyrus, Flow, TUF, Strix, Scar, ProArt, Vivobook, Zenbook, Expertbook, ROG Ally, and many more.

what-happens-when. An attempt to answer the age old interview question "What happens when you type google.com into your browser and press enter?"

SoulsGym. Gymnasium extension for DarkSouls III, Elden Ring, and other Souls games

lightning-Hivemind. Lightning Training strategy for HiveMind

peft. 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Dialog. A PyTorch Implementation of japanese chatbot using BERT and Transformer's decoder

wasm4-raycaster. A demo raycaster for the WASM-4 engine.

shad-go. Golang programming course

FBGEMM. FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/

bitsandbytes. Accessible large language models via k-bit quantization for PyTorch.

Practical_RL. A course in reinforcement learning in the wild

UnnaturalScrollWheels. Invert scroll direction for physical scroll wheels while maintaining "Natural" scrolling for trackpads on MacOS

matchering. 🎚️ Open Source Audio Matching and Mastering

speedscope. 🔬 A fast, interactive web-based viewer for performance profiles.

efficient-dl-systems. Efficient Deep Learning Systems course materials (HSE, YSDA)

vk-clones. Python

qiskit. Qiskit is an open-source SDK for working with quantum computers at the level of extended quantum circuits, operators, and primitives.

4chan-x. Adds various features to anonymous imageboards.