Volodymyr Kyrylov

radae. Radio Autoencoder - transmission of vocoder features over radio channels

lambda-RLM. Method for Long Context RLMs using verifiable Lambda Calculus

SUMO. Python

ml-tidbits. Various ML tidbits in Python/PyTorch and C++

mlx-skills. A repo of useful MLX skills.

neural-os. Python

PSD. Perturbed Saddle-escape Descent (PSD): a first-order optimizer that escapes strict saddle points in nonconvex problems.

nano-vllm. Nano vLLM

gearmulator. Low Level Emulation of classic VA synths & effects of the late 90s/2000s by emulating the used ICs

ibtop. Real-time terminal monitor for InfiniBand networks - htop for high-speed interconnects

silkworm. Async web scraping framework on top of Rust. Works with Free-threaded Python (`PYTHON_GIL=0`).

macapptree. Repository for macos accessibility parser

LLMxFM_ICRL. Code for the "Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning" paper

uni-circ-le. Python

super-monotonic-align. Python

SuperpositionScaling. Superposition Yields Robust Neural Scaling

Replication-Dataset-Scripts. Scripts for recreating the Replication Dataset for Fundamental Frequency Estimation. Part of the dissertation "Pitch of Voiced Speech in the Short-Time Fourier Transform". © 2020, Bastian Bechtold. All rights reserved.

bota. 🏆 OpenAI Open Model Hackathon Winner - Wildcard category. bota is an autonomous Dota 2 agent powered by GPT-OSS-20B and vLLM, created for OpenAI Open Model Hackathon.

epymarl. An extension of the PyMARL codebase that includes additional algorithms and environment support

egglog. egraphs + datalog!

openai-imo-2025-proofs.

HeavyBall. Efficient optimizers

olleh. An Ollama-compatible CLI for Apple's Foundation Models

delayed-streams-modeling. Kyutai's Speech-To-Text and Text-To-Speech models based on the Delayed Streams Modeling framework.

DailyTalk. Official repository of DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech, ICASSP 2023

mirage. Mirage Persistent Kernel: Compiling LLMs into a MegaKernel

attractors-and-smoothness-RNN. Companion code to the 2020 AISTATS paper: ''Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness"

formal-conjectures. A collection of formalized statements of conjectures in Lean.

chibi-scheme. Official chibi-scheme repository

chatgpt-shell. A multi-llm Emacs shell (ChatGPT, Claude, DeepSeek, Gemini, Kagi, Ollama, Perplexity) + editing integrations

ner_for_fem. Jupyter Notebook

nanoGPT-mup. The simplest, fastest repository for training/finetuning medium-sized GPTs.

training-hot-swap. Pytorch script hot swap: Change code without unloading your LLM from VRAM

packMP3. A compression program for further compressing MP3 audio files

DeepZip. NN based lossless compression

Dzip-torch. Dzip: improved general-purpose lossless compression based on novel neural network modeling

recurrent-pretraining. Pretraining and inference code for a large-scale depth-recurrent language model

SOAP_MUON. Combining SOAP and MUON

tg-extract-history. Downloading of messages from Telegram's groups

unlp-2025-shared-task. UNLP 2025 Shared Task on Detecting Social Media Manipulation

R3GAN. Code for NeurIPS 2024 paper - The GAN is dead; long live the GAN! A Modern Baseline GAN - by Huang et al.

kv-memory-brain. Official Code Repository for the paper "Key-value memory in the brain"

ghostty-shaders. A repository containing many free shaders to use with ghostty (the terminal)

streaming-drl. Deep reinforcement learning without experience replay, target networks, or batch updates.

syncx. Official code repository for NeurIPS 2024 paper "Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discovery"

BitNet. Official inference framework for 1-bit LLMs

imp-bias-ssm-poison. Code for reproducing experiments in "The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels"

REPA. [ICLR'25 Oral] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think

moshi. Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audio codec.

unit-scaling. A library for unit scaling in PyTorch

ghstack. Submit stacked diffs to GitHub on the command line

abile. State-space model perspective on rating systems (pairwise comparisons).

ADAS. [ICLR 2025] Automated Design of Agentic Systems

Agentless. Agentless🐱: an agentless approach to automatically solve software development problems

brainstorm. Fast, flexible and fun neural networks.

ulogme. Automatically collect and visualize usage statistics in Ubuntu/OSX environments.

symvae-aistats2024. Implementations of experiments from the paper.

Grendel-GS. [ICLR 2025 Oral] On Scaling Up 3D Gaussian Splatting Training

HMM_sample_code. CUDA 12.2 HMM demos

PracticalSessions2024. Jupyter Notebook

purejaxql. Simple single-file baselines for Q-Learning in pure-GPU setting

SimplifyingImbalancedTraining. Python

nanotron. Minimalistic large language model 3D-parallelism training

ncps. PyTorch and TensorFlow implementation of NCP, LTC, and CfC wired neural models

sglang. SGLang is a high-performance serving framework for large language models and multimodal models.

rusty-dawg. Rust library for indexing and quickly searching large pretraining corpora

PRISE. Codebase for PRISE: Learning Temporal Action Abstractions as a Sequence Compression Problem

STAR-Adapt. Code for paper "Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models"

mamba. Mamba SSM architecture

chunktd. Code accompanying "Sequence Compression Speeds Up Credit Assignment in Reinforcement Learning"

simple-evals. Python

video-occupancy-models. Python

sgp. Official repository for the paper "Scalable Spatiotemporal Graph Neural Networks" (AAAI 2023)

hydra. Official implementation of "Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers"

mistral-finetune. Python

OpenSTL. OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

WhisperS2T. An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

nanodo. Python

torbi. Viterbi decoding in PyTorch

SurrogateGradientFlossing. This repository contains the implementation code for manuscript: Improving Surrogate Gradient Learning in Spiking Neural Netowrks Using Dynamical Systems Theory

sgdstore. Augmented RNN memory via live SGD

mlx_parallm. Fast parallel LLM inference for MLX

sparse_autoencoder. Python

Ctrl-G. Python

SafeOpt. Safe Bayesian Optimization

rnn-weights-representation-learning. Python

lambda_vit. Python

SmartPlay. SmartPlay is a benchmark for Large Language Models (LLMs). Uses a variety of games to test various important LLM capabilities as agents. SmartPlay is designed to be easy to use, and to support future development of LLMs.

grokfast. Official repository for the paper "Grokfast: Accelerated Grokking by Amplifying Slow Gradients"

ssl-pretext-multi-led. Python

deface. Video anonymization by face detection

MINT-1T. 🍃 MINT-1T: A one trillion token multimodal interleaved dataset.

long-context-asr. Code for the paper: How Much Context Does My Attention-Based ASR System Need?

distil-whisper. Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

Samba. [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling

ISED. Python

xlstm. Official repository of the xLSTM.

attention-cnn. Source code for "On the Relationship between Self-Attention and Convolutional Layers"

dagr. Code for the paper "Low Latency Automotive Vision with Event Cameras", published in Nature

moeut. Python

holograph. A new kind of visual programming

modula. 🧱 Modula software package

diamond. DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model. NeurIPS 2024 Spotlight.

neural-gpu. Code for the Neural GPU model originally described in "Neural GPUs Learn Algorithms"

fundus. A very simple news crawler with a funny name

rq-vae-transformer. The official implementation of Autoregressive Image Generation using Residual Quantization (CVPR '22)

sane_tikz. Reconquer the canvas: beautiful Tikz figures without clunky Tikz code

discord-activity-starter. A stater template for building Discord activities ⚽️

test. Measuring Massive Multitask Language Understanding | ICLR 2021

needle-in-a-needlestack. Python

zett. Code for Zero-Shot Tokenizer Transfer

feed-supermaster. https://github.com/egorsmkv/feed-master with some changes

Zoph_RNN. C++/CUDA toolkit for training sequence and sequence-to-sequence models across multiple GPUs

ThunderKittens. Tile primitives for speedy kernels

pheme. Python

LongICLBench. Code and Data for "Long-context LLMs Struggle with Long In-context Learning" [TMLR2025]

lazy-training-CNN. Python

linear_open_lm. A repository for research on medium sized language models.

Curse-of-memory. Curse-of-memory phenomenon of RNNs in sequence modelling

DoRA_ICLR24. This repo contains the official implementation of ICLR 2024 paper "Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video""

One2Branch. Python

shuffle-coding. Lossless compression of sets, graphs and other unordered data

error-free-differentiable-swap-functions. Official repository of "Generalized Neural Sorting Networks with Error-Free Differentiable Swap Functions," which has been presented at ICLR 2024

causallm_icl. Python

alam. The official code for Averaged Low-Precision Activation (ALAM)

s2pg. Code of the Paper "Time-Efficient Reinforcement Learning with Stochastic Stateful Policies"

llm.c. LLM training in simple, raw C/CUDA

champ. [ECCV 2024] Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance

MultiscalePINNs. Python

urnn. Unitary Evolution Recurrent Neural Network

role-decomposition. Python

dragoman. Python

ModelZooDataset. Model Zoos published at the NeurIPS 2022 Dataset & Benchmark track: "Model Zoos: A Dataset of Diverse Populations of Neural Network Models"

PINNs. Physics Informed Deep Learning: Data-driven Solutions and Discovery of Nonlinear Partial Differential Equations

lighteval. Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

v6502r. visual6502 remixed

fregrad. Code repository for FreGrad

fastfit. FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes

morton. C

graph-of-thoughts. Official Implementation of "Graph of Thoughts: Solving Elaborate Problems with Large Language Models"

Sketchbook. 3D playground built on three.js and cannon.js.

video2game. Code release of Video2Game

LangToGroup. Implementation of Isoperimetric and Isodiametric Functions of Groups in Haskell

gsheets-connection. Python

speech-trident. Awesome speech/audio LLMs, representation learning, and codec models

d3pm. Minimal Implementation of a D3PM in pytorch

Multi-Agent-Reinforcement-Learning-Environment. Hello, I pushed some python environments for Multi Agent Reinforcement Learning.

JaxMARL. Multi-Agent Reinforcement Learning with JAX

jumanji. 🕹️ A diverse suite of scalable reinforcement learning environments in JAX

gymnax. RL Environments in JAX 🌍

ctc-copy. [EMNLP'23] Code for "Non-autoregressive Text Editing with Copy-aware Latent Alignments".

HGRN2. HGRN2: Gated Linear RNNs with State Expansion

torchcomp. Differentiable dynamic range controller in PyTorch.

weightnorm. Example code for Weight Normalization, from "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks"

weight_norm. Lasagne code for weight normalization

heinsen_attention. Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)

recurrentgemma. Open weights language model from Google DeepMind, based on Griffin.

rnngineering. Engineering the state of RNN language models (Mamba, RWKV, etc.)

llm-adaptive-attacks. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks [ICLR 2025]

LASP. Linear Attention Sequence Parallelism (LASP)

SmallInitEmb. LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

gptcore. Fast modular code to create and train cutting edge LLMs

schedule_free. Schedule-Free Optimization in PyTorch

tokenization-scorer. Simple-to-use scoring function for arbitrarily tokenized texts.

cifar10-airbench. CIFAR-10 speedruns: 94% in 2.6 seconds and 96% in 27 seconds

gfn-lm-tuning. Jupyter Notebook

libre-caslon-condensed. An open-source condensed serif font family derived from the beloved Libre Caslon.

openui. OpenUI let's you describe UI using your imagination, then see it rendered live.

DADApy. Distance-based Analysis of DAta-manifolds in python

mad-lab. A MAD laboratory to improve AI architecture designs 🧪

rnn-icrag. Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"

ssm-benchmark. Long-range arena (LRA) experimets for paper

latent-diffusion-for-language. Python

rr-experiments. Experiments with rational recurrences, pooling functions, and absolute values.

rational-recurrences. Implementation for "Rational Recurrences", Peng et al., EMNLP 2018.

maxtext. A simple, performant, and scalable Jax LLM!

ml-planner. Python

RLHF-Reward-Modeling. Recipes to train reward model for RLHF.

lightning-thunder. PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

isat. In situ adaptive tabulation (ISAT) python implementation

locost-summarization. Assembly

ozIMMU. FP64 equivalent GEMM by the Ozaki scheme with Int8 Tensor Cores

geotorch. Constrained optimization toolkit for PyTorch

FIND. FIND: search For Inductive biases IN Deep seq2seq

neural_testbed. Jupyter Notebook

enn. Python

big_vision. Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

NumericsOfML. Notes for the Numerics of Machine Learning Lecture Course at the University of Tübingen

grok. Python

HVM2. A massively parallel, optimal functional runtime in Rust

snac. Multi-Scale Neural Audio Codec (SNAC) compresses audio into discrete codes at a low bitrate

Next-Token-Failures. Python

transformer-debugger. Python

GEAR. GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

bayesian_flow_networks. A PyTorch implementation of Bayesian flow networks (Graves et al., 2023).

bayesian_lora. Bayesian Low-Rank Adaptation for Large Language Models

ryg_rans. Simple rANS encoder/decoder (arithmetic coding-ish entropy coder).

flora-opt. This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.

jax-sph. JAX-SPH: A Differentiable Smoothed Particle Hydrodynamics Framework

autocrit. A repository for transformer critique learning and generation