Max Morrison

unredact. A simple tool for reading in poorly redacted documents and reproducing their origional form

dacvae. DACVAE

chatterbox. SoTA open-source TTS

BridgeVoC. This is the repository for the work "BridgeVoC: Revitalizing Neural Vocoder from a Restoration Perspective".

SageAttention. [ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

csm. A Conversational Speech Generation Model

P. The P programming language.

Zonos. Zonos-v0.1 is a leading open-weight text-to-speech model trained on more than 200k hours of varied multilingual speech, delivering expressiveness and quality on par with—or even surpassing—top TTS providers.

stable-codec. A family of state-of-the-art Transformer-based audio codecs for low-bitrate high-quality audio coding.

moshi. Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audio codec.

DNSMOSPro. Official implementation of DNSMOS Pro (accepted at INTERSPEECH 2024).

awesomeMLSys. An ML Systems Onboarding list

grafx. GRAFX: An Open-Source Library for Audio Processing Graphs in PyTorch

MARS5-TTS. MARS5 speech model (TTS) from CAMB.AI

IMS-Toucan. Controllable and fast Text-to-Speech for over 7000 languages!

MR-MT3. MR-MT3: Memory Retaining Multi-Track Music Transcription to Mitigate Instrument Leakage

CoMoSVC. CoMoSVC: One-Step Consistency Model Based Singing Voice Conversion & Singing Voice Clone

stream-vc. An unofficial PyTorch implementation of the StreamVC(Real-Time Low-Latency Voice Conversion)

genanki. A Python 3 library for generating Anki decks

anki-decks. Anki decks I've created

bungee. A C++ library for time-stretching and pitch-shifting audio with high quality in realtime or offline.

TTS. 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

VoicePAT. VoicePAT is a modular and efficient toolkit for voice privacy research, with main focus on speaker anonymization.

VoiceCraft. Zero-Shot Speech Editing and Text-to-Speech in the Wild

SoundSeek. SoundSeek is a new interface to search through your sound file libraries

tango. A family of diffusion models for text-to-audio generation.

stable-audio-metrics. Metrics for evaluating music and audio generative models – with a focus on long-form, full-band, and stereo generations.

fairseq. Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

GPT-SoVITS. 1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

wespeaker. Research and Production Oriented Speaker Verification, Recognition and Diarization Toolkit

Formant-estimation. Python

Diff-HierVC. Official Pytorch Implementation of "Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation"

StyleTTS2. StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models

Diff-Foley. Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models

dinosr. DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

edm. Elucidating the Design Space of Diffusion-Based Generative Models (EDM)

mamba. Mamba SSM architecture

torch-nansypp. NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis

nansypp. Unofficial implementation of NANSY++ in Pytorch Lightning

marker. Convert PDF to markdown + JSON quickly with high accuracy

multitask. Python

pflowtts_pytorch. Unofficial implementation of NVIDIA P-Flow TTS paper

vits2_pytorch. unofficial vits2-TTS implementation in pytorch

stable-audio-tools. Generative models for conditional audio generation

AudioSep. Official implementation of "Separate Anything You Describe"

autogen. A programming framework for agentic AI

torchscale. Foundation Architecture for (M)LLMs

audiocraft. Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

descript-audio-codec. State-of-the-art audio codec with 90x compression factor. Supports 44.1kHz, 24kHz, and 16kHz mono/stereo audio.

audiotools. Object-oriented handling of audio data, with GPU-powered augmentations, and more.

vocos. Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

panphon. Python package and data files for manipulating phonological segments (phones, phonemes) in terms of universal phonological features.

safetensors. Simple, safe way to store and distribute tensors

YourTTS. YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

x-transformers. A concise but complete full-attention transformer with a set of promising experimental features from various papers

AcademiCodec. AcademiCodec: An Open Source Audio Codec Model for Academic Research

soundstorm-pytorch. Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

rotary-embedding-torch. Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

AppFlowy. Bring projects, wikis, and teams together with AI. AppFlowy is the AI collaborative workspace where you achieve more without losing control of your data. The leading open source Notion alternative.

musiclm-pytorch. Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

naturalspeech2-pytorch. Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

AlignTTS. Implementation of the AlignTTS

gpt4all. GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

ml-ane-transformers. Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)

monotonic_align. Monotonic Alignment Search

radtts. Provides training, inference and voice conversion recipes for RADTTS and RADTTS++: Flow-based TTS models with Robust Alignment Learning, Diverse Synthesis, and Generative Modeling and Fine-Grained Control over of Low Dimensional (F0 and Energy) Speech Attributes.

ChatGPT. Reverse engineered ChatGPT API

webdataset. A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.

audio-ai-timeline. A timeline of the latest AI models for audio generation, starting in 2023!

lyra. A Very Low-Bitrate Codec for Speech Compression

vall-e. An unofficial PyTorch implementation of the audio LM VALL-E

whisperX. WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

sheetsage. Transcribe music into lead sheets!

MB-iSTFT-VITS. Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform

SGConv. Jupyter Notebook

encodec. State-of-the-art deep learning based audio codec supporting both mono 24 kHz audio and stereo 48 kHz audio.

xformers. Hackable and optimized Transformers building blocks, supporting a composable construction.

charsiu. Charsiu: A neural phonetic aligner.

HN-UnifiedSourceFilterGAN. Python

SRD-VC. Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversion (Interspeech 2022)

CharsiuG2P. Multilingual G2P in 100 languages

audiolm-pytorch. Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

spafe. :sound: spafe: Simplified Python Audio Features Extraction

perceptions. Perceptions of Probability and Numbers

s4. Structured state space sequence models

annotated-s4. Implementation of https://srush.github.io/annotated-s4

torchinterp1d. 1D interpolation for pytorch

annotated_latex_equations. Examples of how to create colorful, annotated equations in Latex using Tikz.

Montreal-Forced-Aligner. Command line utility for forced alignment using Kaldi

denoising-diffusion-gan. Tackling the Generative Learning Trilemma with Denoising Diffusion GANs https://arxiv.org/abs/2112.07804

ppg-vc. PPG-Based Voice Conversion

speechbrain. A PyTorch-based Speech Toolkit

hifi-gan. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

WaveGrad. Implementation of WaveGrad high-fidelity vocoder from Google Brain in PyTorch.

HeadphoneCheck. Headphone screening task to facilitate web-based auditory experiments.

wavelet_prosody_toolkit. Python

prosody. Helsinki Prosody Corpus and A System for Predicting Prosodic Prominence from Text

pyloudnorm. Flexible audio loudness meter in Python with implementation of ITU-R BS.1770-4 loudness algorithm

fac-via-ppg. Foreign Accent Conversion by Synthesizing Speech from Phonetic Posteriorgrams (Interspeech'19)

nwb. A toolkit for React, Preact, Inferno & vanilla JS apps, React libraries and other npm modules for the web, with no configuration (until you need it)

audioset_tagging_cnn. Python

react-player. A React component for playing a variety of URLs, including file paths, YouTube, Facebook, Twitch, SoundCloud, Streamable, Vimeo, Wistia and DailyMotion

voice_datasets. 🔊 A comprehensive list of open-source datasets for voice and sound computing (95+ datasets).

coolgpus. GPU fan control for headless Linux

FCN-f0. Fully-Convolutional Network for Pitch Estimation of Speech Signals

PlotNeuralNet. Latex code for making neural networks diagrams

pypYIN. python pYIN

code-server. VS Code in the browser

torch-audiomentations. Fast audio data augmentation in PyTorch. Inspired by audiomentations. Useful for deep learning.

nnAudio. Audio processing by using pytorch 1D convolution network

PyTSMod. An open-source Python library for audio time-scale modification.

PerceptualAudio. Perceptual Metrics of Audio - perceptually relevant loss function. DPAM and CDPAM

UnsupSeg. Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentation (INTERSPEECH 2020)

pytorch_xvectors. Deep speaker embeddings in PyTorch, including x-vectors. Code used in this work: https://arxiv.org/abs/2007.16196

g2ps. Data and code for grapheme-to-phoneme transducers in lots of languages

Phonetisaurus. Phonetisaurus G2P

TD-PSOLA. A simple pitch shifting script (Time-Domain Pitch-Synchronous Overlap and Add)

gentle. gentle forced aligner

pyrubberband. python wrapper for rubberband

argbind. Simple package for binding functions to CLI or config files.

gentle. gentle forced aligner

diffusion. Denoising Diffusion Probabilistic Models

rsmf. Right-size my figures

POP909-Dataset. This is the dataset repository for the paper: POP909: A Pop-song Dataset for Music Arrangement Generation

contrastive-unpaired-translation. Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)

styled-components. Fast, expressive styling for React. Server components, client components, streaming SSR, React Native—one API.

pytorch-lightning. Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

MelNet. Implementation of "MelNet: A Generative Model for Audio in the Frequency Domain"

Resemblyzer. A python package to analyze and compare voices with deep learning

soxbindings. Python bindings for SoX, aiming to replicate a subset of the command line sox utility.

LPCNet. Efficient neural speech synthesis

MMdnn. MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

g2p. g2p: English Grapheme To Phoneme Conversion

CUSP-GX6004.005_ML4Sound. Jupyter Notebook

python_audio_loading_benchmark. Benchmark popular audio i/o packages

espnet. End-to-End Speech Processing Toolkit

streamlit. Streamlit — A faster way to build and share data apps.

gputil. A Python module for getting the GPU status from NVIDA GPUs using nvidia-smi programmically in Python

pysox. Python wrapper around sox.

autovc. AutoVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss

nussl. A flexible source separation library in Python

flask-react-boilerplate. Simple boilerplate for a Flask backend and React client

awesome-python-scientific-audio. Curated list of python software and packages related to scientific research in audio