Ian Shih

VoiceBench. [TACL'26] VoiceBench: Benchmarking LLM-Based Voice Assistants

Awesome-Full-Duplex-SDM. A curated list of full-duplex spoken dialogue models & benchmarks

specplotter. Python library for nice-looking spectrograms.

NanoCodec. NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference

ALMTokenizer2. The open source code of ALMTokenizer2: Towards Low bit-rate and Semantic-rich Audio Tokenizer with Flow-based Scalar Diffusion Transformer Decoder

IguanaTex. A PowerPoint add-in to insert LaTeX equations into PowerPoint presentations on Windows and Mac

VoiceStar. VoiceStar: Robust, Duration-controllable TTS that can Extrapolate

spiritlm. Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".

SyllableLM. Official Code for SyllableLM: Learning Coarse Semantic Units for Speech Language Models

Awesome-Mamba-Collection. A curated collection of papers, tutorials, videos, and other valuable resources related to Mamba.

whisper-flamingo. Whisper-Flamingo [Interspeech 2024] and mWhisper-Flamingo [IEEE SPL 2025] for Audio-Visual Speech Recognition and Translation

dinosr. DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning

MU-LLaMA. MU-LLaMA: Music Understanding Large Language Model

DeepMIR. Teaching material for the course "Deep Learning for Music Analysis and Generation" I taught at National Taiwan University

av-superb. A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models (ICASSP 2024)

Awesome-Referring-Image-Segmentation. :books: A collection of papers about Referring Image Segmentation.

seamless_communication. Foundational Models for State-of-the-Art Speech and Text Translation

awesome-tips.

whisper-at. Code and Pretrained Models for Interspeech 2023 Paper "Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong Audio Event Taggers"

devise. A fast, minimal, responsive Hugo theme for blogs.

espresso. Espresso: A Fast End-to-End Neural Speech Recognition Toolkit

Awesome-Transformer-Attention. An ultimately comprehensive paper list of Vision Transformer/Attention, including papers, codes, and related websites

anticipation. Anticipatory Autoregressive Models

test-repo. A place to test GitHub Actions or anything interesting

JARVIS. JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

the-algorithm. Source code for the X Recommendation Algorithm

RepLAI. Self-supervised algorithm for learning representations from ego-centric video data. Code is tested on EPIC-Kitchens-100 and Ego4D in PyTorch. (NeurIPS 2022)

multimodal. TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

muavic. MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation

AudioLDM. AudioLDM: Generate speech, sound effects, music and beyond, with text.

CLAP. Contrastive Language-Audio Pretraining

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

react-notion-x. Fast and accurate React renderer for Notion. TS batteries included. ⚡️

nextjs-notion-starter-kit. Deploy your own Notion-powered website in minutes with Next.js and Vercel.

Serverless-Forms. Serverless forms templates with NO REDIRECTS powered for FREE by Google.

audio-ai-timeline. A timeline of the latest AI models for audio generation, starting in 2023!

awesome-notion. A curated list of awesome Notion related things.

riffusion-hobby. Stable diffusion for real-time music generation

w2v2-air-traffic. This is a repository for a paper accepted at the 2022 IEEE Spoken Language Technology Workshop (SLT 2022)

pop2piano. Official Repo of the paper "Pop2Piano : Pop Audio-based Piano Cover Generation"

Conference-Acceptance-Rate. Acceptance rates for the major AI conferences

multimodal-ml-music. List of academic resources on Multimodal ML for Music

av_hubert. A self-supervised learning framework for audio-visual speech

T5lephone. Code for T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5

ThemeTransformer. The official implementation of Theme Transformer. A Theme-based music generation. IEEE TMM

CapDec. CapDec: SOTA Zero Shot Image Captioning Using CLIP and GPT2, EMNLP 2022 (findings)

Mubert-Text-to-Music. A simple notebook demonstrating prompt-based music generation via Mubert API

jukedrummer. Python

vector-quantize-pytorch. Vector (and Scalar) Quantization, in Pytorch

SpeechCLIP. SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model, Accepted to IEEE SLT 2022

word-discovery. Word Discovery in Visually Grounded, Self-Supervised Speech Models

ai-audio-startups. Community list of startups working with AI in audio and music technology

icones. ⚡️ Icon Explorer with Instant searching, powered by Iconify

awesome-holography. A curated list of resources on holographic displays.

muzic. Muzic: Music Understanding and Generation with Artificial Intelligence

aws-lex-noisy-spoken-language-understanding.

SlotGated-SLU. Slot-Gated Modeling for Joint Slot Filling and Intent Prediction

zerospeech2021. Zerospeech Challenge 2021: validation and evaluation software

mmm. MMM: Modular MIDI Manipulator

s3prl. Self-Supervised Speech Pre-training and Representation Learning Toolkit

fairseq. Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

Leetcode_company_frequency. Collection of leetcode company tag problems. Periodically updating.

NTUEESAAD_SlackCleaner. Python

Paper-Writing-Tips. MLNLP社区用来帮助大家避免论文投稿小错误的整理仓库。 Paper Writing Tips

git-rebase-via-merge. Resolve conflicts with minimum pain.

gradio. Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!

GradCache. Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint

synesthesia. Python

figaro. Source code for "FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control"

transformers. 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

s4. Structured state space sequence models

visual_midi. Converts a pretty midi sequence to a bokeh plot.

accelerate. 🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

ee_quals. :books: Resources and guide for preparing for Stanford's electrical engineering qualification exams.

phd-2048. Move the bricks to complete your PhD

REMI_PyTorch. PYTorch version of REMI

sensor_your_music. A tool for transmitting phone sensors to puredata with WebRTC

cmd2. cmd2 - quickly build feature-rich and user-friendly interactive command line applications in Python

webrtc-firebase-demo. Video Chat with WebRTC and Firebase

hierarchical-structure-analysis. Algorithm and Data for paper "Automatic Detection of Hierarchical Structure and Influence of Structure on Melody, Harmony and Rhythm in Popular Music"

GiantMIDI-Piano. Python

SusPedal-Gen. This is the repository for Learning to Generate Piano Music With Sustain Pedals

picard. PICARD - Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. PICARD is a ServiceNow Research project that was started at Element AI.

midi2Tiles. A tool for creating synthesia-like piano tiles effect from midi files.

MIDI-BERT. [JCMS 2024] This is the official repository for the paper, MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Understanding.

syn2midi. Create pianobooster midi from youtube piano video lessons (Synthesia).

midi2video. convert midi file to piano video with highlighted keys

MuseMorphose. PyTorch implementation of MuseMorphose (published at IEEE/ACM TASLP), a Transformer-based model for music style transfer.

BachelorProject. Python

Doc. This is the documentation website generated with https://docusaurus.io/.

MusicChain. 🎹🎵🎶 A platform to make Original and Cover Visible and Valuable.

login-system. Login system for NTUEE Makerspace

AAG-Visualizer. 🌆 🏙 🌃 Viz.js Graphviz - An Elegant Visualizer for And-Inverter Graph

Magisk. The Magic Mask for Android

submarine. Submarine is Cloud Native Machine Learning Platform.

CADContest2020. C++

NTUEECourseWebsite2020. JavaScript

chorder. A chord identifier and harmonizer for MIDI files

LightDance-RPi. Source codes that run on RPi for controlling the light hardware.

DSP_2019. 2019 Fall, Digital Speech Processing

Automatic-Forehead-Thermometer. C++

Repair-Automatic-Forehead-Thermometer. Python

NTUEE_light_dance. Light Dance of NTUEE Night, 2020 Spring

dsnp_test_cases. 💻 Test cases for NTUEE Course "Data Structures and Programming".

en.javascript.info. Modern JavaScript Tutorial