Fan Bao

Vidu-S1. Vidu S1: A Real-Time Interactive Video Generation Model

minWM. A Minimal and Elegant Framework & Tutorial for Real-Time Interactive World Models

TurboDiffusion. TurboDiffusion: 100–200× Acceleration for Video Diffusion Models

SageAttention. [ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

MNBVC. MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

MetaCLIP. NeurIPS 2025 Spotlight; ICLR2024 Spotlight; CVPR 2024; EMNLP 2024

consistencydecoder. Consistency Distilled Diff VAE

lm-evaluation-harness. A framework for few-shot evaluation of language models.

datablations. Scaling Data-Constrained Language Models

open_llama. OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

RedPajama-Data. The RedPajama-Data repository contains code for preparing large datasets for training large language models.

cc_net. Tools to download and cleanup Common Crawl data

LLMDataHub. A quick guide (especially) for trending instruction finetuning datasets

ViTDet. Unofficial implementation for [ECCV'22] "Exploring Plain Vision Transformer Backbones for Object Detection"

LLMSurvey. The official GitHub page for the survey paper "A Survey of Large Language Models".

Baichuan2. A series of large language models developed by Baichuan Intelligent Technology

VisCPM. [ICLR'24 spotlight] Chinese and English Multimodal Large Model Series (Chat and Paint) | 基于CPM基础模型的中英双语多模态大模型系列

Diffusion-Models-Papers-Survey-Taxonomy. Diffusion model papers, survey, and taxonomy

generative_agents. Generative Agents: Interactive Simulacra of Human Behavior

StableSR. [IJCV2024] Exploiting Diffusion Prior for Real-World Image Super-Resolution

Awesome-LLM. Awesome-LLM: a curated list of Large Language Model

awesome-digital-human. Digital Human Resource: 2D/3D/4D Human Modeling, Avatar Generation & Animation, Clothed People Digitalization, Virtual Try-On, etc.

baselines. OpenAI Baselines: high-quality implementations of reinforcement learning algorithms

tianshou. An elegant PyTorch deep reinforcement learning library.

Awesome-Multimodal-Research. A curated list of Multimodal Related Research.

awesome-multimodal-ml. Reading list for research topics in multimodal machine learning

Baichuan-7B. A large-scale 7B pretraining language model developed by BaiChuan-Inc.

FastChat. An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

awesome-LLMs-In-China. 中国大模型

Mr.-Ranedeer-AI-Tutor. A GPT-4 AI Tutor Prompt for customizable personalized learning experiences.

annotated_deep_learning_paper_implementations. 🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

roformer. Rotary Transformer

awesome. 😎 Awesome lists about all kinds of interesting topics

Awesome-Multimodal-Large-Language-Models. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models

controlvideo. Official implementation for "ControlVideo: Adding Conditional Control for One Shot Text-to-Video Editing"

LAVIS. LAVIS - A One-stop Library for Language-Vision Intelligence

RWKV-LM. RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

prolificdreamer. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (NeurIPS 2023 Spotlight)

StableStudio. Community interface for generative AI

ControlNet-v1-1-nightly. Nightly release of ControlNet 1.1

tiktoken. tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Anime2Sketch. A sketch extractor for anime/illustration.

T2I-Adapter. T2I-Adapter

Prompt-Diffusion. Official PyTorch implementation of the paper "In-Context Learning Unlocked for Diffusion Models"

alpaca-lora. Instruct-tune LLaMA on consumer hardware

MUSE-Pytorch. An in-context conditioning version of MUSE with pre-trained checkpoints.

IF. Python

Kandinsky-2. Kandinsky 2 — multilingual text2image latent diffusion model

cc2dataset. Easily convert common crawl to a dataset of caption and document. Image/text Audio/text Video/text, ...

MOSS. An open-source tool-augmented conversational language model from Fudan University

chatgpt-academic-prompt-helper. 本项目是一个油猴脚本 (Tampermonkey)，旨在便于网页版 Chatgpt 的 prompt 输入，并内置了一些常用的学术 prompt 模板。

StableLM. StableLM: Stability AI Language Models

MiniGPT-4. Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

Caption-Anything. Caption-Anything is a versatile tool combining image segmentation, visual captioning, and ChatGPT, generating tailored captions with diverse controls for user preferences. https://huggingface.co/spaces/TencentARC/Caption-Anything https://huggingface.co/spaces/VIPLab/Caption-Anything

AutoGPT. AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

dolly. Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

FindTheChatGPTer. ChatGPT爆火，开启了通往AGI的关键一步，本项目旨在汇总那些ChatGPT的开源平替们，包括文本大模型、多模态大模型等，为大家提供一些便利

MultiDiffusion. Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)

mixture-of-diffusers. Mixture of Diffusers for scene composition and high resolution image generation

multidiffusion-upscaler-for-automatic1111. Tiled Diffusion and VAE optimize, licensed under CC BY-NC-SA 4.0

open_clip. An open source implementation of CLIP.

Grounded-Segment-Anything. Grounded SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything

GPT-4-LLM. Instruction Tuning with GPT-4

awesome-gpt4. A curated list of prompts, tools, and resources regarding the GPT-4 language model.

JARVIS. JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

gpt_academic. 为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。

Llama-X. Open Academic Research on Improving LLaMA to SOTA LLM

karlo. Python

nanoGPT. The simplest, fastest repository for training/finetuning medium-sized GPTs.

minGPT. A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

evals. Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

stanford_alpaca. Code and documentation to train Stanford's Alpaca models, and generate the data.

ChatGLM-6B. ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

unidiffuser. Code and models for the paper "One Transformer Fits All Distributions in Multi-Modal Diffusion"

optimate. A collection of libraries to optimise AI model performances

diffusers. 🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.

ControlNet. Let us control diffusion models!

llama. Inference code for Llama models

awesome-chatgpt-prompts-zh. ChatGPT 中文调教指南。各种场景使用指南。学习怎么让它听你的话。

metaseq. Repo for external large-scale work

DALLE-pytorch. Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

BigModelPapers. A collection of some large model papers.

fvcore. Collection of common code that's shared among different research projects in FAIR computer vision team.

seminar_multimodal_dl. https://slds-lmu.github.io/seminar_multimodal_dl/

tuning_playbook. A playbook for systematically maximizing the performance of deep learning models.

EEGSDE. This is the official implementation for Equivariant Enengy-guided SDE for Inverse Molecule Design (ICLR 2023)

sd-webui-image-sequence-toolkit. Extension for AUTOMATIC111's WebUI

DiffEdit-stable-diffusion. An unofficial implement of DiffEdit on stable-diffusion

Optimus. Optimus: the first large-scale pre-trained VAE language model

xformers. Hackable and optimized Transformers building blocks, supporting a composable construction.

stable-diffusion-videos. Create 🔥 videos with Stable Diffusion by exploring the latent space and morphing between text prompts

diffusion_experiments. An implementation of DiffEdit using Stable Diffusion

U-ViT. A PyTorch implementation of the paper "All are Worth Words: A ViT Backbone for Diffusion Models".

Versatile-Diffusion. Versatile Diffusion: Text, Images and Variations All in One Diffusion Model, arXiv 2022 / ICCV 2023

Awesome-Text-to-Image. (ෆ`꒳´ෆ) A Survey on Text-to-Image Generation/Synthesis.

neuraltalk2. Efficient Image Captioning code in Torch, runs on GPU

coco-caption. kdexd/coco-caption@de6f385

EGSDE. Official implementation for "EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations" (NIPS 2022)