Yujun Shi

DiffDoctor. [ICCV 2025] DiffDoctor: Diagnosing Image Diffusion Models Before Treating

HPSv3. Official implementation of HPSv3: Towards Wide-Spectrum Human Preference Score (ICCV2025)

Next-Patch-Prediction. [AAAI26] Next Patch Prediction

ConsisID. [CVPR 2025 Highlight🔥] Identity-Preserving Text-to-Video Generation by Frequency Decomposition

ROICtrl. Code for [CVPR 2025] ROICtrl: Boosting Instance Control for Visual Generation

MovieGenBench. Movie Gen Bench - two media generation evaluation benchmarks released with Meta Movie Gen

LightningDrag. Experiencing lightning fast (~1s) and accurate drag-based image editing

mar. PyTorch implementation of MAR+DiffLoss https://arxiv.org/abs/2406.11838

ChronoMagic-Bench. [NeurIPS 2024 D&B Spotlight🔥] ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation

VisualSketchpad. Codes for Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models

BrushNet. [ECCV 2024] The official implementation of paper "BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion"

IC-Light. More relighting!

Depth-Anything-V2. [NeurIPS 2024] Depth Anything V2. A More Capable Foundation Model for Monocular Depth Estimation

CFGpp. Official repository for "CFG++: manifold-constrained classifier free guidance for diffusion models" (ICLR2025)

LlamaGen. Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

SGA. [ICML 2024] LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery

Retrieval-based-Voice-Conversion-WebUI. Easily train a good VC model with voice data <= 10 mins!

3DitScene. [ICLR 2025] 3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting

HunyuanDiT. Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

MambaOut. MambaOut: Do We Really Need Mamba for Vision? (CVPR 2025)

LLaVA-NeXT. Python

ASAP. [ICRA 2024] ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility

llama3. The official Meta Llama 3 GitHub site

video2game. Code release of Video2Game

MagicTime. [TPAMI 2025🔥] MagicTime: Time-lapse Video Generation Models as Metamorphic Simulators

magic-boost. Python

VAR. [NeurIPS 2024 Best Paper Award][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

co-tracker. CoTracker is a model for tracking any point (pixel) on a video.

scaling_on_scales. When do we not need larger vision models?

PixArt-alpha. PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis

DragDiffusion. [CVPR2024, Highlight] Official code for DragDiffusion

FeatUp. Official code for "FeatUp: A Model-Agnostic Frameworkfor Features at Any Resolution" ICLR 2024

edm2. EDM2 and Autoguidance -- Official PyTorch implementation

grok-1. Grok open release

Envision3D. Envision3D: One Image to 3D with Anchor Views Interpolation

RRNet. [CVPR2024] The official implementation of paper Relation Rectification in Diffusion Model

PixArt-sigma. PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation

GaLore. GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

distrifuser. [CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

Open-Sora-Plan. This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

piecewise-rectified-flow. PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Accelerator (NeurIPS 2024)

DiT. Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

StableCascade. Official Code for Stable Cascade

MoE-LLaVA. 【TMM 2025🔥】 Mixture-of-Experts for Large Vision-Language Models

InfoBatch. Lossless Training Speed Up by Unbiased Dynamic Data Pruning

Machine-Mindset. An MBTI Exploration of Large Language Models

DiffMorpher. Official Code for DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing (CVPR 2024)

MotionCtrl. Official Code for MotionCtrl [SIGGRAPH 2024]

DragVideo-Official. This is the official implementation for DragVideo

VideoSwap. Code for [CVPR 2024] VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence

avatarstudio. AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

magic-animate. [CVPR 2024] Official repository for "MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model"

xagen. [NeurIPS 2023] XAGen: 3D Expressive Human Avatars Generation

SDE-Drag. Official PyTorch implementation for ICLR2024 paper "The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing"

consistencydecoder. Consistency Distilled Diff VAE

GETAvatar. [ICCV 2023] GETAvatar: Generative Textured Meshes for Animatable Human Avatars

Repfusion. Python

latent-consistency-model. Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

LanguageBind. 【ICLR 2024🔥】 Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment

Hallucination-Attack. Attack to induce LLMs within hallucinations

MVDream-threestudio. 3D generation code for MVDream

FreeU. FreeU: Free Lunch in Diffusion U-Net (CVPR2024 Oral)

Rerender_A_Video. [SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation

MVDream. code page placeholder

FreeDoM. [ICCV 2023] Official PyTorch implementation for the paper "FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model"

magic-edit. MagicEdit: High-Fidelity Temporally Coherent Video Editing

magic-avatar. MagicAvatar: Multimodal Avatar Generation and Animation

ViT-Lens. [CVPR 2024] ViT-Lens: Towards Omni-modal Representations

Dataset_Quantization. [ICCV2023] Dataset Quantization

Awesome-Video-Diffusion. A curated list of recent diffusion models for video generation, editing, and various other applications.

Fooocus. Focus on prompting and generating

MM-Vet. MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024)

q-diffusion. [ICCV 2023] Q-Diffusion: Quantizing Diffusion Models.

awesome-test-time-adaptation. Collection of awesome test-time (domain/batch/instance) adaptation methods

bubogpt. BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs

AvatarGen. [Preprint] AvatarGen: A 3D Generative Model for Animatable Human Avatars

ECON. [CVPR'23, Highlight] ECON: Explicit Clothed humans Optimized via Normal integration

ICON. [CVPR'22] ICON: Implicit Clothed humans Obtained from Normals

AnimateDiff. Official implementation of AnimateDiff.

FreeDrag. [CVPR 2024] Official implementation of FreeDrag: Feature Dragging for Reliable Point-based Image Editing

NeRF-Editing. [CVPR 2022] Code for "NeRF-Editing: Geometry Editing of Neural Radiance Fields"

generative-models. Generative Models by Stability AI

DDPM_inversion. Official pytorch implementation of the paper: "An Edit Friendly DDPM Noise Space: Inversion and Manipulations". CVPR 2024.

stylegan-t. [ICML'23] StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis

DragGAN. Official Code for DragGAN (SIGGRAPH 2023)

NCLaw. [ICML 2023] Learning Neural Constitutive Laws from Motion Observations for Generalizable PDE Dynamics

ChatExcel.

ChatLaw. ChatLaw：A Powerful LLM Tailored for Chinese Legal. 中文法律大模型

pix2pix-zero. Zero-shot Image-to-Image Translation [SIGGRAPH 2023]

Asymmetric_VQGAN. Jupyter Notebook

dift. [NeurIPS'23] Emergent Correspondence from Image Diffusion

Drag3D. DragGAN meets GET3D for interactive mesh generation and editing.

Mix-of-Show. NeurIPS 2023, Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models

Make-A-Protagonist. Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts

DragGAN. Unofficial Implementation of DragGAN - "Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold" （DragGAN 全功能实现，在线Demo，本地部署试用，代码、模型已全部开源，支持Windows, macOS, Linux）

threestudio. A unified framework for 3D content generation.

ImageBind. ImageBind One Embedding Space to Bind Them All

plug-and-play. Official Pytorch Implementation for “Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation” (CVPR 2023)

IF. Python

MasaCtrl. [ICCV 2023] Consistent Image Synthesis and Editing

ELITE. ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation (ICCV 2023, Oral)

rome. Locating and editing factual associations in GPT (NeurIPS 2022)

SAVC. [CVPR 2023] Learning with Fantasy: Semantic-Aware Virtual Contrastive Constraint for Few-Shot Class-Incremental Learning

VideoCrafter. VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models

diffusion-classifier. Diffusion Classifier leverages pretrained diffusion models to perform zero-shot classification without additional training

Text2Video-Zero. [ICCV 2023 Oral] Text-to-Image Diffusion Models are Zero-Shot Video Generators

muse-maskgit-pytorch. Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

FedDecorr. [ICLR2023] Official Implementation of Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning (https://arxiv.org/abs/2210.00226)

CLVQA. [AAAI2023] Symbolic Replay: Scene Graph as Prompt for Continual Learning on VQA Task (Oral)

pv3d. Python

FedBN. [ICLR'21] FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

FedCLS. [ICLR2023] Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning (https://arxiv.org/abs/2210.00226)

REPAIR. Code release for REPAIR: REnormalizing Permuted Activations for Interpolation Repair

balance. The balance python package offers a simple workflow and methods for dealing with biased data samples when looking to infer from them to some target population of interest.

FLamby. Cross-silo Federated Learning playground in Python. Discover 7 real-world federated datasets to test your new FL strategies and try to beat the leaderboard.

spikformer. ICLR 2023, Spikformer: When Spiking Neural Network Meets Transformer

assets.

Federated-Learning. 联邦学习

mage. A PyTorch implementation of MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis

Assemble-Them-All. [SIGGRAPH Asia 2022] Assemble Them All: Physics-Based Planning for Generalizable Assembly by Disassembly

PFLlib. Master Federated Learning in 2 Hours—Run It on Your PC!

FedSim. A coupled vertical federated learning framework that boosts the model performance with record similarities (NeurIPS 2022)

CW-RGP. An official implementation of CW-RGP (NeurIPS 2022, spotlight).

mcr2. Official Implementation of Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction (2020)

targeted-supcon. A PyTorch implementation of the paper Targeted Supervised Contrastive Learning for Long-tailed Recognition

DeRy. [NeurIPS2022] Deep Model Reassembly

breaching. Breaching privacy in federated learning scenarios for vision and text

CEC-CVPR2021. Pytorch code for CVPR2021 paper "Few-Shot Incremental Learning with Continually Evolved Classifiers"

AVA-AVD. Python

leaf. Leaf: A Benchmark for Federated Settings

LIRF. Code for ECCV 2022 paper “Learning with Recoverable Forgetting”

estimateMI. PyTorch implementation of the estimator proposed in the paper "Estimating Differential Entropy under Gaussian Convolutions"

KnowledgeFactor. [ECCV2022] Factorizing Knowledge in Neural Networks

mae. PyTorch implementation of MAE https//arxiv.org/abs/2111.06377

ObsAtk. Implementation of "Towards Adversarially Robust Deep Image Denoising" (IJCAI 2022)

FedScale. FedScale is a scalable and extensible open-source federated learning (FL) platform.

Multitask-learning-over-graphs-simulation. A toy example for illustrating multitask learning under smoothness

paper_reading.

NIID-Bench. Federated Learning Benchmark - Federated Learning on Non-IID Data Silos: An Experimental Study (ICDE 2022)

DINE. code for our CVPR 2022 paper "DINE: Domain Adaptation from Single and Multiple Black-box Predictors"

ELI. (CVPR 2022) Energy-based Latent Aligner for Incremental Learning

HowToLiveLonger. 程序员延寿指南 | A programmer's guide to live longer

pytorch-video-recognition. PyTorch implemented C3D, R3D, R2Plus1D models for video activity recognition.

PoseTriplet. [CVPR 2022] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision (Oral)

CwD. Official Implementation of CVPR 2022 paper: "Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning"

Point-MAE. [ECCV2022] Masked Autoencoders for Point Cloud Self-supervised Learning

PyHessian. PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks

OpenFed. A Comprehensive and Versatile Open-Source Federated Learning Framework

OpenPCDet. OpenPCDet Toolbox for LiDAR-based 3D Object Detection.

PointRCNN. PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud, CVPR 2019.

giraffe. This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"

DIT. [TNNLS] Full Transformer Framework for Robust Point Cloud Registration with Deep Information Interaction

TQVSR. [Findings of EMNLP 2022] AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant

FedTorch. FedTorch is a generic repository for benchmarking different federated and distributed learning algorithms using PyTorch Distributed API.

DDS. Semantic Edge Detection with Diverse Deep Supervision

TisODE. On Robustness of Neural Ordinary Differential Equations

mvp. NeurIPS-2021: Direct Multi-view Multi-person 3D Human Pose Estimation

envpool. C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

CIFS. CIFS: Improving Adversarial Robustness of CNNs via Channel-wise Importance-based Feature Selection

bolei_awesome_posters. CVPR and NeurIPS poster examples and templates

Efficient_SAM. Python

SS-IL-Official. Python

pnp-detr. Implementation of ICCV21 paper: PnP-DETR: Towards Efficient Visual Analysis with Transformers

geodesic_continual_learning. This is a public repository for:

Barlow-Twins-HSIC. Python

barlowtwins. PyTorch implementation of Barlow Twins.

awesome-NeRF. A curated list of awesome neural radiance fields papers

continuum. A clean and simple data loading library for Continual Learning

pytorch3d. PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

awesome-implicit-representations. A curated list of resources on implicit neural representations.

recovering-unbiased-scene-graphs. Official implementation of "Recovering the Unbiased Scene Graphs from the Biased Ones" (ACMMM 2021)

coin. Pytorch implementation of COIN, a framework for compression with implicit neural representations 🌸

volo. VOLO: Vision Outlooker for Visual Recognition

VisionPermutator. MLP-Like Vision Permutator for Visual Recognition (PyTorch)

LV-BERT. LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)

PoseAug. [CVPR 2021 Best Paper Award Candidate] PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation, (Oral, Best Paper Award Finalist)

DenseCL. Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021 Oral.

Refiner_ViT. Python

Transformer-SSL. This is an official implementation for "Self-Supervised Learning with Swin Transformers".

awesome-self-supervised-learning. A curated list of awesome self-supervised methods

PyContrast. PyTorch implementation of Contrastive Learning methods

Awesome_Continual-Lifelong-Incremental_learning.

Multi-View-Information-Bottleneck. Implementation of Multi-View Information Bottleneck

AutoOED. AutoOED: Automated Optimal Experimental Design Platform

pytorch_geometric. Graph Neural Network Library for PyTorch

SupContrast. PyTorch implementation of "Supervised Contrastive Learning" (and SimCLR incidentally)

SHOT-plus. code for our TPAMI 2021 paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

ATDOC. code released for our CVPR 2021 paper "Domain Adaptation with Auxiliary Target Domain-Oriented Classifier"

BA3US. code for our ECCV 2020 paper "A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation"

SHOT. code released for our ICML 2020 paper "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation"

BLIP. Official Implementation of CVPR2021 paper: Continual Learning via Bit-Level Information Preserving

BMP. [CVPR 2021] Body Meshes as Points.

FACIL. Framework for Analysis of Class-Incremental Learning with 12 state-of-the-art methods and 3 baselines.

essentials_for_CIL. Official repository of the paper 'Essentials for Class Incremental Learning'

dino. PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

vissl. VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.

dvit_repo. Python

rl-generalization-paper. A list of papers regarding generalization in (deep) reinforcement learning

TokenLabeling. Pytorch implementation of "All Tokens Matter: Token Labeling for Training Better Vision Transformers"

pylustrator. Visualisations of data are at the core of every publication of scientific research results. They have to be as clear as possible to facilitate the communication of research. As data can have different formats and shapes, the visualisations often have to be adapted to reflect the data as well as possible. We developed Pylustrator, an interface to directly edit python generated matplotlib graphs to finalize them for publication. Therefore, subplots can be resized and dragged around by the mouse, text and annotations can be added. The changes can be saved to the initial plot file as python code.