Yahui Liu

Awesome_Think_With_Images. Resources and paper list for "Thinking with Images for LVLMs". This repository accompanies our survey on how LVLMs can leverage visual information for complex reasoning, planning, and generation.

EfficientMoE. Official implementation of "Efficient Training of Diffusion MoE models: A Practical Recipe"

BLIP3o. Official implementation of BLIP3o-Series

deepseek-ocr-encoder. A handy and elastic encoder for DeepSeek OCR vision tasks

DiffusionNFT. [ICLR 2026 Oral] DiffusionNFT: Online Diffusion Reinforcement with Forward Process

Claudable. Claudable is an open-source web builder that leverages local CLI agents, such as Claude Code, Codex, Gemini CLI, Qwen Code, and Cursor Agent, to build and deploy products effortlessly.

VibeVoice. Open-Source Frontier Voice AI

DreamOn. Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas

MiroFlow. 🏆 Top-1 on 5+ benchmarks | Web UI | Supports MiroThinker, Claude, Kimi, OpenAI

Qwen-Image. Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing.

AR-GRPO. Training Autoregressive Image Generation models via Reinforcement Learning

slime. slime is an LLM post-training framework for RL Scaling.

siiRL. siiRL: Shanghai Innovation Institute RL Framework for Advanced LLMs and Multi-Agent Systems

tau-bench. Code and Data for Tau-Bench

Leanabell-Prover-V2. Verifier-integrated reasoning for formal theorem proving via RL.

CodeTest. C++

seqpe. Python

UNITE. official code for "Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrieval"

DiffusionReward. Public code and model

ControlAR. [ICLR 2025] ControlAR: Controllable Image Generation with Autoregressive Models

Leanabell-Prover. Through RL training, stimulate the reflective capabilities of the prover model.

CritiqueFineTuning. Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate" [COLM 2025]

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

FlashMLA. FlashMLA: Efficient Multi-head Latent Attention Kernels

simple_GRPO. A very simple GRPO implement for reproducing r1-like LLM thinking.

Goedel-Prover. Python

DeepSeek-R1.

LoRA. Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

flux. Official inference repo for FLUX.1 models

Lora-for-Diffusers. The most easy-to-understand tutorial for using LoRA (Low-Rank Adaptation) within diffusers framework for AI Generation Researchers🔥

OneDiffusion. Official implementation of OneDiffusion paper (CVPR 2025)

VAR. [NeurIPS 2024 Best Paper Award][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

HunyuanVideo. HunyuanVideo: A Systematic Framework For Large Video Generation Model

LLMs-from-scratch. Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

PerVFI. Official code base of "Perception-Oriented Video Frame Interpolation via Asymmetric Blending" (CVPR 2024), also denoted as ''PerVFI''.

InternGPT. InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

consistency_models. Official repo for consistency models.

ZoeDepth. Metric depth estimation from a single image

neural-light-transport. Neural Light Transport for Relighting and View Synthesis

Auto-Photoshop-StableDiffusion-Plugin. A user-friendly plug-in that makes it easy to generate stable diffusion images inside Photoshop using either Automatic or ComfyUI as a backend.

stable-diffusion-webui. Stable Diffusion web UI

MobileStyleGAN.pytorch. An official implementation of MobileStyleGAN in PyTorch

stable-diffusion. A latent text-to-image diffusion model

textual_inversion. Jupyter Notebook

SelfTraining-6D. Python

flexconv. Code repository for the ICLR 2022 paper "FlexConv: Continuous Kernel Convolutions With Differentiable Kernel Sizes" https://openreview.net/forum?id=3jooF27-0Wy

3dvideogeneration. 3D-Aware Video Generation

ARF-svox2. Artistic Radiance Fields

PandA. [ICLR'23] Code to reproduce the results in the paper "PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs"

DDQ. (CVPR2023)Dense Distinct Query for End-to-End Object Detection

DALLE2-pytorch. Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

DE-Net. [AAAI 2023] Dynamic Text-guided Image Editing Adversarial Networks

MJP. [CVPR 2023] An official Pytorch implementation of "Masked Jigsaw Puzzle: A Versatile Position Embedding for Vision Transformers".

CogVideo. text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

imagen-pytorch. Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

vit-spurious-robustness. Python

stylegan-mmuit. [TMM 2022] ISF-GAN.

CoFiPruning. [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408

E2EVE. Official repository for the paper: "End-to-End Visual Editing with a Generatively Pre-Trained Artist"

MAGIC. Language Models Can See: Plugging Visual Controls in Text Generation

FastDifferentiableMatSqrt. ICLR22 "Fast Differentiable Matrix Square Root" and T-PAMI extension

how-do-vits-work. (ICLR 2022 Spotlight) Official PyTorch implementation of "How Do Vision Transformers Work?"

PPLM. Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.

SongNet. Code for ACL 2020 paper "Rigid Formats Controlled Text Generation"：https://www.aclweb.org/anthology/2020.acl-main.68/

TaCL. [NAACL'22] TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning

VQ-Diffusion. Python

ibot. iBOT :robot:: Image BERT Pre-Training with Online Tokenizer (ICLR 2022)

ChildPredictor. ChildPredictor: A Child Face Prediction Framework with Disentangled Learning. IEEE TMM, 2022

ExpressiveFaceVideoEncoding. The official repository of "Encode-in-Style: Latent-based Video Encoding using StyleGAN2"

RandSAC.

DualStyleGAN. [CVPR 2022] Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer

TokenCut. (CVPR 2022) Pytorch implementation of "Self-supervised transformers for unsupervised object discovery using normalized cut"

PTI. Official Implementation for "Pivotal Tuning for Latent-based editing of Real Images" (ACM TOG 2022) https://arxiv.org/abs/2106.05744

Hire-Wave-MLP.pytorch. Implementation of Hire-MLP: Vision MLP via Hierarchical Rearrangement and An Image Patch is a Wave: Phase-Aware Vision MLP.

antialiased-cnns. pip install antialiased-cnns to improve stability and accuracy

webpage-template. Simple project webpage template. Originally used in Colorful Image Colorization. ECCV, 2016.

SimCTG. [NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation

MAE-pytorch. Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

e2e_lfmmi. E2E system with LF-MMI; word N-gram for Mandarin

VTs-Drloc. [NeurIPS 2021] Official codes for "Efficient Training of Visual Transformers with Small Datasets".

JoJoGAN. Official PyTorch repo for JoJoGAN: One Shot Face Stylization

dqd. A python implementation of differentiable quality diversity.

DynamicViT. [NeurIPS 2021] [T-PAMI] DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification

MeTeoR. Python

RepVGG. RepVGG: Making VGG-style ConvNets Great Again

d2c. PyTorch implementation of D2C: Diffuison-Decoding Models for Few-shot Conditional Generation.

Image-Super-Resolution-via-Iterative-Refinement. Unofficial implementation of Image Super-Resolution via Iterative Refinement by Pytorch

RNW. This is the official repository for Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark (ICCV 2021).

stylegan3. Official PyTorch implementation of StyleGAN3

Robust-Differentiable-SVD. Python

C2-Matching. Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.

MASA-SR. MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution (CVPR2021)

Reference-Based-Sketch-Image-Colorization-ImageNet. PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

liif. Learning Continuous Image Representation with Local Implicit Image Function, in CVPR 2021 (Oral)

Oscar. Oscar and VinVL

C2M. [ICCV 2021] Click to Move: Controlling Video Generation with Sparse Motion

awesome-vision-language-pretraining-papers. Recent Advances in Vision and Language PreTrained Models (VL-PTMs)

pixelsynth. [ICCV 2021] PixelSynth: Generating a 3D-Consistent Experience from a Single Image

GANSketching. Sketch Your Own GAN: Customizing a GAN model with hand-drawn sketches.

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

GIFT. Code for "GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNs" NeurIPS 2019

NeuRay. [CVPR2022] Neural Rays for Occlusion-aware Image-based Rendering

CvT. This is an official implementation of CvT: Introducing Convolutions to Vision Transformers.

VideoSuperResolution. A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.

Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image/Video Restoration.

stylegan2-ada-pytorch. StyleGAN2-ADA - Official PyTorch implementation

sam. SAM: Sharpness-Aware Minimization (PyTorch)

CoCosNet-v2. CoCosNet v2: Full-Resolution Correspondence Learning for Image Translation

esvit. EsViT: Efficient self-supervised Vision Transformers

MaskFormer. Per-Pixel Classification is Not All You Need for Semantic Segmentation (NeurIPS 2021, spotlight)

EGVSR. Efficient & Generic Video Super-Resolution

anycost-gan. [CVPR 2021] Anycost GANs for Interactive Image Synthesis and Editing

data-efficient-gans. [NeurIPS 2020] Differentiable Augmentation for Data-Efficient GAN Training

gan-ensembling. Invert and perturb GAN images for test-time ensembling

editnerf. Editing a Conditional Radiance Field

DSNeRF. Code release for DS-NeRF (Depth-supervised Neural Radiance Fields)

DeepI2I. Image-to-image translation, knowledge transfer

Pretrained-IPT. Python

PyTorch-BYOL. PyTorch implementation of Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning

Video-Swin-Transformer. This is an official implementation for "Video Swin Transformers".

alias-free-gan. Alias-Free GAN project website and code

MM-CelebA-HQ-Dataset. [CVPR 2021] Multi-Modal-CelebA-HQ: A Large-Scale Text-Driven Face Generation and Understanding Dataset

3DFaceReconstruction-LAP. The code of paper 'Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection'

webvid. Large-scale text-video dataset. 10 million captioned short videos.

stylegan2-clip-approach. Navigating StyleGAN2 w latent space using CLIP

GazeAnimation. Give a portrait face, move the gaze up (ACM MM 2020)

self-supervised. Whitening for Self-Supervised Representation Learning | Official repository

clean-fid. PyTorch - FID calculation with proper image resizing and quantization steps [CVPR 2022]

ASLNet. Python

solo-learn. solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning

TransDA. Official pytorch implement for “Transformer-Based Source-Free Domain Adaptation”

copyisallyouneed. Code for our ACL2021 paper Neural Machine Translation with Monolingual Translation Memory

mmagic. OpenMMLab Multimodal Advanced, Generative, and Intelligent Creation Toolbox. Unlock the magic 🪄: Generative-AI (AIGC), easy-to-use APIs, awsome model zoo, diffusion models, for text-to-image generation, image/video restoration/enhancement, etc.

Transformer-SSL. This is an official implementation for "Self-Supervised Learning with Swin Transformers".

stylegan2-pytorch. Implementation of Analyzing and Improving the Image Quality of StyleGAN (StyleGAN 2) in PyTorch

genforce. An efficient PyTorch library for deep generative modeling.

GLEAN.

TtT. code for ACL2021 paper "Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction"

Twins. Two simple and effective designs of vision transformer, which is on par with the Swin transformer

frechet-bert-distance. Findings of ACL 2021

dino. PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

mlp-mixer-pytorch. An All-MLP solution for Vision, from Google AI

SmoothingLatentSpace. [CVPR 2021] Smoothing the Disentangled Latent Style Space for Unsupervised I2I Translation

Swin-Transformer. This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".

VISTA-Net. The code release for "Variational Structured Attention Networks for Visual Dense Representation Learning"

PyTorch-Encoding. A CV toolkit for my papers.

TransDepth. Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

SGINet. Code for the paper Semantic-Guided Inpainting Network for Complex UrbanScenes Manipulation

PVT. Official implementation of PVT series

TTSR. [CVPR'20] TTSR: Learning Texture Transformer Network for Image Super-Resolution

SinGAN. Official pytorch implementation of the paper: "SinGAN: Learning a Generative Model from a Single Natural Image"

DeepSegmentor. DeepCrack + Colab [DeepSegmentor] Crack and Road detection based on IA web app

crack-detector. Web app powered by AI to detect cracks on any surface. CHECK LICENSES!

image-super-resolution. 🔎 Super-scale your images and run experiments with Residual Dense and Adversarial Networks.

DALLE-pytorch. Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

RG-UNIT. RG-UNIT, ACM MM 2020.

detr. End-to-End Object Detection with Transformers

AutoDL-Projects. Automated deep learning algorithms implemented in PyTorch.

google-research. Google Research

SegVAE. Implementation of ECCV 2020 paper: "Controllable Image Synthesis via SegVAE". Project page: https://yccyenchicheng.github.io/SegVAE/. Paper: https://arxiv.org/abs/2007.08397.

DCL. NeurIPS 2020, Debiased Contrastive Learning

Pluralistic-Inpainting. [CVPR 2019]: Pluralistic Image Completion

generative_inpainting. DeepFill v1/v2 with Contextual Attention and Gated Convolution, CVPR 2018, and ICCV 2019 Oral

interfacegan. [CVPR 2020] Interpreting the Latent Space of GANs for Semantic Face Editing

bhtsne. Barnes-Hut t-SNE

facenet. Face recognition using Tensorflow

ALAE. [CVPR2020] Adversarial Latent Autoencoders

AMR-gs. AMR Parsing via Graph-Sequence Iterative Inference

LGGAN. [CVPR 2020] Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation

classifier-balancing. This repository contains code for the paper "Decoupling Representation and Classifier for Long-Tailed Recognition", published at ICLR 2020

stargan. StarGAN - Official PyTorch Implementation

RankIQA. The rep for the RankIQA paper in ICCV 2017

mentornet. Code for MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks

stylegan2. StyleGAN2 - Official TensorFlow Implementation

GAN_stability. Code for paper "Which Training Methods for GANs do actually Converge? (ICML 2018)"

sentencepiece. Unsupervised text tokenizer for Neural Network-based text generation.

gpt-2. Code for the paper "Language Models are Unsupervised Multitask Learners"

Attention-Gated-Networks. Use of Attention Gates in a Convolutional Neural Network / Medical Image Classification and Segmentation

celeba-hq-modified. Modified h5tool.py make user getting celeba-HQ easier

DSRG. Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing (CVPR 2018).

pytorch-LearnedRandomWalker. Pytorch implementation of the LearnedRandomWalker module

ReDO. Code for paper Unsupervised Object Segmentation by Redrawing

stylegan. StyleGAN - Official TensorFlow Implementation

PyTorch-GAN. PyTorch implementations of Generative Adversarial Networks.

L-GM-loss. Implementation of our accepted CVPR 2018 paper "Rethinking Feature Distribution for Loss Functions in Image Classification"

lsoftmax-pytorch. The Pytorch Implementation of L-Softmax

pavement-crack-detection. C++

dvc.org. 📖 DVC documentation website

pconf. [NeurIPS 2018] Code for the paper "Binary Classification from Positive-Confidence Data"

DRIT. Learning diverse image-to-image translation from unpaired data

python-torchfile. Deserialize Lua torch-serialized objects from Python

H-DenseUNet. TMI 2018. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes

fairseq. Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

pose-gan. Python

lveg. Gaussian Mixture Latent Vector Grammars

FCOS. FCOS: Fully Convolutional One-Stage Object Detection (ICCV'19)

Pose-Transfer. Code for the paper Progressive Pose Attention for Person Image Generation in CVPR19 (Oral).

thyme. math/cs publications I find useful

MUNIT. Multimodal Unsupervised Image-to-Image Translation

PerceptualSimilarity. LPIPS metric. pip install lpips