Weihao Yu

awesome-world-action-models.

Q-ARVD. Python

Mix-Quant. Python

Video-Metaphorical-Understanding. [arxiv] ViMU: Benchmarking Video Metaphorical Understanding

AnyFlow. Flow Map OPD for AnyStep Video Diffusion

Evolving-Visual-Generation. [Roadmap] Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling

Awesome-VLA-Safety. [Arxiv] Vision-Language-Action Safety: Threats, Challenges, Evaluations, and Mechanisms

DMax. DMax: Aggressive Parallel Decoding for dLLMs

Video-MME-v2. Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

GeoSR. The code for paper 'Make Geometry Matter for Spatial Reasoning'

ViFeEdit. [Preprint] ViFeEdit: A Video-Free Tuner of Your Video Diffusion Transformer

NoLan. Python

MHLA. [ICLR 2026🔥] MHLA: Restoring Expressivity of Linear Attention via Token-Level Multi-Head

SpotEdit. SpotEdit:Selective Region Editing in Diffusion Transformers

WorldWarp. Long-range camera-conditioned scene generation from one single image.

Mooncake. Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Agent-Memory-Paper-List. The paper list of "Memory in the Age of AI Agents: A Survey"

Awesome-RL-for-LRMs. A Survey of Reinforcement Learning for Large Reasoning Models

Awesome-Agent-Scientists. Paper list of agent for science

streaming-vlm. StreamingVLM: Real-Time Understanding for Infinite Video Streams

Awesome-Self-Evolving-Agents. [Survey] A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems

In-Video-Instructions. [Arxiv 2025] In-Video Instructions: Visual Signals as Generative Control

Awesome-Agentic-MLLMs. Agentic MLLMs

A-mem. The code for NeurIPS 2025 paper "A-Mem: Agentic Memory for LLM Agents"

awesome-large-multimodal-agents.

Awesome-LLM-Long-Context-Modeling. 📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

awesome-ai-for-science. A curated list of awesome AI tools, libraries, papers, datasets, and frameworks that accelerate scientific discovery — from physics and chemistry to biology, materials, and beyond.

trl. Train transformer language models with reinforcement learning.

Protein-LLM-Survey. Large Language Models in Protein: A Comprehensive Survey

MIRIX. Mirix is a multi-agent personal assistant designed to track on-screen activities and answer user questions intelligently. By capturing real-time visual data and consolidating it into structured memories, Mirix transforms raw inputs into a rich knowledge base that adapts to your digital experiences.

TraceAnything. [ICLR 2026] Trace Anything: Representing Any Video in 4D via Trajectory Fields

OmniVerifier. [ICLR 2026 Oral & ICML 2026] Generative Universal Verifier as Multimodal Meta-Reasoner

AHN. AHN: Artificial Hippocampus Networks for Efficient Long-Context Modeling

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

A-Comprehensive-Survey-For-Long-Context-Language-Modeling. A Comprehensive Survey on Long Context Language Modeling

Awesome-RL-for-Multimodal-Foundation-Models. 📖 This is a repository for organizing papers, codes and other resources related to Visual Reinforcement Learning.

VeOmni. VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

tiktoken. tiktoken is a fast BPE tokeniser for use with OpenAI's models.

harmony. Renderer for the harmony response format to be used with gpt-oss

gpt-oss. gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

IEAP. [NeurIPS 2025] IEAP: Image Editing As Programs with Diffusion Models

DLLM-Survey. [Arxiv] Discrete Diffusion in Large Language and Multimodal Models: A Survey

gpt-fast. Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

torchtune. PyTorch native post-training library

UniWorld. UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

VeriThinker. [NeurIPS 2025] VeriThinker: Learning to Verify Makes Reasoning Model Efficient

Dimple. Dimple, the first Discrete Diffusion Multimodal Large Language Model

Thinkless. [NeurIPS 2025] Thinkless: LLM Learns When to Think

Magma. [CVPR 2025] Magma: A Foundation Model for Multimodal AI Agents

flaxformer. Python

DeepSeek-V3. Python

unsloth. Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

CLEAR. [NeurIPS 2025] Official PyTorch implementation of paper "CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up".

DragAnything. [ECCV 2024] DragAnything: Motion Control for Anything using Entity Representation

genesis-world. Simulation platform for general-purpose robotics & embodied AI learning.

R-MeeTo. Give us minutes, we give back a faster Mamba. The official implementation of "Faster Vision Mamba is Rebuilt in Minutes via Merged Token Re-training".

SMPLer. "SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation", TPAMI 2024

VILA. VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.

CVPR2025-Florence-VL. Python

HunyuanVideo. HunyuanVideo: A Systematic Framework For Large Video Generation Model

MovieBench. [CVPR 2025] A Hierarchical Movie Level Dataset for Long Video Generation

ROICtrl. Code for [CVPR 2025] ROICtrl: Boosting Instance Control for Visual Generation

OminiControl. [ICCV 2025 Highlight] OminiControl: Minimal and Universal Control for Diffusion Transformer

apiprompting. [ECCV 2024] API: Attention Prompting on Image for Large Vision-Language Models

LightningDrag. Experiencing lightning fast (~1s) and accurate drag-based image editing

ai-for-grant-writing. A curated list of resources for using LLMs to develop more competitive grant applications.

kat. [ICLR2025] Kolmogorov-Arnold Transformer

LinFusion. Official PyTorch and Diffusers Implementation of "LinFusion: 1 GPU, 1 Minute, 16K Image"

sglang. SGLang is a high-performance serving framework for large language models and multimodal models.

LiteFocus. [Interspeech 2024] LiteFocus is a tool designed to accelerate diffusion-based TTA model, now implemented with the base model AudioLDM2.

KANbeFair. A More Fair and Comprehensive Comparison between KAN and MLP

vico. Vico: Compositional Video Generation as Flow Equalization

megalodon. Reference implementation of Megalodon 7B model

MInference. [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

MiCo. [ICCV 2025] Explore the Limits of Omni-modal Pretraining at Scale

opencompass. OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

VLMEvalKit. Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

visual_anagrams. Code for the paper "Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models"

NeuralLineage. Code for CVPR 2024 Oral "Neural Lineage"

lmms-eval. One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

CVinW_Readings. A collection of papers on the topic of ``Computer Vision in the Wild (CVinW)''

LlamaGen. Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

learning-to-cache. [NeurIPS 2024] Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching

FinRobot. FinRobot: An Open-Source AI Agent Platform for Financial Analysis using LLMs 🚀 🚀 🚀

efficient-kan. An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

MoRF. Receptive field as experts

MLLA. [NeurIPS 2024] Official repository of MLLA

Mamba_State_Space_Model_Paper_List. [Mamba-Survey-2024] Paper list for State-Space-Model/Mamba and it's Applications

anything-unsegmentable. (CVPR 2024) "Unsegment Anything by Simulating Deformation"

Apollo. Multilingual Medicine: Model, Dataset, Benchmark, Code

iLLaMA. Adapting LLaMA Decoder to Vision Transformer

2d-gaussian-splatting. [SIGGRAPH'24] 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Interactive-CD-tool.

pytorch-image-models. The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

TransNeXt. [CVPR 2024] Code release for TransNeXt model

sailor-llm. [EMNLP-2024] ⚓️ Sailor: Open Language Models for South-East Asia

cosmo. Python

DeepCache. [CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

OFA. Official repository of OFA (ICML 2022). Paper: OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

UltraFastBERT. The repository for the code of the UltraFastBERT paper

DragDiffusion. [CVPR2024, Highlight] Official code for DragDiffusion

Big-Interleaved-Dataset. Big-Interleaved-Dataset

GPT4Tools. GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the user to interact with images during a conversation.

LLaVA-Plus-Codebase. LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

OBELICS. Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

awesome-test-time-adaptation. Collection of awesome test-time (domain/batch/instance) adaptation methods

MetaCLIP. NeurIPS 2025 Spotlight; ICLR2024 Spotlight; CVPR 2024; EMNLP 2024

coyo-dataset. COYO-700M: Large-scale Image-Text Pair Dataset

object-centric-learning-framework. Python

awesome-ml-fairness. Papers and online resources related to machine learning fairness

InternGPT. InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

LLM-Pruner. [NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

open_llama. OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

CLIP_benchmark. CLIP-like model evaluation

Multimodal-GPT. Multimodal-GPT

symbolic-instruction-tuning. The official repository for the paper "From Zero to Hero: Examining the Power of Symbolic Tasks in Instruction Tuning".

Understanding-CoT. Jupyter Notebook

Image2Paragraph. [Image 2 Text Para] Transform Image into Unique Paragraph with ChatGPT, BLIP2, OFA, GRIT, Segment Anything, ControlNet.

Paint-Anything. An interactive demo based on Segment-Anything for stroke-based painting which enables human-like painting.

Inpaint-Anything. Inpaint anything using Segment Anything and inpainting models.

Awesome-Anything. General AI methods for Anything: AnyObject, AnyGeneration, AnyModel, AnyTask, AnyX

EditAnything. Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)

stable-dreamfusion. Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.

Adan. Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

SSF. [NeurIPS'22] This is an official implementation for "Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning".

point2seq. Python

CwD. Official Implementation of CVPR 2022 paper: "Mimicking the Oracle: An Initial Phase Decorrelation Approach for Class Incremental Learning"

awesome-NeRF. A curated list of awesome neural radiance fields papers

PVT. Official implementation of PVT series

OBST. Your fruity companion for transformers

mvp. NeurIPS-2021: Direct Multi-view Multi-person 3D Human Pose Estimation

recovering-unbiased-scene-graphs. Official implementation of "Recovering the Unbiased Scene Graphs from the Biased Ones" (ACMMM 2021)

volo. VOLO: Vision Outlooker for Visual Recognition

VisionPermutator. MLP-Like Vision Permutator for Visual Recognition (PyTorch)

NES. code for the paper Neural Epitome Search for for architecture-agnostic compression

LReasoner. The code of Paper "Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text".

BLIP. Official Implementation of CVPR2021 paper: Continual Learning via Bit-Level Information Preserving

TokenLabeling. Pytorch implementation of "All Tokens Matter: Token Labeling for Training Better Vision Transformers"

CoordAttention. Code for our CVPR2021 paper coordinate attention

T2T-ViT. ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

ConvBert. Python

SHOT. code released for our ICML 2020 paper "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation"

BERT-related-papers. BERT-related papers

awesome-machine-learning-interpretability. A curated list of awesome responsible machine learning resources.

spacy-transformers. 🛸 Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy

horovod. Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

xlnet. XLNet: Generalized Autoregressive Pretraining for Language Understanding

mmaction. An open-source toolbox for action understanding based on PyTorch

grover. Code for Defending Against Neural Fake News, https://rowanzellers.com/grover/

mmf. A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

hellaswag. HellaSwag: Can a Machine _Really_ Finish Your Sentence?

mmdetection. OpenMMLab Detection Toolbox and Benchmark

OctConv. Code for paper

Papers. The list of some conference papers.

self-critical.pytorch. Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.

r2c. Recognition to Cognition Networks (code for the model in "From Recognition to Cognition: Visual Commonsense Reasoning", CVPR 2019)

state-of-the-art-result-for-machine-learning-problems. This repository provides state of the art (SoTA) results for all machine learning problems. We do our best to keep this repository up to date. If you do find a problem's SoTA result is out of date or missing, please raise this as an issue or submit Google form (with this information: research paper name, dataset, metric, source code and year). We will fix it immediately.

ACMN-Pytorch. Visual Question Reasoning on General Dependency Tree

colah.github.io. HTML