shihao wang

locate-anything.cpp. Port of Nvidia LocateAnything-3B on ggml

InfiniDepth. [CVPR 2026] InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields

TVEdit.

DepthMaster. DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images

TorchUMM. A unified multimodal model toolkit

WildDet3D. Allen Institute for AI: WildDet3D: Scaling Promptable 3D Detection in the Wild

Awesome-WAM. A curated, continuously updated reading list, paper blogs, and resources for World Action Models (WAMs) in embodied AI.

dexbotic. Dexbotic: Open-Source Vision-Language-Action Toolbox

py123d. 123D: A Unified Library for Multi-Modal Autonomous Driving Data

robometer. Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons

SenseNova-U1. SenseNova-U series: Native Unified Paradigm with NEO-unify from the First Principles

Memorize-When-Needed. Python

Recursive-Multimodal-Agent. Python

starVLA. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing

lingbot-va. [RSS 2026] Causal video-action world model for generalist robot control

VOSR. [CVPR2026] VOSR: A Vision-Only Generative Model for Image Super-Resolution

Awesome-VLM-Streaming-Video. 📚 A curated collection of papers and open-source code repositories dedicated to the application of Vision-Language Models (VLMs) for streaming video.

V-CAST. V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models

GDPO. Official code for GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution

MICo-150K. Official repository for the paper "MICo-150K: A Comprehensive Dataset for Multi-Image Composition".

drivelaw. [CVPR2026] DriveLaW: Unifying Planning and Video Generation in a Latent Driving World

awesome-embodied-vla-va-vln. A curated list of state-of-the-art research in embodied AI, focusing on vision-language-action (VLA) models, vision-language navigation (VLN), and related multimodal learning approaches.

Self-Transcendence. [ECCV 2026] Official code repository for "Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?"

Photo3D. Photo3D: Advancing Photorealistic 3D Generation through Structure‑Aligned Detail Enhancement

MixKV. [ICLR 2026] Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models

Awesome-DLMs. The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".

FedEcho. Official implementation of FedEcho

RLinf. RLinf: Reinforcement Learning Infrastructure for Embodied and Agentic AI

awesome-ai-research-writing. Elevate your AI research writing, no more tedious polishing ✨

VAGEN. World model reasoning RL for multi-turn VLM agents

CoCoEdit. [ICML 2026] Official PyTorch implementation of paper “CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning”

MemSkill. MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents

GroundingSuite. [ICCV 2025] GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding

Embodied-AI-Guide. [Lumina具身智能社区] 具身智能技术指南 Embodied-AI-Guide

MediaCrawler. 小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫

MagiAttention. A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

WeDLM. WeDLM: The fastest diffusion language model with standard causal attention and native KV cache compatibility, delivering real speedups over vLLM-optimized baselines.

Seed-1.8. Jupyter Notebook

GroundCUA. GroundCUA

ShowUI. [CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.

Multi-Crit. Python

ViP3DEdit. [AAAI26] ViP3DE: Fast Multi-view Consistent 3D Editing with Video Priors

Hydra-NeXt. Python

Awesome-Spatial-Intelligence-in-VLM. A paper list for spatial reasoning

Awesome-Spatial-VLMs. [Awesome-Spatial-VLMs] This repository is the official, community-maintained resource for the survey paper: Spatial Intelligence in Vision-Language Models: A Comprehensive Survey;

LocateAnything3D.

BEVDilation. [AAAI'26] BEVDilation: LiDAR-Centric Multi-Modal Fusion for 3D Object Detection

trl. Train transformer language models with reinforcement learning.

VideoNSA. VideoNSA: Native Sparse Attention Scales Video Understanding

MiniVeo3-Reasoner. Thinking with Videos from Open-Source Priors. We reproduce chain-of-frames visual reasoning by fine-tuning open-source video models. Give it a star 🌟 if you find it useful.

Rex-Omni. [CVPR2026] Detect Anything via Next Point Prediction

DP2O-SR. [NeurIPS 2025] DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution

awesome-described-object-detection. A curated list of papers and resources related to Described Object Detection, Open-Vocabulary/Open-World Object Detection and Referring Expression Comprehension. Updated frequently and pull requests welcomed.

SDLM. Sequential Diffusion Language Model (SDLM) enhances pre-trained autoregressive language models by adaptively determining generation length and maintaining KV-cache compatibility, achieving high efficiency and throughput.

NSARM. NSARM: Next-Scale Autoregressive Modeling for Robust Real-World Image Super-Resolution

OpenCUA. [NeurIPS 2025 Spotlight] OpenCUA: Open Foundations for Computer-Use Agents

DNAEdit_code. [NeurIPS 2025 Spotlight] Official implementation for DNAEdit: Direct Noise Alignment for Text-Guided Rectified Flow Editing

flash-linear-attention. 🚀 Efficient implementations for emerging model architectures

OmniParser. A simple screen parsing tool towards pure vision based GUI agent

SoM. [arXiv 2023] Set-of-Mark Prompting for GPT-4V and LMMs

GPSToken. The official code for paper "GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation"

Ultra-Sparse-Memory-Network. Python

Compositional-Visual-Reasoning-Survey. Explain Before You Answer: A Survey on Compositional Visual Reasoning

FiVE-Bench. [ICCV 2025] FiVE-Bench: A Fine-grained Video Editing Benchmark for Evaluating Emerging Diffusion and Rectified Flow Models

arxiv_daily_aigc. An AI-driven daily arXiv paper crawler, analyzer, and organizer tool, focusing on AIGC

FedRD. Official implementation of FedRD

awesome-grounding. awesome grounding: A curated list of research papers in visual grounding

AffordanceNet. [ICCV 2025] RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Grasping

RoboSpatial. Python

ERQA. Embodied Reasoning Question Answer (ERQA) Benchmark

PAM. Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videos

MaSS13K. [CVPR 2025] Official code repository for "MaSS13K: A Matting-level Semantic Segmentation Benchmark"

Spatial-Mamba. [ICLR2025] Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion

VideoITG. [CVPR 2026 Highlight] VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding

OpenThinkIMG. OpenThinkIMG is an end-to-end open-source framework that empowers Large Vision-Language Models to think with images.

awesome-foundation-agents. About Awesome things towards foundation agents. Papers / Repos / Blogs / ...

Awesome-video-super-resolution-diffusion. A curated list of resources for video super-resolution using diffusion models.

SeeSR. [CVPR2024] SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution

OSEDiff. [NeurlPS2024] One-Step Effective Diffusion Network for Real-World Image Super-Resolution

Awesome_Think_With_Images. Resources and paper list for "Thinking with Images for LVLMs". This repository accompanies our survey on how LVLMs can leverage visual information for complex reasoning, planning, and generation.

Awesome-Thinking-With-Images. Latest open-source "Thinking with images" (O3/O4-mini) papers, covering training-free, SFT-based, and RL-enhanced methods for "fine-grained visual understanding".

DeepEyes. Python

MMaDA. MMaDA - Open-Sourced Multimodal Large Diffusion Language Models (dLLMs with block diffusion, mixed-CoT, unified RL)

Seed1.5-VL. Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning, achieving state-of-the-art performance on 38 out of 60 public benchmarks.

Awesome-Controllable-Video-Generation. [ArXiv 2025] A survey about controllable video generation: This repo is the official awesome of "Controllable video generation: A survey"

Slow-Fast-Video-Multimodal-LLM. Python

PR1. [NeurIPS 2025] Official code implementation of Perception R1: Pioneering Perception Policy with Reinforcement Learning

Search-R1. Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

multimodal_r1_papers. Deepseek RL (GRPO)-Inspired Research for Vision & Multimodal Reasoning

RoboVerse. RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning

awesome-VLLMs. This repository collects papers on VLLM applications. We will update new papers irregularly.

InsViE. Official PyTorch implementation of paper “InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Construction”

DAPO. An Open-source RL System from ByteDance Seed and Tsinghua AIR

TriplaneTurbo. [CVPR2025] Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data

EasyR1. EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL

CG-Bench. Python

DeepSeek-R1.

Diffusion4D. [NeurIPS 2024] Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models

finetrainers. Scalable and memory-optimized training of diffusion models

Koala-36M. Official implementation of the paper "Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content".

mochi. The best OSS video generation models, created by Genmo

Video-XL. 🔥🔥First-ever hour scale video understanding models

Pyramid-Flow. [ICLR 2025] Pyramidal Flow Matching for Efficient Video Generative Modeling

EasyAnimate. 📺 An End-to-End Solution for High-Resolution and Long Video Generation Based on Transformer Diffusion

Emu3. Next-Token Prediction is All You Need

all-seeing. [ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"

GOT-OCR2.0. Official code implementation of General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

Oryx. [ICLR 2025] MLLM for On-Demand Spatial-Temporal Understanding at Arbitrary Resolution

Qwen3-VL. Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

IDKB. Official repository for paper "Can LVLMs Obtain a Driver’s License? A Benchmark Towards Reliable AGI for Autonomous Driving"

C2ANet. Python

Eagle. Eagle: Frontier Vision-Language Models with Data-Centric Strategies

TimeChat. [CVPR 2024] TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

Awesome-LLMs-for-Video-Understanding. 🔥🔥🔥 [IEEE TCSVT] Latest Papers, Codes and Datasets on Vid-LLMs.

VITA. ✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Flash-VStream. This is the official implementation of ICCV 2025 "Flash-VStream: Efficient Real-Time Understanding for Long Video Streams"

Video-LLaVA. 【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

PLLaVA. Official repository for the paper PLLaVA

videollm-online. VideoLLM-online: Online Video Large Language Model for Streaming Video (CVPR 2024)

ShareGPT4Video. [NeurIPS 2024] An official implementation of "ShareGPT4Video: Improving Video Understanding and Generation with Better Captions"

open_clip. An open source implementation of CLIP.

LCL. [NeurIPS 2024] Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning

HA-DPO. Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization

SeVa. [MM2024, oral] "Self-Supervised Visual Preference Alignment" https://arxiv.org/abs/2404.10501

anole. [Extended verision ICLR 2025 Blog Track] Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

HPHS. [IROS 2024] HPHS: Hierarchical Planning based on Hybrid Frontier Sampling for Unknown Environments Exploration

chameleon. Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.

Awesome-Diffusion-for-Image-Translation. A collection of papers on Diffusion for Image-to-Image Translation and Style Transfer

awesome-image-translation. Collection of awesome resources on image-to-image translation.

LLM101n. LLM101n: Let's build a Storyteller

TinyLLaVA_Factory. A Framework of Small-scale Large Multimodal Models

wayve_scenes. Codebase for the WayveScenes101 Dataset

DataOptim. A collection of visual instruction tuning datasets.

Delphi. Official Code Release of Delphi

LLaVA-pp. 🔥🔥 LLaVA++: Extending LLaVA with Phi-3 and LLaMA-3 (LLaVA LLaMA-3, LLaVA Phi-3)

ADA-Track. Offical implementation of CVPR2024 paper ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association.

Awesome-LLMs-meet-Multimodal-Generation. 🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

Awesome-Multimodal-Large-Language-Models. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models

GAM-Depth. GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints

VAR. [NeurIPS 2024 Best Paper Award][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

LLaVA-NeXT. Python

OmniDrive. Python

MGM. Official repo for "Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models"

RayDN. [ECCV 2024] Ray Denoising (RayDN): Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection

sledge. [ECCV'24] SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic

Open-Sora-Plan. This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

WidthFormer. [IROS 2024 Oral Presentation] WidthFormer: Toward Efficient Transformer-based BEV View Transformation

Depth-Anything. [CVPR 2024] Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation

PillarNeSt. The Official Implementation of PillarNeSt

Sparse4D. Jupyter Notebook

NuInstruct.

llm-reasoners. A library for advanced large language model reasoning

LingoQA. [ECCV 2024] Official GitHub repository for the paper "LingoQA: Visual Question Answering for Autonomous Driving"

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

BEV-Planner. Python

Awesome-DriveLM. 📚 A collection of resources and papers on Large Language Models in autonomous driving

OccWorld. [ECCV 2024] 3D World Model for Autonomous Driving

MiniGPT-4. Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

Occ4cast. Occ4cast: LiDAR-based 4D Occupancy Completion and Forecasting

transformers. 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

GPT4V-AD-Exploration. On the Road with GPT-4V(ision): Explorations of Utilizing Visual-Language Model as Autonomous Driving Agent

Far3D. [AAAI2024] Far3D: Expanding the Horizon for Surround-view 3D Object Detection

Awesome-LLM4AD. A curated list of awesome LLM/VLM/VLA/World Model for Autonomous Driving(LLM4AD) resources (continually updated)

StateTransformer. Python

AgentTuning. AgentTuning: Enabling Generalized Agent Abilities for LLMs

waymax. A JAX-based simulator for autonomous driving research.

GPT-Driver. Learning to Drive with GPT

Driving-with-LLMs. PyTorch implementation for the paper "Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving"

DriveLM. [ECCV 2024 Oral] DriveLM: Driving with Graph Visual Question Answering

LongLoRA. Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

World-Model. A paper list of world model

DQTrack. Official PyTorch implementation of End-to-end 3D Tracking with Decoupled Queries [ICCV 2023]

DriveDreamer. [ECCV 2024] DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving

StreamMapNet. Python

CoDeF. [CVPR'24 Highlight] Official PyTorch implementation of CoDeF: Content Deformation Fields for Temporally Consistent Video Processing

MV2D. Code for "Object as Query: Lifting any 2D Object Detector to 3D Detection"

Awesome-Learn-3D-From-2D. A collection of resources on learning 3D object from 2D images.

stream3dppe. StreamPETR with 3dppe Extension

mars. MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving

RoMe. Python

DOS. [CVPR 2023] ReasonNet: End-to-End Driving with Temporal and Global Reasoning

Emu. Emu Series: Generative Multimodal Models from BAAI

plant. [CoRL'22] PlanT: Explainable Planning Transformers via Object-Level Representations

GPT4RoI. (ECCVW 2025)GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest

Voyager. An Open-Ended Embodied Agent with Large Language Models

S-NeRF. [ICLR 2023 & TPAMI 2025] S-NeRF: Neural Radiance Fields for Street Views

3DTrans. An open-source codebase for exploring autonomous driving pre-training

carla_garage. [ICCV'23] Hidden Biases of End-to-End Driving Models & A starter kit for the CARLA leaderboard 2.0.

tuplan_garage. [CoRL'23] Parting with Misconceptions about Learning-based Vehicle Motion Planning

xmuda. Cross-Modal Unsupervised Domain Adaptationfor 3D Semantic Segmentation

ChatAny. 🌻 一键拥有你自己的 ChatGPT+众多AI 网页服务 | One click access to your own ChatGPT+Many AI web services

omni3d. Code release for "Omni3D A Large Benchmark and Model for 3D Object Detection in the Wild"

vMAP. [CVPR 2023] vMAP: Vectorised Object Mapping for Neural Field SLAM

EFG. An Efficient, Flexible, and General deep learning framework that retains minimal.

mile. PyTorch code for the paper "Model-Based Imitation Learning for Urban Driving".

PythonRobotics. Python sample codes and textbook for robotics algorithms.

ThinkTwice. [CVPR 2023] Pytorch implementation of ThinkTwice, a SOTA Decoder for End-to-end Autonomous Driving under BEV.

suds. Scalable Urban Dynamic Scenes

awesome-end-to-end-autonomous-driving. A curated list of awesome End-to-End Autonomous Driving resources (continually updated)

Online_Map_Construction_Benchmark. A benchmark for online map construction task.