Jiehui Huang

cosmos. NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.

Bernini. Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.

OmniVoice. High-Quality Voice Cloning TTS for 600+ Languages

viitor-voice-nar. Python

UnityShots. This project is the official implementation of "UnityShots: Memory-Driven Multi-Shot Audio-Video Generation with Boundary-Aware Gating"

InterleaveThinker. Python

WoG. [ICML 2026] 🏂 World Guidance: World Modeling in Condition Space for Action Generation

Flash-WAM. Python

AnyFlow. Flow Map OPD for AnyStep Video Diffusion

Gamma-World. Implementation of Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

le-wm. Official code base for LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

Talker-T2AV. ACM MM 2026 Talker-T2AV Joint Talking Audio-Video Generation with Autoregressive Diffusion Modeling

HumanNet. HumanNet: Scaling Human-centric Video Learning to One Million Hours

Awesome-Video-World-Models-with-AR-Diffusion. A Curated List of Awesome Video World Models with AR Diffusion: Covering Algorithms, Applications, and Infrastructure, Aimed at Serving as a Comprehensive Resource for Researchers, Practitioners, and Enthusiasts.

scintegration. single cell integration

Motus. Official code of Motus: A Unified Latent Action World Model

RoboTwin. RoboTwin 2.0 Offical Repo

Sana. SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

FastWAM. Official codebase for Fast-WAM: Do World Action Models Need Test-time Future Imagination?

HY-World-2.0. HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

GR00T-WholeBodyControl. Welcome to GR00T Whole-Body Control (WBC)! This is a unified platform for developing and deploying advanced humanoid controllers. This includes: Decoupled WBC models used in NVIDIA Isaac-Gr00t, Gr00t N1.5 and N1.6 and GEAR-SONIC

VerseCrafter. VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control

CineBrain. Official repository of the CineBrain

VideoMAE. [NeurIPS 2022 Spotlight] VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

scprototransformer. scProtoTransformer: Scalable Reference Mapping Across Molecules, Cells and Donors

Being-H. Being-H is BeingBeyond's family of human-centric embodied foundation models.

openpi. Python

DiffRhythm. Di♪♪Rhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion

ACE-Step. ACE-Step: A Step Towards Music Generation Foundation Model

genmedia-izumi-agent. Python

robocasa. RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots

awesome-agentic-world-modeling. Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond

Research-Paper-Writing-Skills. Skill package for ML/CV/NLP paper writing, curated and adapted from Prof. Peng Sida's open notes for Codex, Claude Code, and Gemini.

learning_research. 本人的科研经验

Lotus-2. Official implementation of "Lotus-2: Advancing Geometric Dense Prediction with Powerful Image Generative Model"

transfusion-pytorch. Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

AI-Research-SKILLs. Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

Gen-Searcher. Gen-Searcher: Reinforcing Agentic Search for Image Generation

CutClaw. Agentic Hours-Long Video Editing via Music Synchronization

StoryMem. Official code for StoryMem: Multi-shot Long Video Storytelling with Memory

dreamzero. Code to pretrain, fine-tune, and evaluate DreamZero and run sim & real-world evals

Isaac-GR00T. NVIDIA Isaac GR00T N1.7 - A Foundation Model for Generalist Robots.

DreamDojo. Official Codebase for "DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos" (ICML 2026)

DIRECT-Claw. A multi-agent framework for automated video mashup creation.

cursor-api-proxy. API proxy for calling endpoints using `agent` cli from Cursor

Unify-Agent. 🐧 Unify-Agent: An end-to-end unified multimodal agent for faithful, knowledge-grounded image generation.

waoowaoo. 首家工业级全流程 AI 影视生产平台。Industry-first professional AI Agent platform for controllable film & video production. From shorts to live-action with Hollywood-standard workflows.

lumenx. LumenX Studio，AI 短漫剧一站式生产平台。它能够将小说文本转化为动态视频，打通了从剧本分析、角色定制、分镜绘制到视频合成的完整创作链路。

huobao-drama. 🎬 火宝短剧 - 基于AI的一站式短剧生成平台 《一句话生成完整短剧，从剧本到成片全自动化》 Huobao Drama - An AI-Powered End-to-End Short Drama Generator "One Sentence to Complete Drama: Fully Automated from Script to Final Video"

IntVideoGameEditor. TypeScript

libtv-skills. Python

Block-Approximate-Sparse-Attention. [CVPR26] Official implementation of CVPR2026 Findings Paper 'Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention'

triattention. TriAttention — Efficient long reasoning with trigonometric KV cache compression. Enables OpenClaw local deployment on memory-constrained GPUs.

alimama-video-narrator. Research code for ACL2024 paper: "Synchronized Video Storytelling: Generating Video Narrations with Structured Storyline"

skills. Public repository for Agent Skills

claude-code. Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

wx4py. 微信4.x自动化机器人：群聊机器人、批量群发（支持文件）、批量设置群公告、聊天记录导出、群管理 | 5分钟上手

ReconVLA. Official implementation of ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver.

RollingForcing. [ICLR 2026] Official Repo for Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

daVinci-MagiHuman. Python

HY-WorldPlay. HY-World 1.5: A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency

AutoResearchClaw. Fully autonomous & self-evolving research from idea to paper. Chat an Idea. Get a Paper. 🦞

LongLive. Long Video Gen Infrastructure

awesome-claude-skills. A curated list of awesome Claude Skills, resources, and tools for customizing Claude AI workflows

ReBalance. [ICLR 2026] Efficient Reasoning with Balanced Thinking

skills. Give your agents the power of the Hugging Face ecosystem

X-Dub. Try X-Dub to sync any character in a video with any audio you like | Official repository for "From Inpainting to Editing: Unlocking Robust Mask-Free Visual Dubbing via Generative Bootstrapping"

ID-LoRA. [ECCV 2026] Generate high resolution videos with a custom voice and appearance, based on LTX-2/LTX-2.3 + Identity In-Context LoRA

Helios. Helios: Real Real-Time Long Video Generation Model

starVLA. StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing

ABot-Manipulation. Python

lingbot-vla. A Pragmatic VLA Foundation Model

OpenHumanVid. [CVPR 2025] A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation

DiariZen. A toolkit for speaker diarization.

JavisDiT. [ICLR 2026] Official implementation of JavisDiT and JavisDiT++ series.

whisperX. WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

genesis-world. Simulation platform for general-purpose robotics & embodied AI learning.

VibeVoice. Open-Source Frontier Voice AI

jonbarron.github.io. HTML

LatentForcing. Python

MiMo-VL. MiMo-VL

VideoREPA. [NeurIPS 2025] VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models

inspiration_tree. Jupyter Notebook

ACE-Step-1.5. The most powerful local music generation model that outperforms almost all commercial alternatives, supporting Mac, AMD, Intel, and CUDA devices.

Bagel. Open-source unified multimodal model

VideoWorld. [CVPR 2025] VideoWorld is a simple generative model that learns purely from unlabeled videos—much like how babies learn by observing their environment.

openclaw. Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

Qwen-Image-Layered. Qwen-Image-Layered: Layered Decomposition for Inherent Editablity

DeepSeek-OCR-2. Visual Causal Flow

lingbot-world. Advancing Open-source World Models

MOVA. MOVA: Towards Scalable and Synchronized Video–Audio Generation

Reasoning-Visual-World. Official repository for "Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models", https://arxiv.org/abs/2601.19834

jepa. PyTorch code and models for V-JEPA self-supervised learning from video.

iREPA. [ICLR 2026] Official implementation for What matters for Representation Alignment: Global Information or Spatial Structure?

Awesome-Unified-Multimodal-Models. 📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Action100M. A Large-scale Video Action Dataset

Search-R1. Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

LiveTalk. Python

LTX-2. Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

LTX-Video. Official repository for LTX-Video

arxiv-latex-cleaner. arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

MMAudio. [CVPR 2025] MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis

oasis. 🏝️ OASIS: Open Agent Social Interaction Simulations with One Million Agents.

Ovi. Python

AutoMV. Python

Gymnasium. A standard API for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)

NitroGen. A Foundation Model for Generalist Gaming Agents

CLD. Code release for: Controllable Layer Decomposition for Reversible Multi-Layer Image Generation

SDAR. SDAR (Synergy of Diffusion and AutoRegression), a large diffusion language model（1.7B, 4B, 8B, 30B）

NEO. NEO Series: Native Vision-Language Models from First Principles

sam-audio. The repository provides code for running inference with the Meta Segment Anything Audio Model (SAM-Audio), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

MemFlow. Official Implementation of "MemFlow: Flowing Adaptive Memory for Consistent and Efficient Long Video Narratives"

BFT. Python

ReCamDriving. Official implementation of "ReCamDriving: LiDAR-Free Camera-Controlled Novel Trajectory Video Generation"

UnityVideo. [CVPR 2026]UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generation

alphaFM-go. Go

diffusion-pipe. A pipeline parallel training script for diffusion models.

HunyuanVideo-1.5. HunyuanVideo-1.5: A leading lightweight video generation model

cambrian-s. Cambrian-S: Towards Spatial Supersensing in Video

Emu3.5. Native Multimodal Models are World Learners

HunyuanWorld-1.0. Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels with Hunyuan3D World Model

DynamicTree. DynamicTree: Interactive Real Tree Animation via Sparse Voxel Spectrum

UniVerse-1-code. The official UniVerse-1 code.

BrickGPT. [ICCV 2025 Best Paper] Official repository for BrickGPT, the first approach for generating physically stable toy brick models from text prompts.

DreamOmni2. This project is the official implementation of 'DreamOmni2: Multimodal Instruction-based Editing and Generation (CVPR2026 Highlight)''

PhysMaster. Official repository of PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learning

LVSM. [ICLR 2025 Oral] Official code for "LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias"

VideoCanvas. Official Code of "VideoCanvas: Unified Video Completion from Arbitrary Spatiotemporal Patches via In-Context Conditioning"

Pyramid-Flow. [ICLR 2025] Pyramidal Flow Matching for Efficient Video Generative Modeling

scKAN. scKAN: Interpretable Single-cell Analysis for Cell-type-specific Gene Discovery and Drug Repurposing via Kolmogorov-Arnold Networks

art-msra. [CVPR 2025] Official repo for ART:Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation

4DNeX. 4DNeX: Feed-Forward 4D Generative Modeling Made Easy

LaVieID. Python

Pandora. Pandora: Towards General World Model with Natural Language Actions and Video States

PhyGenBench. [ICML2025] The code and data of Paper: Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

Qwen-Image. Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing.

Waver. Industry-level video foundation model for unified Text-to-Video (T2V) and Image-to-Video (I2V) generation.

Aether. [ICCV 2025 & ICCV 2025 RIWM Outstanding Paper] Aether: Geometric-Aware Unified World Modeling

Wan2.2. Wan: Open and Advanced Large-Scale Video Generative Models

VisionThink. [NeurIPS 2025] Efficient Reasoning Vision Language Models

Pusa-VidGen. Pusa: Thousands Timesteps Video Diffusion Model

LLaDA. Official PyTorch implementation for "Large Language Diffusion Models"

Awesome-Text-Diffusion-Models. [IJCAI'23] The official Github page of the paper "Diffusion Models for Non-autoregressive Text Generation: A Survey".

modalnext. Repository of "Modal-NexT: toward unified heterogeneous cellular data integration"

VideoX-Fun. 📹 A more flexible framework that can generate videos at any resolution and creates videos from images.

SeePhys. [NeurIPS 2025] Official implementation for the paper "SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics Reasoning"

Wan2.1. Wan: Open and Advanced Large-Scale Video Generative Models

VACE. [ICCV 2025] Official implementations for paper: VACE: All-in-One Video Creation and Editing

HunyuanCustom. HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation

theia. Theia: Distilling Diverse Vision Foundation Models for Robot Learning

LanDiff. The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation

FAR. Code for: "Long-Context Autoregressive Video Modeling with Next-Frame Prediction"

aurora. [ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

VideoAgent. Official implementation of "Self-Improving Video Generation"

ttt-video-dit. Official PyTorch implementation of One-Minute Video Generation with Test-Time Training

FramePack. Lets make video diffusion practical!

SkyReels-V1. SkyReels V1: The first and most advanced open-source human-centric video foundation model

automq. Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.

EMOPortraits. Official implementation of EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars

InfiniteYou. 🔥 [ICCV 2025 Highlight] InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity

textgrad. TextGrad: Automatic ''Differentiation'' via Text -- using large language models to backpropagate textual gradients. Published in Nature.

VideoAlign. [NeurIPS 2025] Improving Video Generation with Human Feedback

Awesome-MCoT. Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

Image-Generation-CoT. [CVPR 2025] The First Investigation of CoT Reasoning (RL, TTS, Reflection) in Image Generation

Qwen3-VL. Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

DeepSeek-VL2. DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

ImageReward. [NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation

align_sd. Better Aligning Text-to-Image Models with Human Preference. ICCV 2023

ddpo. Code for the paper "Training Diffusion Models with Reinforcement Learning"

Video-P2P. Video-P2P: Video Editing with Cross-attention Control

SpeechT5. Unified-Modal Speech-Text Pre-Training for Spoken Language Processing

TeleSpeech-ASR. Python

ComposeAnyone. Config files for my GitHub profile.

RAJNet. Python

ConsistI2V. ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation [TMLR 2024]

ConsistentID. [TPAMI 2026] ConsistentID : Portrait Generation with Multimodal Fine-Grained Identity Preserving

HunyuanVideo. HunyuanVideo: A Systematic Framework For Large Video Generation Model

MagicMirror. [ICCV 2025] MagicMirror: ID-Preserved Video Generation in Video Diffusion Transformers

ReVideo. NeurIPS 2024

Magic-Me. Codes for ID-Specific Video Customized Diffusion

VMix. Official code for VMix: Improving Text-to-Image Diffusion Model with Cross-Attention Mixing Control

CogVideo. text and image to video generation: CogVideoX (2024) and CogVideo (ICLR 2023)

ControlNeXt. Controllable video and image Generation, SVD, Animate Anyone, ControlNet, ControlNeXt, LoRA

ConsisID. [CVPR 2025 Highlight🔥] Identity-Preserving Text-to-Video Generation by Frequency Decomposition

awesome-diffusion-categorized. collection of diffusion model papers categorized by their subareas

IP-Adapter. The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.

InstantID-Rome. InstantID-ROME: Improved Identity-Preserving Generation in Seconds 🔥

StoryMaker. StoryMaker: Towards consistent characters in text-to-image generation

RealisHuman. Code of RealisHuman[AAAI 2025]: A Two-Stage Approach for Refining Malformed Human Parts in Generated Images

Qihoo-T2X. Efficient DiT architecture for text2any tasks, ICLR2025

ResearchProposal. Full PhD Research Proposal

research-proposal. My PhD open research proposal

Embodied_AI_Paper_List. [Embodied-AI-Survey-2025] Paper List and Resource Repository for Embodied AI

FedGM. "Multi-Source Collaborative Gradient Discrepancy Minimization for Federated Domain Generalization", AAAI 2024, MindSpore version

morehopqa. The evaluation code for the paper "MoreHopQA: More Than Multi-hop Reasoning"

prov-gigapath. Prov-GigaPath: A whole-slide foundation model for digital pathology from real-world data

M3D. M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models

HunyuanDiT. Hunyuan-DiT : A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

AutoStudio. [CVPRW 2026] AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation