Junyi Zhang

RATs. Implementation of paper "Playful Agentic Robot Learning"

PiD. PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion

threadweaver. The implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language Models

cap-x. A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation

LoGeR. Reimplementation of LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory

VisGym. Official Repository of VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents

Multiplex-Thinking. Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge

HY-WorldPlay. HY-World 1.5: A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency

St4RTrack. Official Implementation of paper "St4RTrack: Simultaneous 4D Reconstruction and Tracking in the World"

whole_body_tracking. Python

SJTU-Anonymous_Forum. 【无可奉告】——上海交大匿名论坛

LaCT. Code release for paper "Test-Time Training Done Right"

Sparse-VideoGen. [ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention

VideoMimic. Visual Imitation Enables Contextual Humanoid Control. CoRL 2025, Best Student Paper Award.

videomimic-1.github.io. HTML

dreamerv3. Mastering Diverse Domains through World Models

cleandift. Jupyter Notebook

unsloth. Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

viser-gradio-embed. Embedding Viser in Gradio

stable-virtual-camera. Stable Virtual Camera: Generative View Synthesis with Diffusion Models

vggt. [CVPR 2025 Best Paper Award] VGGT: Visual Geometry Grounded Transformer

HSfM_RELEASE. Code release for paper "Reconstructing People, Places, and Cameras", In CVPR 2025 (Highlight)

DELTA_densetrack3d. DELTA: Dense Efficient Long-range 3D Tracking for Any video (ICLR 2025)

TinyZero. Minimal reproduction of DeepSeek R1-Zero

mvdust3r. Open source impl of **MV-DUSt3R+ Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds** from Meta Reality Labs. Project page https://mv-dust3rp.github.io/

DenseMatcher. Official Repository for DenseMatcher Learning 3D Semantic Correspondence for Category-Level Manipulation from One Demo

ArtEq. Generalizing Neural Human Fitting to Unseen Poses With Articulated SE(3) Equivariance (ICCV2023)

EPro-PnP. [CVPR 2022 Best Student Paper] EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

mochi. The best OSS video generation models, created by Genmo

RCF-UnsupVideoSeg. [CVPR 2023] Segmenting objects in videos without human annotations 🤯: Official implementation for Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Grouping

CurrMask. Learning Versatile Skills with Curriculum Masking (NeurIPS 2024)

sam2. The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

Specular-Gaussians. [NeurIPS 2024] Official implementation of "Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting"

monst3r. Official Implementation of paper "MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion"

time_reversal. Official repo of "Time Reversal Fusion" (ECCV2024)

sanpo_dataset. Python

DepthCrafter. [CVPR 2025 Highlight] DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos

MatrixCity. [ICCV 2023] MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyond.

dynamic_stereo. [CVPR 2023] DynamicStereo: Consistent Dynamic Depth from Stereo Videos.

DytanVO. [ICRA'23] DytanVO: Visual Odometry in Dynamic Environments

viser. Web-based 3D visualization in Python

motiongrouping. Repository accompanying the paper: Self-supervised Video Object Segmentation by Motion Grouping. C. Yang, H. Lamdouar, E. Lu, A. Zisserman, W. Xie.

SCOOP. Self-Supervised Correspondence and Optimization-Based Scene Flow (CVPR 2023)

flowsam. [ACCV 2024 (Oral)] Official Implementation of "Moving Object Segmentation: All You Need Is SAM (and Flow)" Junyu Xie, Charig Yang, Weidi Xie, Andrew Zisserman

casualSAM. JavaScript

spring_utils. Additional utility code for the Spring dataset and benchmark

flux. Official inference repo for FLUX.1 models

robust_cvd. Robust Consistent Video Depth Estimation

Unsupervised-Learning-of-Robust-Spectral-Shape-Matching. SIGGRAPH23: Unsupervised Learning of Robust Spectral Shape Matching

spider-match. 🕷️ SpiderMatch: 3D Shape Matching with Global Optimality and Geometric Consistency

samurai. SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections - NeurIPS2022

deep-video-mvs. Code for "DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion" (CVPR 2021)

mast3r. Grounding Image Matching in 3D with MASt3R

MVSGaussian. [ECCV 2024] MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo

Gaussian-Wild. Official implementation of the paper "Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections" [ECCV 2024]

pips2. PIPs++

4D-Rotor-Gaussians. Python

GART. GART: Gaussian Articulated Template Models

Spatial-AST. 🦇 Encoder of BAT (Learning to Reason about Spatial Sounds with Large Language Models)

leapvo. [CVPR 2024] LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry

BLINK_Benchmark. This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.org/abs/2404.12390 [ECCV 2024]

dot. Dense Optical Tracking: Connecting the Dots

GLIGEN. Open-Set Grounded Text-to-Image Generation

nvs. Python

dino-tracker. Official Pytorch Implementation for “DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video” (ECCV 2024)

lab4d. A framework for 4D reconstruction from monocular videos.

2d-gaussian-splatting. [SIGGRAPH'24] 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

gcd. Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesis (ECCV 2024 Oral) - Official Implementation

dynibar. Implementation of DynIBaR Neural Dynamic Image-Based Rendering (CVPR 2023)

robust-dynrf. An algorithm for reconstructing the radiance field of a dynamic scene from a casually-captured video.

cop3d. Common Pets in 3D

STAG4D. Official Implementation for STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians

gaussian-grouping. [ECCV'2024] Gaussian Grouping for open-world Anything reconstruction, segmentation and editing.

dreamgaussian4d. [arXiv 2023] DreamGaussian4D: Generative 4D Gaussian Splatting

Hierarchical-Localization. Visual localization made easy with hloc

dycheck. Official JAX Implementation of Monocular Dynamic View Synthesis: A Reality Check (NeurIPS 2022)

floater-free-gaussian-splatting. An unofficial implementation of absGS

frame-interpolation. FILM: Frame Interpolation for Large Motion, In ECCV 2022.

Sandwich. Bidirectional Mapping between Action Physical-Semantic Space

SpacetimeGaussians. [CVPR 2024] Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis

diff-gaussian-rasterization. Cuda

apt-local-install. Tool for installing apt packages without root permission in user local space (aptli).

SC-GS. [CVPR 2024] Code for SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes

Deformable-3D-Gaussians. [CVPR 2024] Official implementation of "Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction"

4d-gaussian-splatting. [ICLR 2024] Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting

threestudio. A unified framework for 3D content generation.

Consistent4D. [ICLR 2024] Official Implementation of Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video

SJTUThesis. 上海交通大学 LaTeX 论文模板 | Shanghai Jiao Tong University LaTeX Thesis Template

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

ppr. Code for PPR: Physically Plausible Reconstruction from Monocular Videos (ICCV 23).

GRM. Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation

Diff-Plugin. [CVPR 2024] Official code release of our paper "Diff-Plugin: Revitalizing Details for Diffusion-based Low-level tasks"

VideoCrafter. VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models

ZeroNVS. Python

GeoAware-SC. Official Implementation of paper "Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence"

OCLR_model. [NeurIPS 2022] Segmenting Moving Objects via an Object-Centric Representation. Junyu Xie, Weidi Xie, Andrew Zisserman.

FeatUp. Official code for "FeatUp: A Model-Agnostic Frameworkfor Features at Any Resolution" ICLR 2024

rigidmask. Code for "Learning to Segment Rigid Motions from Two Frames". CVPR 2021.

omni3d. Code release for "Omni3D A Large Benchmark and Model for 3D Object Detection in the Wild"

WHAM. Python

particle-sfm. ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild. ECCV 2022.

ViT-Prisma. ViT Prisma is a mechanistic interpretability library for Vision and Video Transformers (ViTs).

4dfy. 4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling

madrona. C++

TripoSR. TripoSR: Fast 3D Object Reconstruction from a Single Image

Panda-70M. [CVPR 2024] Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

dust3r. DUSt3R: Geometric 3D Vision Made Easy

Diffusion-3D-Features. Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features [CVPR 2024]

VidToMe. Official Pytorch Implementation for "VidToMe: Video Token Merging for Zero-Shot Video Editing" (CVPR 2024)

DiffusionLight. [CVPR 2024] code release for "DiffusionLight: Light Probes for Free by Painting a Chrome Ball"

svd-temporal-controlnet. Python

visual-interaction-networks-pytorch. This's an implementation of deepmind Visual Interaction Networks paper using pytorch

groundingLMM. [CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

diffusion-motion-transfer. Official Pytorch Implementation for "Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer""

CLEVRER. PyTorch implementation of ICLR 2020 paper "CLEVRER: CoLlision Events for Video REpresentation and Reasoning"

jepa. PyTorch code and models for V-JEPA self-supervised learning from video.

co-tracker. CoTracker is a model for tracking any point (pixel) on a video.

LGM. [ECCV 2024 Oral] LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation.

image-sculpting. Code release for Image Sculpting: Precise Object Editing with 3D Geometry Control [CVPR 2024]

python-fire. Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.

PoseAnything. A Graph-Based Approach for Category-Agnostic Pose Estimation [ECCV 2024]

InstantID. InstantID: Zero-shot Identity-Preserving Generation in Seconds 🔥

hpf. Official PyTorch Implementation of Hyperpixel Flow, ICCV 2019

SCOT. CVPR 2020, Semantic Correspondence as an Optimal Transport Problem, Pytorch Implementation.

AwA-Pose. Jupyter Notebook

demistfy_correspondence. Code for the ECCV22 paper Demystifying Unsupervised Semantic Correspondence Estimation

SEINE. [ICLR 2024] SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction

Vlogger. [CVPR2024] Make Your Dream A Vlog

Diff-Protect. 🛡️[ICLR'2024] Toward effective protection against diffusion-based mimicry through score distillation, a.k.a SDS-Attack

TheChosenOne. Unofficial implementation of the paper "The Chosen One: Consistent Characters in Text-to-Image Diffusion Models"

MotionCtrl. Official Code for MotionCtrl [SIGGRAPH 2024]

Denoising-ViT. This is the official code release for our work, Denoising Vision Transformers.

AnimateDiff. Official implementation of AnimateDiff.

FreeNoise. [ICLR 2024] Code for FreeNoise based on VideoCrafter

MotionDirector. [ECCV 2024 Oral] MotionDirector: Motion Customization of Text-to-Video Diffusion Models.

Awesome-Video-Diffusion. A curated list of recent diffusion models for video generation, editing, and various other applications.

VGen. Official repo for VGen: a holistic video generation ecosystem for video generation building on diffusion models

emotion2vec. [ACL 2024] Official PyTorch code for extracting features and training downstream models with emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation

Jigsaw. [NeurIPS 2023] Jigsaw: Learning to Assemble Multiple Fractured Objects

generative-models. Generative Models by Stability AI

pyramid-discrete-diffusion. [ECCV 2024 Oral] Pyramid Diffusion for Fine 3D Large Scene Generation

libcom. Image composition toolbox: everything you want to know about image composition/compositing or object/subject insertion/addition/compositing.

consistencydecoder. Consistency Distilled Diff VAE

DriveDownloader. Download resources from online storage with ONLY ONE command line!!

VCN. Volumetric Correspondence Networks for Optical Flow, NeurIPS 2019.

gaussian-splatting. Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"

tabilize. Simple code for generating a color-coded latex table from raw data

py-thin-plate-spline. Code for computing interpolating / approximating thin plate splines.

XX-Net. A proxy tool to bypass GFW.

scene-representation-diffusion-model. Linear probe found representations of scene attributes in a text-to-image diffusion model

doppelgangers. Doppelgangers: Learning to Disambiguate Images of Similar Structures

e2e_multi_view_matching. End2End Multi-View Feature Matching with Differentiable Pose Optimization

NeTI. Official Implementation for "A Neural Space-Time Representation for Text-to-Image Personalization" (SIGGRAPH Asia 2023)

view_neti. [ECCV 2024] Viewpoint Textual Inversion: Discovering Scene Representations and 3D View Control in 2D Diffusion Models

FreeU. FreeU: Free Lunch in Diffusion U-Net (CVPR2024 Oral)

DiT. Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

cycle-diffusion. [ICCV 2023] A latent space for stochastic diffusion models

InstructDiffusion. PyTorch implementation of InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions.

sdxl_inversions. Jupyter Notebook

SyncDreamer. [ICLR 2024 Spotlight] SyncDreamer: Generating Multiview-consistent Images from a Single-view Image

Dynamic3DGaussians. Python

MasaCtrl. [ICCV 2023] Consistent Image Synthesis and Editing

InterDiff. [ICCV 2023] Official PyTorch implementation of the paper "InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion"

DragDiffusion. [CVPR2024, Highlight] Official code for DragDiffusion

ControlNet. Generate videos that interpolate between two given images

TEDi. Python

zero-shot-pose. Code for the paper "Zero-Shot Category-Level 6D Pose Estimation"

diffusion_hyperfeatures. Official PyTorch Implementation for Diffusion Hyperfeatures, NeurIPS 2023

distill-sd. Segmind Distilled diffusion

TokenFlow. Official Pytorch Implementation for "TokenFlow: Consistent Diffusion Features for Consistent Video Editing" presenting "TokenFlow" (ICLR 2024)

3DCoMPaT-v2. 3DCoMPaT++: An improved large-scale 3D vision dataset for compositional recognition

csm. Code release for "Canonical Surface Mapping via Geometric Cycle Consistency"

noah-research. Noah Research

OSX. [CVPR 2023] Official implementation of the paper "One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer"

HybrIK. Official code of "HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation", CVPR 2021

ViTPose. The official repo for [NeurIPS'22] "ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation" and [TPAMI'23] "ViTPose++: Vision Transformer for Generic Body Pose Estimation"

PETR. [ECCV2022] PETR: Position Embedding Transformation for Multi-View 3D Object Detection & [ICCV2023] PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images

self-corr-pose. Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild, ICLR 2023

DiffSketch. DiffSketch: Plug-and-Play Diffusion Features for Zero-Shot Line-Drawing-to-Sketch Synthesis. Course project of CS3511, built on pnp-diffusion.

Magic123. [ICLR'24] Official PyTorch Implementation of Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

d2-net. D2-Net: A Trainable CNN for Joint Description and Detection of Local Features

caps. Jupyter Notebook

SuperPoint. Efficient neural feature detector and descriptor

UMR. Self-supervised Single-view 3D Reconstruction

navi. Jupyter Notebook

LightGlue. LightGlue: Local Feature Matching at Light Speed (ICCV 2023)

LayoutGeneration. Python

kubric. A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.

dreamsim. DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data (NeurIPS 2023 Spotlight) / / / / When Does Perceptual Alignment Benefit Vision Representations? (NeurIPS 2024)

tapnet. Tracking Any Point (TAP)

GTR. Global Tracking Transformers, CVPR 2022

Track-Anything. Track-Anything is a flexible and interactive tool for video object tracking and segmentation, based on Segment Anything, XMem, and E2FGVI.

whisper. Robust Speech Recognition via Large-Scale Weak Supervision

sd-dino. Official Implementation of paper "A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence"

banmo. BANMo Building Animatable 3D Neural Models from Many Casual Videos

concept-fusion. Code release for ConceptFusion [RSS 2023]

awesome-tips.

VLPart. [ICCV2023] VLPart: Going Denser with Open-Vocabulary Part Segmentation

ElundusCoreApp. Desktop application to convert text-to-speech to preview Twitch donations.

sd-complements-dino.github.io. HTML