Fei Xia

GMR. [ICRA 2026] GMR: General Motion Retargeting. Retarget human motions into diverse humanoid robots in real time on CPU. Retargeter for TWIST.

awesome-mcp-servers. A collection of MCP servers.

openpi. Python

research-website-template. React + Next.js template for research websites (for PhD students, researchers, etc)

open-pi-zero. Re-implementation of pi0 vision-language-action (VLA) model from Physical Intelligence

alphafold3. AlphaFold 3 inference pipeline.

maxtext. A simple, performant, and scalable Jax LLM!

luckyrobots. Python SDK for LuckyEngine robotics simulation

arx5-sdk. C++ and Python controller for ARX5 robot arm developed by @yihuai-gao (@real-stanford)

SO-ARM100. Standard Open Arm 100

arx5-sdk. SDK for ARX5 robot arm

ACT. Action Chunking Transformer implementation for low cost robot

VQASynth. Compose multimodal datasets 🎹

cookbook. Examples and guides for using the Gemini API

transformer-debugger. Python

Awesome-Generalist-Robots-via-Foundation-Models. Paper list in the survey paper: Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis

Awesome-Robotics-Foundation-Models.

deprecated-generative-ai-python. This SDK is now deprecated, use the new unified Google GenAI SDK.

open_x_embodiment. Jupyter Notebook

GenSim. Generating Robotic Simulation Tasks via Large Language Models

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

paper-qa. High accuracy RAG for answering questions from scientific documents with citations

point-e. Point cloud diffusion for 3D model synthesis

latent-nerf. Official Implementation for "Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures"

mujoco_mpc. Real-time behaviour synthesis with MuJoCo, using Predictive Control

robotic-transformer-pytorch. Implementation of RT1 (Robotic Transformer) in Pytorch

BEHAVIOR-1K. BEHAVIOR-1K: a platform for accelerating Embodied AI research. Join our Discord for support: https://discord.gg/bccR5vGFEx

robotics_transformer. Python

PyChatGPT. ⚡️ Python client for the ChatGPT API with, conversation tracking, proxy support and more.

language-table. Suite of human-collected datasets and a multi-task continuous control benchmark for open vocabulary visuolinguomotor learning.

semantic-abstraction. [CoRL 2022] This repository contains code for generating relevancies, training, and evaluating Semantic Abstraction.

omnigibson-eccv-tutorial. Jupyter Notebook

langchain. The agent engineering platform.

omni3d. Code release for "Omni3D A Large Benchmark and Model for 3D Object Detection in the Wild"

clip-fields. Teaching robots to respond to open-vocab queries with CLIP and NeRF-like neural fields

GET3D. Python

G.pt. Official PyTorch Implementation of "Learning to Learn with Generative Models of Neural Network Checkpoints"

LAVIS. LAVIS - A One-stop Library for Language-Vision Intelligence

Poisson_flow. Code for NeurIPS 2022 Paper, "Poisson Flow Generative Models" (PFGM)

planning-with-deep-neural-networks-a-survey. Planning with Deep Neural Networks: A Survey

SparseNeuS. SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse views

long_stable_diffusion. Long-form text-to-images generation, using a pipeline of deep generative models (GPT-3 and Stable Diffusion)

BIG-bench. Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Awesome-LLM-Robotics. A comprehensive list of papers using large language/multi-modal models for Robotics/RL, including papers, codes, and related websites

stable-diffusion. A latent text-to-image diffusion model

RePaint. Official PyTorch Code and Models of "RePaint: Inpainting using Denoising Diffusion Probabilistic Models", CVPR 2022

VQ-Diffusion. Python

dreamfields-torch. A pytorch implementation of dreamfields with modifications.

ngp_pl. Instant-ngp in pytorch+cuda trained with pytorch-lightning (high quality with high speed, with only few lines of legible code)

nerf_pl. NeRF (Neural Radiance Fields) and NeRF in the Wild using pytorch-lightning

latent-diffusion. High-Resolution Image Synthesis with Latent Diffusion Models

Geo-Neus. Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction (NeurIPS 2022)

nvdiffrec. Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".

torch-ngp. A pytorch CUDA extension implementation of instant-ngp (sdf and nerf), with a GUI.

neurecon. Multi-view 3D reconstruction using neural rendering. Unofficial implementation of UNISURF, VolSDF, NeuS and more.

Video-Pre-Training. Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

MineDojo. Building Open-Ended Embodied Agents with Internet-Scale Knowledge

Awesome-Diffusion-Models. A collection of resources and papers on Diffusion Models

SECANT. [ICML 2021] Official code for SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies

OmniIsaacGymEnvs. Reinforcement Learning Environments for Omniverse Isaac Gym

ACID. ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation

Clumppling. Clumppling (CLUster Matching and Permutation Program that uses integer Linear programmING) for aligning mixed-membership clustering results of population structure analysis.

HowToCook. Programmer's guide about how to cook at home.

r3m. Pre-training Reusable Representations for Robotic Manipulation Using Diverse Human Video Data

PlotNeuralNet. Latex code for making neural networks diagrams

instant-ngp. Instant neural graphics primitives: lightning fast NeRF and more

CME292. CME292 (Winter Quarter 2022) Advanced MATLAB for Scientific Computing, offered by Stanford ICME

ibc. Official implementation of Implicit Behavioral Cloning, as described in our CoRL 2021 paper, see more at https://implicitbc.github.io/

wikiHow_paper_list. A paper list of research conducted based on wikiHow

torchkit. Research boilerplate for PyTorch.

PrimitiveFitting. C++

taskography. Task planning over 3D scene graphs

benchbot_sim_unreal. Retired Isaac Sim 2019.2 wrappers for the BenchBot software stack: https://github.com/qcr/benchbot

benchbot. BenchBot is a tool for seamlessly testing & evaluating semantic scene understanding tools in both realistic 3D simulation & on real robots

OpenTraj. Human Trajectory Prediction Dataset Benchmark (ACCV 2020)

tacto. Simulator of vision-based tactile sensors.

dynamic-video-depth. Code for the SIGGRAPH 2021 paper "Consistent Depth of Moving Objects in Video".

brax. Massively parallel rigidbody physics simulation on accelerator hardware.

bddl-100. Python

open_clip. An open source implementation of CLIP.

fairo. A modular embodied agent architecture and platform for building embodied agents

rmp_nav. Python

torchshard. Slicing a PyTorch Tensor Into Parallel Shards

vnn-neural-implicits. Neural implicit reconstruction experiments for the Vector Neuron paper

alias-free-gan. Alias-Free GAN project website and code

BehaviorChallenge2021. Python

alphafold. Open source code for AlphaFold 2.

FreeViewSynthesis. Code repository for "Free View Synthesis", ECCV 2020.

softlearning. Softlearning is a reinforcement learning framework for training maximum entropy policies in continuous domains. Includes the official implementation of the Soft Actor-Critic algorithm.

stable-baselines3. PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

manipulathor. ManipulaTHOR, a framework that facilitates visual manipulation of objects using a robotic arm

megaverse. High-throughput simulation platform for Artificial Intelligence reseach

psdr-cuda. Path-space differentiable renderer

Correlated-Sequential-Halving. This is a codebase for computing the medoid of a dataset accompanying the paper "Ultra Fast Medoid Identification via Correlated Sequential Halving" https://arxiv.org/abs/1906.04356

DialoGPT. Large-scale pretraining for dialogue

gpt-neo. An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

dl-visuals. Over 200 figures and diagrams of the most popular deep learning architectures and layers FREE TO USE in your blog posts, slides, presentations, or papers.

packnet-sfm. TRI-ML Monocular Depth Estimation Repository

robodesk. Python

EvolvingGraphicalPlanner. official implementation for "Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation"

phylogenetic-cnn. Jupyter Notebook

PopPhy-CNN. Python

MLP-Mixer-pytorch. Unofficial implementation of MLP-Mixer: An all-MLP Architecture for Vision

virtualhome. API to run VirtualHome, a Multi-Agent Household Simulator

PoseRBPF. A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking

vnn. Vector Neurons: A General Framework for SO(3)-Equivariant Networks

pytorchvideo. A deep learning library for video understanding research.

differentiable_volumetric_rendering. This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Robust-Differentiable-SVD. Python

gradsim. Differentiable simulation for system identification and visuomotor control

PicassoPlus. Geometric deep learning for surface parsing from 3-D triangular meshes

deepKNet. PointNet-based 3D deep learning model designed for decoding the structure-property relationship for generic crystalline materials.

strips. A python implementation of the STRIPS planning algorithm

awesome-domain-adaptation. A collection of AWESOME things about domain adaptation

sfepy. Main SfePy repository

uncertainty-toolbox. Uncertainty Toolbox: a Python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

spatial-action-maps. Learning robotic mobile manipulation with deep reinforcement learning

UWCNN. Code and Datasets for "Underwater Scene Prior Inspired Deep Underwater Image and Video Enhancement", Pattern Recognition, 2019

RandLA-Net. 🔥RandLA-Net in Tensorflow (CVPR 2020, Oral & IEEE TPAMI 2021)

kinematic-feasibility-rl. Learning Kinematic Feasibility through Reinforcement Leanring: http://rl.uni-freiburg.de/research/kinematic-feasibility-rl

Objectron. Objectron is a dataset of short, object-centric video clips. In addition, the videos also contain AR session metadata including camera poses, sparse point-clouds and planes. In each video, the camera moves around and above the object and captures it from different views. Each object is annotated with a 3D bounding box. The 3D bounding box describes the object’s position, orientation, and dimensions. The dataset contains about 15K annotated video clips and 4M annotated images in the following categories: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops, and shoes

DALL-E. PyTorch package for the discrete VAE used for DALL·E.

bullet3. Bullet Physics SDK: real-time collision detection and multi-physics simulation for VR, games, visual effects, robotics, machine learning etc.

iGibsonChallenge2021. Python

StableViewSynthesis. Python

xls. XLS: Accelerated HW Synthesis

ravens. Train robotic agents to learn pick and place with deep learning for vision-based manipulation in PyBullet. Transporter Nets, CoRL 2020.

allennlp. An open-source NLP research library, built on PyTorch.

rebiber. A simple tool to update bib entries with their official information (e.g., DBLP or the ACL anthology).

gifski. GIF encoder based on libimagequant (pngquant). Squeezes maximum possible quality from the awful GIF format.

PSGNets. Neural Networks that convert input movies into Physical Scene Graphs (PSGs)

nimblephysics. Nimble: Physics Engine for Biomechanics and Deep Learning

GON. Gradient Origin Networks - a new type of generative model that is able to quickly learn a latent representation without an encoder

deformable-ravens. Code for ICRA 2021 paper: https://arxiv.org/abs/2012.03385

DeepIM-PyTorch. PyTorch implementation of the DeepIM framework

PoseCNN-PyTorch. PyTorch implementation of the PoseCNN framework

HMS. The repository of the code base of "Multi-Layer Semantic and Geometric Modeling with Neural Message Passing in 3D Scene Graphs for Hierarchical Mechanical Search"

nerf-pytorch. A PyTorch implementation of NeRF (Neural Radiance Fields) that reproduces the results.

caspr. Code for NeurIPS 2020 paper "CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations"

ml-hypersim. Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

OMG-Planner. An Optimization-based Motion and Grasp Planner

awesome-neural-planner. personal paper reading on neural motion planner and controller

v4r. C++

SNGAN-AdaBelief. Train SN-GAN with AdaBelief

Adabelief-Optimizer. Repository for NeurIPS 2020 Spotlight "AdaBelief Optimizer: Adapting stepsizes by the belief in observed gradients"

actionet. 3D household task-based dataset created using customised AI2-THOR.

Generative-3D-Part-Assembly. Generative 3D Part Assembly via Dynamic Graph Learning, NeurIPS 2020

pytorch_sac. PyTorch implementation of Soft Actor-Critic (SAC)

drq. DrQ: Data regularized Q

allenact. An open source framework for research in Embodied-AI from AI2.

GSDN. Generative Sparse Detection Networks for 3D Single-shot Object Detection

NVISII. C++

tiny-differentiable-simulator. Tiny Differentiable Simulator is a header-only C++ and CUDA physics library for reinforcement learning and robotics with zero dependencies.

nerf-pytorch. A PyTorch re-implementation of Neural Radiance Fields

ActionGenome. A video database bridging human actions and human-object relationships

kuka_rl. Reinforcement Learning Experiments using PyBullet

ManifoldPlus. ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle Soups

mitsuba2. Mitsuba 2: A Retargetable Forward and Inverse Renderer

e2cnn. E(2)-Equivariant CNNs Library for Pytorch

Mitsuba2PointCloudRenderer. rendering (optionally temporal) point clouds using Mitsuba2

TopologyLayer. A Topology Layer for Machine Learning : Persistent Homology + Features for PyTorch

blender-cli-rendering. Blender Python scripts for rendering images directly from command-line interface

npbg. Neural Point-Based Graphics

nerf. Code release for NeRF (Neural Radiance Fields)

Microsoft-Rocketbox. Microsoft Rocketbox is now available with MIT license! The library of 115 rigged avatars offers flexibility, enabling the easy use of animations across characters and the mixing and matching of texture elements.

reveal-md. reveal.js on steroids! Get beautiful reveal.js presentations from any Markdown file

missing-semester. The Missing Semester of Your CS Education 📚

pytorch-lightning. Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

scene-representation-networks. Official Pytorch implementation of Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations

DDSL. DDSL: Deep Differential Simplex Layer for Neural Networks

open-ucn. The first fully convolutional metric learning for geometric/semantic image correspondences.

furniture. IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Tasks

PyFleX. Customized Python APIs for NVIDIA FleX

3DSceneGraph. The data skeleton from "3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera" http://3dscenegraph.stanford.edu

kaolin. A PyTorch Library for Accelerating 3D Deep Learning Research

cvxpylayers. Differentiable convex optimization layers

Physics-Based-Deep-Learning. Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

RLBench. A large-scale benchmark and learning environment.

procedural-objects. Procedural object generation for robotic manipulation

plane-opt-rgbd. Plane-Based Optimization of Geometry and Texture for RGB-D Reconstruction of Indoor Scenes

DoorGym. Open source domain randomized door opening training environment

Megatron-LM. Ongoing research training transformer models at scale

GPT2-Chinese. Chinese version of GPT2 training code, using BERT tokenizer.

neural_rerendering_in_the_wild. Python

PPO-PyTorch. Minimal implementation of clipped objective Proximal Policy Optimization (PPO) in PyTorch

FrameNet. FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image

Open3D-PointNet. Open3D PointNet implementation with PyTorch

NOCS_CVPR2019. [CVPR2019 Oral] Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation on Python3, Tensorflow, and Keras

iGibson. A Simulation Environment to train Robots in Large Realistic Interactive Scenes

Replica-Dataset. The Replica Dataset v1 as published in https://arxiv.org/abs/1906.05797 .

GSPN. GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

visual-prior. Code for Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Visuomotor Policies. Arxiv preprint 2018. Alexander Sax, Bradley Emi, Amir R. Zamir, Silvio Savarese, Leonidas Guibas, Jitendra Malik.

3D-SIS. [CVPR'19] 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans

MinkowskiEngine. Minkowski Engine is an auto-diff neural network library for high-dimensional sparse tensors

lcnn. LCNN: End-to-End Wireframe Parsing

choochoo. Training Diary

SoftRas. Project page of paper "Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning"

Meddit. Jupyter Notebook

Deep-Learning-in-Production. In this repository, I will share some useful notes and references about deploying deep learning-based models in production.