Zhenhailong Wang

PlanBench-XL. Official Repository for our paper: PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems

Brick-Composer. Public Repository for "Brick-Composer: Using MLLMs for Assembly with Diverse Bricks"

AdaPlanBench. Official Repository for our paper: AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

claude-code. Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands. All original source code is the property of Anthropic.

a-evolve. The official repository of "Position: Agentic Evolution is the Path to Evolving LLMs".

lejepa. Python

matter-js. a 2D rigid body physics engine for the web ▲● ■

graphium. Graphium: Scaling molecular GNNs to infinity.

minimol. MiniMol is a 10M-parameters molecular fingerprinting model pre-trained on >3300 biological and quantum tasks

ADMET_XGBoost. Accurate ADMET Prediction with XGBoost

DeepPurpose. A Deep Learning Toolkit for DTI, Drug Property, PPI, DDI, Protein Function Prediction (Bioinformatics)

TDC. Therapeutics Commons (TDC): Multimodal Foundation for Therapeutic Science

mCLM. Python

Qwen3-Coder. Qwen3-Coder is the code version of Qwen3, the large language model series developed by Qwen team.

Mini-o3. Official Code for "Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search"

GTA. [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2

ROLL. An Efficient and User-Friendly Scaling Library for Reinforcement Learning with Large Language Models

TreeVGR. [ICLR'26] Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology

llms-interview-questions. 🟣 LLMs interview questions and answers to help you prepare for your next machine learning and data science interview in 2026.

GTA1. Python

PAPO. Official repo for "PAPO: Perception-Aware Policy Optimization for Multimodal Reasoning"

ross. [ICLR'25] Reconstructive Visual Instruction Tuning

V-DPO. Preference Learning for LLaVA

r1_reward. ✨✨ [ICLR 2026] R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning

BLIP3o. Official implementation of BLIP3o-Series

ncsa_tutorial. Tutorial for UIUC NCSA Cluster

VL-Rethinker. The official code of "VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning" [NeurIPS25]

VAGEN. World model reasoning RL for multi-turn VLM agents

dymu. Python

arxiv-latex-cleaner. arXiv LaTeX Cleaner: Easily clean the LaTeX code of your paper to submit to arXiv

lmm-r1. Extend OpenRLHF to support LMM RL training for reproduction of DeepSeek-R1 on multimodal tasks.

vila-u. [ICLR 2025] VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

VARGPT-v1.1. VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning

ILLUME_plus. [CVPR2025] Official Implementation of ILLUME+

EasyR1. EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

Skywork-R1V. Skywork-R1V is an advanced multimodal AI model series developed by Skywork AI, specializing in vision-language reasoning.

Search-R1. Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

Emu3. Next-Token Prediction is All You Need

VLMEvalKit. Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

VLM-R1. Solve Visual Understanding with Reinforced VLMs

open-r1-multimodal. A fork to add multimodal model training to open-r1

opencompass. OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

R1-V. Witness the aha moment of VLM with less than $3.

LLaVA-Mini. LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

open_clip. An open source implementation of CLIP.

Janus. Janus-Series: Unified Multimodal Understanding and Generation Models

manim. Animation engine for explanatory math videos

genesis-world. Simulation platform for general-purpose robotics & embodied AI learning.

Agent-S. Agent S: an open agentic framework that uses computers like a human

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

efficientvit. Efficient vision foundation models for high-resolution generation and perception.

hart. HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

LLM-Agent-Paper-List. The paper list of the 86-page SCIS cover paper "The Rise and Potential of Large Language Model Based Agents: A Survey" by Zhiheng Xi et al.

MobileAgent. Mobile-Agent: The Powerful GUI Agent Family

mar. PyTorch implementation of MAR+DiffLoss https://arxiv.org/abs/2406.11838

autoregressive-diffusion-pytorch. Implementation of Autoregressive Diffusion in Pytorch

Grounded-SAM-2. Grounded SAM 2: Ground and Track Anything in Videos with Grounding DINO, Florence-2 and SAM 2

istockphoto. 📸 Gracefully download dataset from iStockPhoto.

Awesome-Unified-Multimodal-Models. 📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

Show-o. [ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.

Lumina-mGPT. Official Implementation of "Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining"

qwqjsq. qwqjsq.com 的 最新地址

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

anole. [Extended verision ICLR 2025 Blog Track] Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

VQGAN-LC. Python

SEED-Voken. SEED-Voken: A Series of Powerful Visual Tokenizers

1d-tokenizer. This repo contains the code for 1D tokenizer and generator

Awesome-Video-Datasets. Video datasets

Multimodal-AND-Large-Language-Models. Paper list about multimodal and large language models, only used to record papers I read in the daily arxiv for personal needs.

VDLM. Repo for paper: https://arxiv.org/abs/2404.06479

nerfies.github.io. JavaScript

alphageometry. Python

CHOCOLATE. Code and data for the ACL 2024 Findings paper "Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning"

EEG-To-Text. Python

maze-dataset. maze datasets for investigating OOD behavior of ML systems

torchgeo. TorchGeo: datasets, samplers, transforms, and pre-trained models for geospatial data

Tracking-Anything-with-DEVA. [ICCV 2023] Tracking Anything with Decoupled Video Segmentation

ecole-dataset. Python

atp-video-language. Official repo for CVPR 2022 (Oral) paper: Revisiting the "Video" in Video-Language Understanding. Contains code for the Atemporal Probe (ATP).

Solo-Performance-Prompting. Repo for paper "Unleashing Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration"

tree-of-thought-llm. [NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

LLM-ToolMaker. Jupyter Notebook

Paxion. Repo for paper: "Paxion: Patching Action Knowledge in Video-Language Foundation Models" Neurips 23 Spotlight

viper. Code for the paper "ViperGPT: Visual Inference via Python Execution for Reasoning"

yt-dlp. A feature-rich command-line audio/video downloader

InternVideo. [ECCV2024] Video Foundation Models & Data for Multimodal Understanding

dalle2-laion. Pretrained Dalle2 from laion

imagen-pytorch. Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

DALLE2-pytorch. Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

LookForTheChange. Code for Look for the Change paper published at CVPR 2022

procthor-10k. The ProcTHOR-10K Houses Dataset

procthor. 🏘️ Scaling Embodied AI by Procedurally Generating Interactive 3D Houses

detrex. detrex is a research platform for DETR-based object detection, segmentation, pose estimation and other visual recognition tasks.

diffusers. 🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.

stable-diffusion. A latent text-to-image diffusion model

tfrecord. Standalone TFRecord reader/writer with PyTorch data loaders

alfworld-docker-setup. Python

Wikinews_Pipeline. Get news from Wikipedia page's reference section

MineDojo. Building Open-Ended Embodied Agents with Internet-Scale Knowledge

LAVIS. LAVIS - A One-stop Library for Language-Vision Intelligence

Deep-reinforcement-learning-with-pytorch. PyTorch implementation of DQN, AC, ACER, A2C, A3C, PG, DDPG, TRPO, PPO, SAC, TD3 and ....

MachineLearning_Python. 机器学习算法python实现

stanford-cs-230-deep-learning. VIP cheatsheets for Stanford's CS 230 Deep Learning

rl-fa22-mps. Python

Zemi. Repo for "Zemi: Learning Zero-Shot Semi-Parametric Language Models from Multiple Tasks" ACL 2023 Findings

FiD. Fusion-in-Decoder

DPR. Dense Passage Retriever - is a set of tools and models for open domain Q&A task.

acl2020-openqa-tutorial. ACL2020 Tutorial: Open-Domain Question Answering

RESIN-11. Shell

VidIL. Pytorch code for Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners

flamingo-pytorch. Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

perceiver-io. A PyTorch implementation of Perceiver, Perceiver IO and Perceiver AR with PyTorch Lightning scripts for distributed training

t-zero. Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

metaseq. Repo for external large-scale work

VideoLanguageFuturePred. [EMNLP 2020] What is More Likely to Happen Next? Video-and-Language Future Event Prediction

tilse. Timeline summarization and evaluation.

Food-VR-Game. C#

Canary. A Simple Argument Mining Library

enhanced-subject-verb-object-extraction. Enhanced Subject Word Object Extraction

gpt-neo. An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

ALPRO. Align and Prompt: Video-and-Language Pre-training with Entity Prompts

animeGAN. A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

StarterCode. Starter Code for VALUE benchmark

OpenAI-Api-Unreal. Integration for the OpenAI Api in Unreal Engine

Multitask-Finetuning_CLIP. Code for paper "Rethinking Task Sampling for Few-shot Vision-Language Transfer Learning" COLING 2022 workshop

numerics-notes. Slides/notes and Jupyter notebook demos for an introductory course of numerical analysis/scientific computing

numpy-100. 100 numpy exercises (with solutions)

AttDiscovery. Python

transformers. 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

NLP-progress. Repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks.

pytorch-3dunet. 3D U-Net model for volumetric semantic segmentation written in pytorch

Entity_Ontology_Guide. User Friendly Kairos Entity Ontology

cee498te. repository for CEE 498TE transportation economics

Schema_Composition. GLSL

acl-anthology. Data and software for building the ACL Anthology.

Aida_COVID. Repo for Aida Covid Hackathon src

naacl-org.github.com. NAACL website

SAO. Python

EEG-To-Text. code for AAAI2022 paper "Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification"

rebiber. A simple tool to update bib entries with their official information (e.g., DBLP or the ACL anthology).

wikidata-dump-processor. Import Wikidata json dump (.json.bz2) into Mongodb

DALL-E. PyTorch package for the discrete VAE used for DALL·E.

kirby-s-kitchen-2. A IOS app for creating recipe memo

pytorch-3dunet. 3D U-Net model for volumetric semantic segmentation written in pytorch

MikeWangWZHL.github.io. Github Pages template for academic personal websites, forked from mmistakes/minimal-mistakes

openvslam. OpenVSLAM: A Versatile Visual SLAM Framework

Event_coref. Python

Event_prediction_temporal_classifier. Python

Event_Prediction. Python

cs412_project. Python