Long(Tony) Lian

agents-last-exam. Agents' Last Exam

PostTrainBench. Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours

first-steps-toward-automated-ai-research. Research artifacts from Recursive's automated AI research system

hypergrep. Recursively search directories for a regex pattern

hil-bench. Python

deep-swe. Measuring frontier coding agents on original, long-horizon engineering tasks

OpenClaw-RL. OpenClaw-RL: Train any agent simply by talking

design.md. A format specification for describing a visual identity to coding agents. DESIGN.md gives agents a persistent, structured understanding of a design system.

awesome-design-md. A collection of DESIGN.md files analysis by popular brand design systems. Drop one into your project and let coding agents generate a matching UI.

AutoGaze. AutoGaze automatically removes redundant patches in a video, reducing #tokens in ViT/MLLM by 4x-100x.

Stirrup. The lightweight framework for building agents

OpenResearcher. OpenResearcher: A Fully Open Pipeline for Long-Horizon Deep Research Trajectory Synthesis

fzf. :cherry_blossom: A command-line fuzzy finder

whisper.cpp. Port of OpenAI's Whisper model in C/C++

nvm. Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions

viu. Terminal image viewer with native support for iTerm and Kitty

slime. slime is an LLM post-training framework for RL Scaling.

openui. The Open Standard for Generative UI

open-webui. User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

VisGym. Official Repository of VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents

VPBench. Implementation for VPBench proposed in paper Visually Prompted Benchmarks Are Surprisingly Fragile

DeepResearch. Tongyi Deep Research, the Leading Open-source Deep Research Agent

simple-evals. Python

ghstack. Submit stacked diffs to GitHub on the command line

tau-bench. Code and Data for Tau-Bench

torchtitan. A PyTorch native platform for training generative AI models

threadweaver. The implementation for ThreadWeaver Adaptive Threading for Efficient Parallel Reasoning in Language Models

rate-evals. A comprehensive evaluation pipeline for Vision-Language Models on medical imaging tasks, with built-in support for multi-GPU processing, real-time progress tracking, and disease finding classification.

StringSight. Automatically Analyze your Model Traces

Native-Parallel-Reasoner. [ICML 2026] Reasoning in Parallelism via Self-Distilled RL

K2-Vendor-Verifier. Verify Precision of all Kimi K2 API Vendor

tinker-cookbook. Post-training with Tinker

MMMU. This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

nanochat. The best ChatGPT that $100 can buy.

nanoGPT. The simplest, fastest repository for training/finetuning medium-sized GPTs.

openevolve. Open-source implementation of AlphaEvolve

CT-CLIP. Developing Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography

Parallel-R1. The offical repo for "Parallel-R1: Towards Parallel Thinking via Reinforcement Learning"

Qwen-Agent. Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.

MedEvalKit. MedEvalKit: A Unified Medical Evaluation Framework

Merlin. [Nature 2026] Merlin is a 3D VLM for computed tomography that leverages both structured electronic health records (EHR) and unstructured radiology reports for pretraining.

medgemma. Jupyter Notebook

gpt-oss. gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

harmony. Renderer for the harmony response format to be used with gpt-oss

gemini-cli. An open-source AI agent that brings the power of Gemini directly into your terminal.

POLARIS. Scaling RL on advanced reasoning models

SkyRL. SkyRL: A Modular Full-stack RL Library for LLMs

RL2. Python

Hermes-Function-Calling. Jupyter Notebook

R1-Searcher-plus. R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning

R1-Searcher. R1-searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

RLLoggingBoard. A visuailzation tool to make deep understaning and easier debugging for RLHF training.

DeepEyes. Python

DeepResearcher. Scaling Deep Research via Reinforcement Learning in Real-world Environments.

ReLIFT. Official Repository of "Learning what reinforcement learning can't"

Multiverse.

open-deep-research. An open source deep research clone. AI Agent that reasons large amounts of web data extracted with Firecrawl

nano-vllm. Nano vLLM

AFLDM. [CVPR 2025 Oral] Alias-free Latent Diffusion Models (official implementation)

Enigmata. Resources for the Enigmata Project.

ParScale. Parallel Scaling Law for Language Model — Beyond Parameter and Inference Time Scaling

Entropy-Mechanism-of-RL. The Entropy Mechanism of Reinforcement Learning for Large Language Model Reasoning.

MMR1. [CVPR 2026] MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources

Bagel. Open-source unified multimodal model

s1. s1: Simple test-time scaling

inference_scaling. Python

Mixture-of-Transformers. Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models. TMLR 2025.

Downstream-Dinov2. Downstream-Dino-V2: A GitHub repository featuring an easy-to-use implementation of the DINOv2 model by Facebook for downstream tasks such as Classification, Semantic Segmentation and Monocular depth estimation.

tips. TIPSv2 (CVPR'26) and TIPS (ICLR'25)

x-reasoner. X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains

open-thoughts. Fully open data curation for reasoning models

Speech. A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

SEED-X. Multimodal Models in Real World

RadFM. The official code for "Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data".

LISA. Project Page for "LISA: Reasoning Segmentation via Large Language Model"

open-instruct. AllenAI's post-training codebase

Sana. SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

Step1X-Edit. A SOTA open-source image editing model, which aims to provide comparable performance against the closed-source models like GPT-4o and Gemini 2 Flash.

miniF2F. Formal to Formal Mathematics Benchmark

tree-of-thoughts. Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

tree-of-thought-llm. [NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

TTRL. [NeurIPS 2025] TTRL: Test-Time Reinforcement Learning

ml-space-benchmark. Code and data for "Does Spatial Cognition Emerge in Frontier Models?"

describe-anything. [ICCV 2025] Implementation for Describe Anything: Detailed Localized Image and Video Captioning

APR. [COLM 2025] Code for Paper: Learning Adaptive Parallel Reasoning with Language Models

litellm. Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

opencompass. OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

python-sdk. The official Python SDK for Model Context Protocol servers and clients

ThinkLite-VL. Python

Awesome-RL-based-Reasoning-MLLMs. This repository provides valuable reference for researchers in the field of multimodality, please start your exploratory travel in RL-based Reasoning MLLMs!

EasyControl. Implementation of "EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer"(ICCV2025)

Seed-Thinking-v1.5.

PPE. Jupyter Notebook

multimodal_rewardbench. Multimodal RewardBench

Liger-Kernel. Efficient Triton Kernels for LLM Training

MM-EUREKA. MM-EUREKA: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning

sokoban-solver. 推箱子解决方案 sokoban AI solver

VAGEN. World model reasoning RL for multi-turn VLM agents

Search-R1. Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

DAPO. An Open-source RL System from ByteDance Seed and Tsinghua AIR

Video-R1. Video-R1: Reinforcing Video Reasoning in MLLMs [🔥the first paper to explore R1 for video]

radgraph. Python

GREEN. [EMNLP, Findings 2024] a radiology report generation metric that leverages the natural language understanding of language models to identify and explain clinically significant errors in candidate reports

cosmos-reason1. Cosmos-Reason1 models understand the physical common sense and generate appropriate embodied decisions in natural language through long chain-of-thought reasoning processes.

CLIP_benchmark. CLIP-like model evaluation

Self-rewarding-reasoning-LLM. Recipes to train the self-rewarding reasoning LLMs.

understand-r1-zero. Understanding R1-Zero-Like Training: A Critical Perspective

sweet_rl. Benchmark and research code for the paper SWEET-RL Training Multi-Turn LLM Agents onCollaborative Reasoning Tasks

open_clip. An open source implementation of CLIP (With TULIP Support)

Prism. A Framework for Decoupling and Assessing the Capabilities of VLMs

Dynasor. [NeurIPS 2025] Simple extension on vLLM to help you speed up reasoning model without training.

atom. [NeurIPS 2025] Atom of Thoughts for Markov LLM Test-Time Scaling

Open-Reasoner-Zero. Official Repo for Open-Reasoner-Zero

VisualSketchpad. Codes for Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models

PixelRefer. The code for PixelRefer & VideoRefer

RAGEN. RAGEN leverages reinforcement learning to train LLM reasoning agents in interactive, stochastic environments.

VideoGLaMM. [CVPR 2025 🔥]A Large Multimodal Model for Pixel-Level Visual Grounding in Videos

groundingLMM. [CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

camel. 🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

owl. 🦉 OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation

VisualThinker-R1-Zero. Explore the Multimodal “Aha Moment” on 2B Model

Artemis. [NeurIPS 2024] Artemis: Towards Referential Understanding in Complex Videos

densecap. Dense image captioning in Torch

POPE. The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Language Models''

ALIA. Augmenting with Language-guided Image Augmentation (ALIA)

open-infra-index. Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

EasyOCR. Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

X-R1. minimal-cost for training 0.5B R1-Zero

Awesome-ML-SYS-Tutorial. My learning notes for ML SYS.

lmms-eval. One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

LIMR. Python

MoBA. MoBA: Mixture of Block Attention for Long-Context LLMs

flashinfer. FlashInfer: Kernel Library for LLM Serving

LLaVA-MORE. [ICCVW 25] LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuning

ubuntu-slurm. Steps to create a small slurm cluster with GPU enabled nodes

rllm. Democratizing Reinforcement Learning for LLMs

ReasonFlux. [NeurIPS 2025 Spotlight] LLM post-training suite — featuring ReasonFlux, ReasonFlux-PRM, and ReasonFlux-Coder.

T1. RL Scaling and Test-Time Scaling (ICML'25)

open-r1. Fully open reproduction of DeepSeek-R1

demystify-long-cot. Python

MMVP. Python

open-r1-multimodal. A fork to add multimodal model training to open-r1

R1-V. Witness the aha moment of VLM with less than $3.

peft. 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

SoM. [arXiv 2023] Set-of-Mark Prompting for GPT-4V and LMMs

simpleRL-reason. Simple RL training for reasoning

MAVIS. [ICLR 2025] Mathematical Visual Instruction Tuning for Multi-modal Large Language Models

gRefCOCO. A benchmark dataset for GREx: GRES, GREC, and GREG [CVPR 2023 & IJCV 2026]

Ref-L4. Evaluation code for Ref-L4, a new REC benchmark in the LMM era

TinyZero. Minimal reproduction of DeepSeek R1-Zero

ml-ferret. Python

anole. [Extended verision ICLR 2025 Blog Track] Anole: An Open, Autoregressive and Native Multimodal Models for Interleaved Image-Text Generation

MiniMax-01. The official repo of MiniMax-Text-01 and MiniMax-VL-01, large-language-model & vision-language-model based on Linear Attention

Kimi-k1.5.

DeepSeek-R1.

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

vllm. A high-throughput and memory-efficient inference and serving engine for LLMs

M3CoT. Python

thinking-in-space. Official repo and evaluation implementation of VSI-Bench

coconut. Training Large Language Model to Reason in a Continuous Latent Space

Visual-CoT. [Neurips'24 Spotlight] Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning

instruct-pix2pix. Python

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

PRIME. Scalable RL solution for advanced reasoning of language models

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

SkyThought. Sky-T1: Train your own O1 preview model within $450

FasterViT. [ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

Slow_Thinking_with_LLMs. A series of technical report on Slow Thinking with LLM

Virgo. Official code of *Virgo: A Preliminary Exploration on Reproducing o1-like MLLM*

VLMEvalKit. Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

IG-LLM. Code for "Re-Thinking Inverse Graphics With Large Language Models"; TMLR 2024

SWE-Gym. Code for Paper: Training Software Engineering Agents and Verifiers with SWE-Gym [ICML 2025]

Mask2Former. Code release for "Masked-attention Mask Transformer for Universal Image Segmentation"

prismer. The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

CrossFlow. [CVPR2025] PyTorch-based reimplementation of CrossFlow, as proposed in 'Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolution'

taskonomy. Taskonomy: Disentangling Task Transfer Learning [Best Paper, CVPR2018]

BLINK_Benchmark. This repo contains evaluation code for the paper "BLINK: Multimodal Large Language Models Can See but Not Perceive". https://arxiv.org/abs/2404.12390 [ECCV 2024]

RADIO. Official repository for "AM-RADIO: Reduce All Domains Into One"

LibreChat. Enhanced ChatGPT Clone: Features Agents, MCP, Skills, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active

stream-of-search. Repository for the paper Stream of Search: Learning to Search in Language

FreeNoise. [ICLR 2024] Code for FreeNoise based on VideoCrafter

RAM. A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).

lingua. Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

LVOS.

MOSE-api. [ICCV 2023] MOSE: A New Dataset for Video Object Segmentation in Complex Scenes

kinetics-dataset. Shell

sd-scripts. Python

Megatron-LM. Ongoing research training transformer models at scale

x-flux. Python

fluxgym. Dead simple FLUX LoRA training UI with LOW VRAM support

sam-home. Segment Anything (SAM) at Home web app using Gradio

huggingface_hub. The official Python client for the Hugging Face Hub.

segment-anything-demo-react-fastapi. This repository is a sample implementation of frontend/backend using SAM code from meta.

meta-sam-demo. Meta's Segment Anything Model (SAM) Demo Site

PyramidDrop. (CVPR 2025) PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

nerfstudio. A collaboration friendly studio for NeRFs

nerf. Code release for NeRF (Neural Radiance Fields)

OpenVid-1M. [ICLR 2025] OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generation

Draw-and-Understand. [ICLR2025] Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

Osprey. [CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"