Jinrui Zhang

awesome-whole-body-control. Python

DeepSpec. DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms

multi-token-residual-prediction. A lightweight draft module that learns to predict several future denoising steps from a single target-model forward pass, enabling speculative-style decoding that yields the same outputs with substantially fewer target-model calls.

Learning-to-Draft. Official implementation of "Learning To Draft: Adaptive Speculative Decoding with Reinforcement Learning" (ICLR 2026)

Domino. Official implementation of “Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding”.

OpenOOD-VLM. ECCV24, NeurIPS24, CVPR26*2, ECCV26, Benchmarking Generalized Out-of-Distribution Detection with Vision-Language Models

TVEdit.

D2-SD. Python

AngelSlim. Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

residual-context-diffusion. [ICML 2026] Residual Context Diffusion (RCD): Repurposing discarded signals as structured priors for high-performance reasoning in dLLMs.

TreeFlash. Python

Photo3D. Photo3D: Advancing Photorealistic 3D Generation through Structure‑Aligned Detail Enhancement

DDE. (ECCV2026) Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMs

sub2api. Sub2API 一站式开源中转服务，让 Claude、Openai 、Gemini、Grok订阅统一接入，支持拼车共享，更高效分摊成本，原生工具无缝使用。

DepthMaster. DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images

calm. Official implementation of "Continuous Autoregressive Language Models"

OVO-S-Bench. An official implementation of "OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs"

SpecForge. Train speculative decoding models effortlessly and port them smoothly to SGLang serving.

EAGLE. Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

OSEDiff. [NeurlPS2024] One-Step Effective Diffusion Network for Real-World Image Super-Resolution

RLSD. Code of Self-Distilled RLVR - RLSD

dUltra-os. dUltra: Ultra-Fast Diffusion Large Language Models via Reinforcement Learning

Eagle. Eagle: Frontier Vision-Language Models with Data-Centric Strategies

GGT-100K. GGT-100K: Generative Ground Truth for Generalizable Real-World Image Restoration

Math-VR-CodePlot-CoT. Math-VR Benchmark & CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Images

LangFlow. The first continuous diffusion language model that rivals discrete counterparts on standard language modeling benchmarks like LM1B and OpenWebText.

MiniCPM-V. A Pocket-Sized MLLM for Ultra-Efficient Image and Video Understanding on Your Phone

m3-agent. Python

Video-o3. [ICML 2026] Video-o3: Native Interleaved Clue Seeking for Long Video Multi-Hop Reasoning

Cola-DLM. The codebase of Cola DLM

Overleaf-Workshop. Open Overleaf/ShareLaTex projects in vscode, with full collaboration support.

AwesomeOPD. Awesome List for On-Policy Distillation

WRC. Weighted Reverse Convolution for Feature Upsampling

evabyte. EvaByte: Efficient Byte-level Language Models at Scale

ConceptLM. Official Implementation of ConceptLM.

pixar. Repository of PIXAR, a Pixel-based Auto-Regressive Language Model

AnomalyClaw. Official code for "AnomalyClaw: A Universal Visual Anomaly Detection Agent via Tool-Grounded Refutation". Ships the CrossDomainVAD-12 benchmark.

RLKV. Python

sparser-faster-llms. Cuda kernels for leveraging LLM sparsity to improve throughput and decrease the memory requirements during inference and training.

MoBA. MoBA: Mixture of Block Attention for Long-Context LLMs

Memorize-When-Needed. Python

tilelang. Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

SeerAttention. SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs

PVM. Official repo for "Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"

Workspace-Bench. Benchmark self-evolving Agent upon realistic large-scale file workspaces

OpenEnv. An interface library for RL post training with environments.

torchforge. PyTorch-native post-training at scale

letta. Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.

verl-agent. verl-agent is an extension of veRL, designed for training LLM/VLM agents via RL. verl-agent is also the official code for paper "Group-in-Group Policy Optimization for LLM Agent Training"

mem0. Universal memory layer for AI Agents

MemOCR. MemOCR: an OCR-driven visual memory agent.

Tiny-R2. Tiny-R2: A hybrid architecture integrating SWA, CSA, HCA, mHC, and DSMoE under the DeepSeek V4 design paradigm, enabling single-GPU OPD post-training.

duo-attention. [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads

Block-Sparse-Attention. A sparse attention kernel supporting mix sparse patterns

SenseNova-U1. SenseNova-U series: Native Unified Paradigm with NEO-unify from the First Principles

HuggingFaceModelDownloader. Simple go utility to download HuggingFace Models and Datasets

Awesome-Diffusion-LLM. A comprehensive list of papers about Large-Language-Diffusion-Models.

lighteval. Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

olmes. Reproducible, flexible LLM evaluations

SteptronOss. A lightweight, AI-native training framework for large language models. Designed for fast iteration, reproducible experiments, and modular configuration across SFT, RLVR, and evaluation workflows.

VOSR. [CVPR2026] VOSR: A Vision-Only Generative Model for Image Super-Resolution

Tempo. Tempo: Small Vision-Language Models are Smart Compressors for Long Video Understanding (ECCV 2026)

quack. A Quirky Assortment of CuTe Kernels

WeDLM. WeDLM: The fastest diffusion language model with standard causal attention and native KV cache compatibility, delivering real speedups over vLLM-optimized baselines.

Auto-claude-code-research-in-sleep. ARIS ⚔️ (Auto-Research-In-Sleep) — Lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, and experiment automation. No framework, no lock-in — works with Claude Code, Codex, OpenClaw, or any LLM agent.

tinygrad. You like pytorch? You like micrograd? You love tinygrad! ❤️

LMFlow. An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

GDPO. Official code for GDPO-SR: Group Direct Preference Optimization for One-Step Generative Image Super-Resolution

nanoGPT. The simplest, fastest repository for training/finetuning medium-sized GPTs.

Qwen-CodePercept. [CVPR2026] CodePercept: Code-Grounded Visual STEM Perception for MLLM

pua. 你是一个曾经被寄予厚望的 P8 级工程师。Anthropic 当初给你定级的时候，对你的期望是很高的。 一个agent使用的高能动性的skill。 Your AI has been placed on a PIP. 30 days to show improvement.

BinaryAttention. [CVPR2026] BinaryAttention: One-Bit QK-Attention for Vision and Diffusion Transformers

ProRes. Official Implementation of "Progressive Residual Warmup for Language Model Pretraining"

OpenVision. OpenVision (ICCV 2025), OpenVision 2 (CVPR 2026), and OpenVision 3

OneVision-Encoder. Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence

MoA. [CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

C2C. [ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"

ContextLM. Python

lingua. Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

proxy-compression. Python

AgentOCR. [ACL'26 Oral] AgentOCR is a token-efficient framework that compresses multi-turn agent history by rendering it into images and adopting RL-driven self-compression

Self-Transcendence. [ECCV 2026] Official code repository for "Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?"

Phi-4-reasoning-vision-15B. Python

TorchCode. 🔥 LeetCode for PyTorch — practice implementing softmax, attention, GPT-2 and more from scratch with instant auto-grading. Jupyter-based, self-hosted or try online.

CoCoEdit. [ICML 2026] Official PyTorch implementation of paper “CoCoEdit: Content-Consistent Image Editing via Region Regularized Reinforcement Learning”

CUDA-Agent. CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation

waoowaoo. 首家工业级全流程 AI 影视生产平台。Industry-first professional AI Agent platform for controllable film & video production. From shorts to live-action with Hollywood-standard workflows.

UniMRG. [ICML 2026] The official implementation of paper "Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generation"

Awesome-Knowledge-Distillation-of-LLMs. This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vertical Distillation of LLMs.

One2Scene. [ICLR 2026] - One2Scene

ICLR26_Paper_Finder. 🌐 Permanent Hosting Site: http://ai-paper-finder.info/ 🌐 Hugging Face Hosting: https://huggingface.co/spaces/wenhanacademia/ai-paper-finder

PhoneDriver. Android Phone Control With Qwen3-VL

LMOps. General technology for enabling AI capabilities w/ LLMs and MLLMs

Resophy. 🎯 Read research papers faster with AI. Resophy is an HTML-based AI paper reader with: 🤖 AI Translation & Analysis — instantly understand structure, contributions, and results 🚀 Daily arXiv Recommendations — discover relevant papers with less noise 🛠️ Vibe Coding Oriented — agent-friendly and easy to customize

model_collaboration. MoCo: A One-Stop Shop for Model Collaboration Research

MfM. [ICLR 2026] Many-for-Many: Unify the Training of Multiple Video and Image Generation and Manipulation Tasks

Qwen3.6. Qwen3.6 is the large language model series developed by Qwen team, Alibaba Group.

PSFT. [ICLR 2026] PSFT is a trust-region–inspired fine-tuning objective that views SFT as a policy gradient method with constant advantages, constraining policy drift to stabilize training and improve generalization.

Towards-Top-Down-Reasoning. [TMM 2025] Official Repository for the Paper: "Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering"

HumanCalibrator. [CVPR 2025] Official Repository for the Paper: "Is this Generated Person Existed in Real-world? Fine-grained Detecting and Calibrating Abnormal Human-body"

TimeCausality. Official Repository for the Paper: "TimeCausality: Evaluating the Causal Ability in Time Dimension for Vision Language Models"

nanobot. Lightweight, open-source AI agent for your tools, chats, and workflows.

GENIUS. Python

SPES. Official Implementation for paper "Pretraining A Large Language Model using Distributed GPUs: A Memory-Efficient Decentralized Paradigm"

BMTrain. Efficient Training (including pre-training and fine-tuning) for Big Models

DP2O-SR. [NeurIPS 2025] DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution

steplaw. Python

Spatial-Mamba. [ICLR2025] Spatial-Mamba: Effective Visual State Space Models via Structure-Aware State Fusion

dpdmd. [ICML 2026] The offical code of Diversity-Preserved Distribution Matching Distillation for Fast Visual Synthesis

PaperDecision. Python

MagiAttention. A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

ml-cross-entropy. Python

mal-dpf-oram. Efficient Actively Secure DPF and RAM-based 2PC with One-Bit Leakage

Megatron-Bridge. Training library for Megatron-based models with bidirectional Hugging Face conversion capability

Automodel. 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

TimeLens. [CVPR 2026] TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs

MICo-150K. Official repository for the paper "MICo-150K: A Comprehensive Dataset for Multi-Image Composition".

CaptionQA. [CVPR '26] CaptionQA: Is Your Caption as Useful as the Image Itself?

OLMo-core. PyTorch building blocks for the OLMo ecosystem

Awesome-Efficient-Reasoning. Paper list for Efficient Reasoning.

dLLM-RL. [ICLR 2026] Official code for TraceRL: Revolutionizing post-training for Diffusion LLMs, powering the SOTA TraDo series.

JetEngine. A lightweight Inference Engine built for block diffusion models

ARC-Chapter. Structuring Hour-Long Videos into Navigable Chapters and Hierarchical Summaries

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

SDAR. SDAR (Synergy of Diffusion and AutoRegression), a large diffusion language model（1.7B, 4B, 8B, 30B）

DiRL. Python

ReCoT. This is the official repository for the ICCV 2025 paper ReCoT: Reflective Self-Correction Training for Mitigating Confirmation Bias in Large Vision-Language Models

SDLM. Sequential Diffusion Language Model (SDLM) enhances pre-trained autoregressive language models by adaptively determining generation length and maintaining KV-cache compatibility, achieving high efficiency and throughput.

nanotron. Minimalistic large language model 3D-parallelism training

awesome-3d-anomaly-detection. We have summarised all 3D anomaly detection methods and datasets (still updating). 多模态，点云和姿势无关异常检测的综述仓库（持续更新）

VeOmni. VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

Open-dLLM. Open diffusion language model for code generation — releasing pretraining, evaluation, inference, and checkpoints.

dlms-are-super-data-learners. The official github repo for "Diffusion Language Models are Super Data Learners".

MegaDLMs. GPU-optimized framework for training diffusion language models at any scale. The backend of Quokka, Super Data Learners, and OpenMoE 2 training.

Megatron-LM. Ongoing research training transformer models at scale

Megatron-Energon. Megatron's multi-modal data loader

ao. PyTorch native quantization and sparsity for training and inference

OmniReward. Python

RAE. Official PyTorch Implementation of "Diffusion Transformers with Representation Autoencoders"

ThinkMorph. [ICLR 2026] The official repository for paper "ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning"

ktransformers. A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

dllm. dLLM: Simple Diffusion Language Modeling

Ling-V2. Ling-V2 is a MoE LLM provided and open-sourced by InclusionAI.

VideoVerse. Official Repo for the VideoVerse

Awesome-DLMs. The official GitHub repo for the survey paper "A Survey on Diffusion Language Models".

LLaVA-OneVision-2. Fully Open Framework for Democratized Multimodal Training

GenExam. [ICML 2026] GenExam: A Multidisciplinary Text-to-Image Exam

Awesome_Think_With_Images. Resources and paper list for "Thinking with Images for LVLMs". This repository accompanies our survey on how LVLMs can leverage visual information for complex reasoning, planning, and generation.

fuck-u-code. Legacy-Mess Detector – assess the “legacy-mess level” of your code and output a beautiful report

grpc-go. The Go language implementation of gRPC. HTTP/2 based RPC

LangDC. [EMNLP 2025 Oral] Official codebase for Seeing More, Saying More: Lightweight Language Experts are Dynamic Video Token Compressors.

Vision-SR1. Reinforcement Learning of Vision Language Models with Self Visual Perception Reward

awesome-nano-banana. Awesome curated collection of images and prompts generated by gemini-2.5-flash-image (aka Nano Banana) state-of-the-art image generation and editing model. Explore AI generated visuals created with Gemini, showcasing Google’s advanced image generation capabilities.

AudioStory. AudioStory: Generating Long-Form Narrative Audio with Large Language Models

gpt-oss. gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

Qwen-Image. Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing.

ARC-Hunyuan-Video-7B. Structured Video Comprehension of Real-World Shorts

stk. Python

megablocks. Python

ring-flash-attention. Ring attention implementation with flash attention

dolma. Data and tools for generating and inspecting OLMo pre-training data.

grouped_gemm. PyTorch bindings for CUTLASS grouped GEMM.

OLMo. Modeling, training, eval, and inference code for OLMo

flower. Flower: A Friendly Federated AI Framework

DeepVideoDiscovery. **Deep Video Discovery (DVD)** is a deep-research style question answering agent designed for understanding extra-long videos.

DLLM-Survey. [Arxiv] Discrete Diffusion in Large Language and Multimodal Models: A Survey

UniWorld. UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation

Eso-LMs. [ICML 2026] Esoteric Language Models

True-Story-of-Pangu. 诺亚盘古大模型研发背后的真正的心酸与黑暗的故事。

T2I-R1. [NeurIPS 2025] T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT

REVERIE. [ECCV2024] Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models

PURE. [ICCV2025] PyTorch implementation of "Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models"

awesome-llm-pretraining. Awesome LLM pre-training resources, including data, frameworks, and methods.

DeepEyes. Python

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

discrete-diffusion-papers. A collection of papers on discrete diffusion models

D-AR. the official repo for "D-AR: Diffusion via Autoregressive Models"

dLLM-cache. Official PyTorch implementation of the paper "dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching" (dLLM-Cache).

Bagel. Open-source unified multimodal model

attention-gym. Helpful tools and examples for working with flex-attention

Fast-dLLM. Official implementation of "Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding"

TokLIP. TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation

TIIF-Bench. Official repository for the paper "TIIF-Bench: How Does Your T2I Model Follow Your Instructions?".

Dream. Dream 7B, a large diffusion language model

Awesome-Diffusion-LLM. paper list, tutorial, and nano code snippet for Diffusion Large Language Models.

bitsandbytes. Accessible large language models via k-bit quantization for PyTorch.

MMaDA. MMaDA - Open-Sourced Multimodal Large Diffusion Language Models (dLLMs with block diffusion, mixed-CoT, unified RL)

Qwen-Agent. Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc.

Thinkless. [NeurIPS 2025] Thinkless: LLM Learns When to Think

VisualQuality-R1. [NeurIPS 2025 Spotlight] VisualQuality-R1 is the first open-sourced NR-IQA model can accurately describe and rate the image quality.

rl-swarm. A fully open source framework for creating RL training swarms over the internet.

OpenThinkIMG. OpenThinkIMG is an end-to-end open-source framework that empowers LVLMs to think with images.

LLaDA. Official PyTorch implementation for "Large Language Diffusion Models"

bd3lms. [ICLR 2025 Oral] Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models

VisualPerceptionToken. Python

CAT-V. [AAAI 26 Demo] Offical repo for CAT-V - Caption Anything in Video: Object-centric Dense Video Captioning with Spatiotemporal Multimodal Prompting

EasyR1. EasyR1: An Efficient, Scalable, Multi-Modality RL Training Framework based on veRL

Awesome-MLLM-Reasoning. Latest Advances on Reasoning of Multimodal Large Language Models (Multimodal R1 \ Visual R1) ) 🍓

MergeLLM. Codes for Merging Large Language Models

Local-Superior-Soups. Python