Weizhi Wang

humanize. From Automated Idea Factory to Realization

mbridge. Bridge Megatron-Core to Hugging Face/Reinforcement Learning

openclaw. Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

gaac. Claude Code Methodology | GAAC=Github-as-a-Context | LLM IS AS GOOD AS YOUR ARE | 胸有成码

CausVid. (CVPR 2025) From Slow Bidirectional to Fast Autoregressive Video Diffusion Models

FastVideo. A unified inference and post-training framework for accelerated video generation.

Self-Forcing. Official codebase for "Self Forcing: Bridging Training and Inference in Autoregressive Video Diffusion" (NeurIPS 2025 Spotlight)

LaCT. Code release for paper "Test-Time Training Done Right"

llm_interview_note. 主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题

Direct-Multitoken-Decoding. The official repo for the paper Direct Multi-token Decoding

google-search. A Playwright-based Node.js tool that bypasses search engine anti-scraping mechanisms to execute Google searches. Local alternative to SERP APIs with MCP server integration.

MiroFlow. 🏆 Top-1 on 5+ benchmarks | Web UI | Supports MiroThinker, Claude, Kimi, OpenAI

MiroThinker. MiroThinker is a deep research agent optimized for complex research and prediction tasks. Our latest models, MiroThinker-1.7, achieves 74.0 and 75.3 on the BrowseComp and BrowseComp Zh, respectively.

DeepEyes. Python

sglang. SGLang is a high-performance serving framework for large language models and multimodal models.

deep_research_bench. DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents

AWorld. Search, understand, reproduce, and improve an idea with ease

SwanLab. ⚡️SwanLab - an open-source, modern-design AI training tracking and visualization tool. Supports Cloud / Self-hosted use. Integrated with PyTorch / Transformers / verl / LLaMA Factory / ms-swift / Ultralytics / MMEngine / Keras etc.

Second-Me. Train your AI self, amplify you, bridge the world

imagehash. A Python Perceptual Image Hashing Module

Search-R1. Search-R1: An Efficient, Scalable RL Training Framework for Reasoning & Search Engine Calling interleaved LLM based on veRL

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

A-Guide-to-Retrieval-Augmented-LLM. an intro to retrieval augmented large language model

codex. Lightweight coding agent that runs in your terminal

functionary. Chat language model that can use tools and interpret the results

open-deep-research. An open source deep research clone. AI Agent that reasons large amounts of web data extracted with Firecrawl

servers. Model Context Protocol Servers

Open-Qwen2VL. [COLM 2025] Open-Qwen2VL: Compute-Efficient Pre-Training of Fully-Open Multimodal LLMs on Academic Resources

ByteCheckpoint. ByteCheckpoint: An Unified Checkpointing Library for LFMs

Awesome-Unified-Multimodal-Models. 📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

android_world. AndroidWorld is an environment and benchmark for autonomous agents

google-research. Google Research

native-sparse-attention. 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"

open-infra-index. Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

VLM-R1. Solve Visual Understanding with Reinforced VLMs

RedStone. The RedStone repository includes code for preparing extensive datasets used in training large language models.

MemoryLLM. The official implementation of the ICML 2024 paper "MemoryLLM: Towards Self-Updatable Large Language Models" and "M+: Extending MemoryLLM with Scalable Long-Term Memory"

LongGenBench. Jupyter Notebook

lm-evaluation-harness. A framework for few-shot evaluation of language models.

LLaVA-Unified. Python

MAmmoTH-VL. (ACL 2025) MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale

MMSci. MMSci: A Multimodal Multi-Discipline Dataset for PhD-Level Scientific Comprehension

Awesome-Video-Datasets. Video datasets

LongMemEval. Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025)

Modality-Integration-Rate. [ICCV 2025] The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".

TikTalk. Python

MemLong. Python

instruct-eval. This repository contains code to quantitatively evaluate instruction-tuned models such as Alpaca and Flan-T5 on held-out tasks.

faiss. A library for efficient similarity search and clustering of dense vectors.

donut. Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

OCRDatasets. A collection of OCR-related datasets

Make-A-Scene. Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

llm-awq. [MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

DeCo. Code for DeCo: Decoupling token compression from semanchc abstraction in multimodal large language models

LaViA. Python

MINT-1T. 🍃 MINT-1T: A one trillion token multimodal interleaved dataset.

retrieval-scaling. Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".

video2dataset. Easily create large video dataset from video urls

how2-dataset. This repository contains code and metadata of How2 dataset

kNN-CUDA. Fast k nearest neighbor search using GPU

dclm. DataComp for Language Models

LLaVA-NeXT. Python

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning: LLaVA (Large Language-and-Vision Assistant) built towards GPT-4V level capabilities.

SoM-LLaVA. [COLM-2024] List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs

ann-benchmarks. Benchmarks of approximate nearest neighbor libraries in Python

VILA. VILA is a family of state-of-the-art vision language models (VLMs) for diverse multimodal AI tasks across the edge, data center, and cloud.

prismatic-vlms. A flexible and efficient codebase for training visually-conditioned language models (VLMs)

scaling_on_scales. When do we not need larger vision models?

Open-Sora. Open-Sora: Democratizing Efficient Video Production for All

grok-1. Grok open release

Long-Context-Data-Engineering. Implementation of paper Data Engineering for Scaling Language Models to 128K Context

MLM_Filter. Official implementation of our paper "Finetuned Multimodal Language Models are High-Quality Image-Text Data Filters".

RLHF-V. [CVPR'24] RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

InternLM-XComposer. InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

OBELICS. Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

Awesome-LLMs-for-Video-Understanding. 🔥🔥🔥 [IEEE TCSVT] Latest Papers, Codes and Datasets on Vid-LLMs.

Awesome-LLM-Long-Context-Modeling. 📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

open_flamingo. An open-source framework for training large multimodal models.

awesome-llm-role-playing-with-persona. Awesome-llm-role-playing-with-persona: a curated list of resources for large language models for role-playing with assigned personas

video-subtitle-extractor. 视频硬字幕提取，生成srt文件。无需申请第三方API，本地实现文本识别。基于深度学习的视频字幕提取框架，包含字幕区域检测、字幕内容提取。A GUI tool for extracting hard-coded subtitle (hardsub) from videos and generating srt files.

TensorRT-LLM. TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

LVIS-INSTRUCT4V.

faiss_tips. Some useful tips for faiss

flash-attention. Fast and memory-efficient exact attention

llm-foundry. LLM training code for Databricks foundation models

InstructionGPT-4. InstructionGPT-4

MiniGPT-4. Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

Awesome-Multimodal-Large-Language-Models. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models

chat2api. chatgpt接口模拟API接口转换网关

alpagasus. Unofficial implementation of AlpaGasus

GRiT. GRiT: A Generative Region-to-text Transformer for Object Understanding (ECCV2024)

MetaCLIP. NeurIPS 2025 Spotlight; ICLR2024 Spotlight; CVPR 2024; EMNLP 2024

datacomp. DataComp: In search of the next generation of multimodal datasets

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

t5x. Python

BLIP. PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

NextStepPredictor. Python

github-trending-backup. Github trending backup by everyday.

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

LongMem. Official implementation of our NeurIPS 2023 paper "Augmenting Language Models with Long-Term Memory".

conceptnet. ConceptNet: a semantic network of common sense knowledge

VL-CheckList. Evaluating Vision & Language Pretraining Models with Objects, Attributes and Relations. [EMNLP 2022]

guidance. A guidance language for controlling large language models.

xformers. Hackable and optimized Transformers building blocks, supporting a composable construction.

FLAIR. Python

web-qa. Python

mmc4. MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

AutoGPT. AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

Alpaca-CoT. We unified the interfaces of instruction-tuning data (e.g., CoT data), multiple LLMs and parameter-efficient methods (e.g., lora, p-tuning) together for easy use. We welcome open-source enthusiasts to initiate any meaningful PR on this repo and integrate as many LLM related technologies as possible. 我们打造了方便研究人员上手和使用大模型等微调平台，我们欢迎开源爱好者发起任何有意义的pr！

transformers. 🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.

langchain. The agent engineering platform.

RWKV-LM. RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

chatgpt-retrieval-plugin. The ChatGPT Retrieval Plugin lets you easily find personal or work documents by asking questions in natural language.

metaseq. Repo for external large-scale work

llama.cpp. LLM inference in C/C++

evals. Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

chain-of-thought-hub. Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

stanford_alpaca. Code and documentation to train Stanford's Alpaca models, and generate the data.

Open-Instruction-Generalist. Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks

Edgeformers. Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks (ICLR 2023)

fastText. Library for fast text representation and classification.

llama. Inference code for Llama models

chatgpt-python. Unofficial Python SDK for OpenAI's ChatGPT

robustlearn. Robust machine learning for responsible AI

bigscience. Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

bi-att-flow. Bi-directional Attention Flow (BiDAF) network is a multi-stage hierarchical process that represents context at different levels of granularity and uses a bi-directional attention flow mechanism to achieve a query-aware context representation without early summarization.

attentions. PyTorch implementation of some attentions for Deep Learning Researchers.

active-example-selection. Active Example Selection for In-Context Learning (EMNLP'22)

readpilot. Read Pilot analyzes online articles and generate Q&A cards for you. Powered by OpenAI & Next.js.

ActGPT. chatbot does what you ask, like open Google search, post a Tweet, etc.

Tune-A-Video. Unofficial implementation of Tune-A-Video

tuning_playbook. A playbook for systematically maximizing the performance of deep learning models.

mesh-transformer-jax. Model parallel transformers in JAX and Haiku

long-summarization. Resources for the NAACL 2018 paper "A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents"

CS595N.

naacl2021-longdoc-tutorial. Python

memory-augmented-transformers. Repo for testing an integration of Memorizing Transformers into pretrained models

meliad. Python

text-to-text-transfer-transformer. Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

WebQA. Shell

x-transformers. A concise but complete full-attention transformer with a set of promising experimental features from various papers

NPDA-KNN-ST. Official implementation of EMNLP'2022 paper "Non-Parametric Domain Adaptation for End-to-End Speech Translation"

Experiments-on-DF-Net-and-GPT-3. Code for "DAISY: An Implementation of Five Core Principles for Transparent and Accountable Conversational AI" 2022. Fork from ACL 2020 Paper "Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog"

dedupe. :id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

fairseq. rebert model codes based on fariseq

tstl_t5_bias. This is our implementation of the T5 bias for fairseq.

dama. CS 3110 final project

TurkishDraughts. A collection of ai, and guis for turkish draughts

GLM-130B. GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)

multiwoz. Source code for end-to-end dialogue model from the MultiWOZ paper (Budzianowski et al. 2018, EMNLP)

memorizing-transformers-pytorch. Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

gpt-2-simple. Python package to easily retrain OpenAI's GPT-2 text-generating model on new texts

vokenization. PyTorch code for EMNLP 2020 Paper "Vokenization: Improving Language Understanding with Visual Supervision"

pg19.

openwebtext. Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.

MIT-Linear-Algebra-Notes. MLNLP: Notes for MIT-Linear-Algebra

VL-commonsense. Python

matplotlib-gallery. Examples of matplotlib codes and plots

webvid. Large-scale text-video dataset. 10 million captioned short videos.

book. 此处收编常用基础书籍

visualization. a collection of visualization function

distributed-faiss. A library for building and serving multi-node distributed faiss indices.

VaLM. VaLM: Visually-augmented Language Modeling. ICLR 2023.

simmc2. Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

nlpapereading.

spatial-commonsense. Source code and data for Things not Written in Text: Exploring Spatial Commonsense from Visual Signals (ACL2022 main conference paper).

GLM. GLM (General Language Model)

CLIP. CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

ColorSurvey. This repository is for "Image Colorization: A Survey and Dataset" paper

bflm. Python

VQA. Python

Conference-Acceptance-Rate. Acceptance rates for the major AI conferences

neurst. Neural end-to-end Speech Translation Toolkit

torch_tem. Implementation of the Tolman Eichenbaum Machine in pytorch

rasa. 💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants

GALAXY. Official repository of the AAAI'2022 paper "GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection"

tod_as_nlg. Official implementation of SIGIR 2022 Paper "Task-Oriented Dialogue System as Natural Language Generation".

E2E-ST-TDA. Official implementation of AAAI'2022 paper "Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement"

UDA-KNN. Python

zs-nmt-dae. Official implementation of EMNLP 2021 Paper "Rethinking Zero-shot Neural Machine Translation: From a Perspective of Latent Variables"

KSTER. code and data for paper "Learning Kernel-Smoothed Machine Translation with Retrieved Examples"

aeda_nlp. Data augmentation for NLP, accepted at EMNLP 2021 Findings

RU-Thesis-Dissertation-Latex-Template. The LaTeX template for Rutgers MS Thesis or PhD Dissertation

wmt16-scripts. scripts and configuration files for Edinburgh neural MT submission to WMT 16 shared translation task

PromptPapers. Must-read papers on prompt-based tuning for pre-trained language models.

MultiWOZ-coref. A multi-domain task-oriented dataset enhanced with annotation corrections and co-reference annotation

LM-BFF. [ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723

adaptive-knn-mt. Python

soloist. Python

China_House. 中国买房相关资料和项目整理，方便查看，持续更新中...

dstc8-meta-dialog. Baseline implementation for DSTC8 User Response Prediction Challenge

SpeechTransProgress. Tracking the progress in end-to-end speech translation

talkthewalk. This repository provides code for reproducing experiments of the paper Talk The Walk: Navigating New York City Through Grounded Dialogue by Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela.

minecraft-dialogue. Java

ParlAI. A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

tfpyth. Putting TensorFlow back in PyTorch, back in TensorFlow (differentiable TensorFlow PyTorch adapters).

conversational-datasets. Large datasets for conversational AI

ccm. This project is a tensorflow implement of our work, CCM.

ToD-BERT. Pre-Trained Models for ToD-BERT

NeuralPipeline_DSTC8. Python