Jiarui Fang（方佳瑞）

LLMSpeculativeSampling. Fast inference from large lauguage models via speculative decoding

long-context-attention. USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

LLMRoofline. Compare different hardware platforms via the Roofline Model for LLM inference tasks.

Odysseus-Transformer. Odysseus: Playground of LLM Sequence Parallelism

ChituAttention. Quantized Attention on GPU

SWCaffe. A Deep Learning Framework customized for Sunway TaihuLight

DPSKV3MFU. Estimate MFU for DeepSeekV3

Distributed-ResNet-Tensorflow. A Distributed ResNet on multi-machines each with one GPU card.

swGEMM. A highly efficient library for GEMM operations on Sunway TaihuLight

swDNN. a highly-efficient library for deep neural networks based on Sunway TaihuLight supercomputer.

PyTorchMemTracer. Depict GPU memory footprint during DNN training of PyTorch

PSTensor. PSTensor provides a way to hack the memory management of tensors in TensorFlow and PyTorch by defining your own C++ Tensor Class.

ColoBloom. Python

DeepSpeedZeRO3Benchmark. A finetuned benchmark scripts for DeepSpeed zero3 stage

SpeculativeDecodingPapers. 📰 Must-read papers and blogs on Speculative Decoding ⚡️

LLM-Viewer. Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

swDNNv1.0. A Deep Learning Library for Sunway TaihuLight

intel-baidu-allreduce. C++

crack_leetcode. 五天刷题，三天模拟！快速掌握leetcode解题套路！

TensorrtBenchmark. Benchmark bert using TensorRT

MoE-Megatron-LM. Python

horovod-resnet. Python

Communication-Efficient-DNN. Python

SMO-SVM. a python implementation of libsvm

ssh-passwd-free. Method to set passwd-free for a set of IPs

TensorNVMe. A Python library transfers PyTorch tensors between CPU and NVMe

cudaMemHook. C++

large-scale-tensorflow-benchmark. benchmark tensorflow for supercomputers

DeepGlobe. Python

MNBVC. MNBVC(Massive Never-ending BT Vast Chinese corpus)超大规模中文语料集。对标chatGPT训练的40T数据。MNBVC数据集不但包括主流文化，也包括各个小众文化甚至火星文的数据。MNBVC数据集包括新闻、作文、小说、书籍、杂志、论文、台词、帖子、wiki、古诗、歌词、商品介绍、笑话、糗事、聊天记录等一切形式的纯文本中文数据。

DTensor. Study PyTorch DTensor

ColossalAI. Colossal-AI: A Unified Deep Learning System for Big Model Era

EasyContext. Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

89757. Python

mochi1-models. The best OSS video generation models

CommTest. Test for PyTorch Async Collective Communication

KsanaLLM. C++

xDiT. xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

Wan2.1. Python

FastFold. Optimizing AlphaFold Training and Inference on GPU Clusters

verl_demo. Python

transformers. 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

getpy. A Vectorized Python Dict/Set

Awesome-LLM-System-Papers.

nnfusion. A flexible and efficient deep neural network (DNN) compiler that generates high-performance executable from a DNN model description.

feifeibear.

ckp_training. Python

ADMM-NeuralNetwork. ADMM-NeuralNetwork was implemented by a potato