DefTruth

CUDA-Learn-Notes. 📚200+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).

lite.ai.toolkit. 🛠 A lite C++ toolkit: contains 100+ Awesome AI models, support MNN, NCNN, TNN, ONNXRuntime and TensorRT. 🎉🎉

Awesome-LLM-Inference. 📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

BlogLearning. 自己的学习历程，重点包括各种好玩的图像处理算法、运动捕捉、机器学习

lite.ai.toolkit.demo. Demos for how to use the shared libs of Lite.AI.ToolKit🚀🚀🌟. (https://github.com/DefTruth/lite.ai.toolkit)

flash-attention-minimal. Flash Attention in ~100 lines of CUDA (forward pass only)

cmake-cookbook. CMake Cookbook recipes.

YOLOP. You Only Look Once for Panopitic Driving Perception.（https://arxiv.org/abs/2108.11250）

nanodet. ⚡Super fast and lightweight anchor-free object detection model. 🔥Only 980 KB(int8) / 1.8MB (fp16) and run 97FPS on cellphone🔥

DefTruth.

FastDeploy. ⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit

awesome-AI-system. paper and its code for AI System

PaddleOCR. Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

hgemm-mma. ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.

YOLOv6. YOLOv6: a single-stage object detection framework dedicated to industrial applications.

RobustVideoMatting. Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

triton. Development repository for the Triton language and compiler

MGMatting. This repository includes the official project of Mask Guided (MG) Matting, presented in our paper: Mask Guided Matting via Progressive Refinement Network

torch-tensorrt. PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

nms.lite.ai.toolkit. 🚀 A lite C++ implementation of hard_nms、soft_nms、blend_nms etc.

ComputeLibrary. The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.

pybind11-Chinese-docs. pybind11中文文档（个人翻译）

PTX-ISA-8.2-zh. 🎉持续更新：CUDA 12.2 PTX-ISA-8.2学习笔记，部分中文翻译 + 个人理解 + 内联汇编示例，讲解CUDA 12.2 PTX-ISA-8.2 汇编指令；进行中.....

Paddle-Lite. Multi-platform high performance deep learning inference engine (飞桨多端多平台高性能深度学习推理引擎）

TensorRT-Model-Optimizer. TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.

TensorRT_Tutorial. C++

TensorRT-LLM. TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

vllm. A high-throughput and memory-efficient inference and serving engine for LLMs

XNNPACK. High-efficiency floating-point neural network inference operators for mobile, server, and Web

vscode-pdfviewer. Show PDF preview in VSCode.

mediapipe. Cross-platform, customizable ML solutions for live and streaming media.

FlyCV. C++

Yolo-FastestV2. :zap: Based on Yolo's low-power, ultra-lightweight universal target detection algorithm, the parameter is only 250k, and the speed of the smart phone mobile terminal can reach ~300fps+

simde. Implementations of SIMD instruction sets for systems which don't natively support them.

PyTorchDocs. PyTorch 官方中文教程包含 60 分钟快速入门教程，强化教程，计算机视觉，自然语言处理，生成对抗网络，强化学习。欢迎 Star，Fork！

awesome-blendshapes. ❤️ 📖 A list of papers about BlendShapes😜~

ffpa-attn-mma. 📚FFPA(Split-D): Yet another Faster Flash Prefill Attention with O(1) GPU SRAM complexity for headdim > 256, ~2x↑🎉vs SDPA EA.

ChatGLM2-6B. ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

TransformerEngine. A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

flash-attention. Fast and memory-efficient exact attention