mohit

gpt2-inference. A GPT-2 inference engine written from scratch in CUDA and C++. Implements custom CUDA kernels for tiled matrix multiplication, LayerNorm, fused attention, transformer blocks, KV cache management, autoregressive token generation, and end-to-end GPT-2 inference with profiling and benchmarking.

Memory-Allocator. Custom memory allocator in C++ built from scratch using mmap. Allocates a 1MB memory pool upfront and carves blocks from it to keep all allocations contiguous. Implements malloc, free, block reuse via free list, and coalescing of adjacent free blocks without any standard library allocation functions.

FlashAttention-CuPy. Flash Attention from scratch, tiled CUDA forward kernel, online softmax with running max and correction factor, recomputation trick in backward, O(N) memory, full forward and backward verified against PyTorch autograd to 1e-6.

kernel-fusion. A CUDA Python experiment demonstrating kernel fusion by combining ReLU and LayerNorm into a single GPU pass and comparing it against the unfused multi-kernel pipeline.

KV-Compression. Implemented and benchmarked KV cache compression methods for LLM inference in Triton. Optimized kernels for KIVI and TurboQuant with Flash Attention integration - up to 7.11× compression on GPT-2 and 7.31× on Qwen3.5 9B.

research-papers. Research implementations focused on inference efficiency and model optimization. Includes custom Triton kernels, LoRA, knowledge distillation pipelines, and more.

Ray-Tracing. Minimal C++ ray tracing implementation built from scratch. Includes a custom vec3 math library for vector operations, generates images via PPM output, and progressively builds core rendering concepts like rays, color computation, and geometry intersection.

VecEngine. A lightweight vector database, retrieval engine, and custom indexer, all built completely from scratch.

Adaptive-ViT. An adaptive Vision Transformer inference system that avoids unnecessary high-resolution computation, achieving ~3× faster inference than static high-res ViT by selectively escalating only when needed.

Fused-KV-Cache. Fused KV cache attention for single-token decode in one CUDA kernel using CuPy RawKernel. One query attending over the full KV cache, dot products, softmax, and weighted V sum computed entirely in shared memory with no score vector written to global memory. 8.5x faster than CuPy at short cache lengths, 2.5x at T_cache=1024.

Scaled_Dot_Product_Kernel. Fused causal scaled dot product attention in a single CUDA kernel using CuPy RawKernel. QK dot products, causal mask, softmax, and AV weighted sum all computed inside one block. No attention matrix written to global memory. Up to 11.6x faster than CuPy at short sequence lengths, breakeven at T=128.

tri-sds. Triton-based EAGLE speculative decoding engine for Qwen3-4B to Qwen3-32B on AMD MI300X. Matches SGLang's acceptance speedup ratios (1.56–2.49×) with fully custom Triton kernels (prefill, GQA decode, EAGLE draft attention, RMSNorm).

Autograd-NN. Implemented a fully connected autograd engine and neural network from scratch in pure Python.

Pluto. a no-code web based machine learning trainer that lets users upload CSVs, select algorithms, visualizations and train models end-to-end.

layernorm_dropout_residual_kernel. Fused LayerNorm + Dropout + Residual add in a single CUDA kernel using CuPy RawKernel. One block per token row, two shared memory reductions for mean and variance, output written in one pass with dropout and residual add folded in. Up to 11.7x faster than the unfused CuPy baseline at small sequence lengths, 2 to 3x at large sizes.

Arena-Allocator. C++ arena allocator built from scratch. Implements a fixed-size memory pool with a bump pointer, zero per-allocation overhead, and O(1) reset. Wraps into a standard library compatible allocator template so it works directly with std::vector, std::unordered_map, and any other STL container.

mlp-block_kernel. Fused MLP block optimization using a custom CUDA kernel in CuPy. Implements in-place GELU fusion between two cuBLAS GEMMs to reduce memory traffic and kernel launches, achieving measurable speedups while preserving numerical correctness.

micrograd-in-cpp. andrej karpathy's micrograd python implementation in c++

customGPT-RAG. custom RAG system that uses local document embeddings and generative AI to provide accurate, context-aware answers from private knowledge.

Fused-Cross-Attention. Python

softmax_dropout_kernel-fuse. Fused masked softmax + dropout in a single CUDA kernel using CuPy RawKernel. 3–5.7x faster than a standard CuPy multi-op baseline across sequence lengths 128–2048.