Sign in
Rare find
mini-infer.
LLM inference engine from scratch — paged KV cache, continuous batching, chunked prefill, prefix caching, speculative decoding, CUDA graph, tensor parallelism, OpenAI-compatible serving
github.com/psmarter/mini-infer
by
@psmarter
Problem
Audience
Keep exploring
Updates
Founder
0
Product
0
No founder updates yet. Check back soon.
🧪
staging