llm-server-from-scratch. FastAPI server for locally serving Gemma 3 270M & OpenAI Whisper with batched inference and streaming support.

github.com/biraj21/llm-server-from-scratch
by @biraj21

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.