Rare find

sarathi-serve. A low-latency & high-throughput serving engine for LLMs

github.com/microsoft/sarathi-serve
by @microsoft

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.