vllm. A high-throughput and memory-efficient inference and serving engine for LLMs

github.com/hiyouga/vllm
by @hiyouga

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.