Rare find

candle-vllm. Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

github.com/EricLBuehler/candle-vllm
by @EricLBuehler

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.