vllm-jukebox. Server that multiplexes multiple LLM models through vLLM backends with automatic model swapping, multi-GPU scheduling, and graceful request draining

github.com/erans/vllm-jukebox

Problem

Audience

Updates

No founder updates yet. Check back soon.