Seonglae ChoSystem-brain routing: signal-led, entropy-aware, ruthlessly clear.
Sits between the real world and LLMs, routing each request to the most suitable model. The project aims to capture signals missing from the request/response/context and combine them to make better decisions. Critical take: the scope is extremely broad, spanning routing, caching, hallucination detection, PII protection, jailbreak defense, and self-learning. Key components include simulation via a Fleet Simulator, a model-training pipeline, and a proposal system. The approach is to manage and simulate many LLM instances as a single fleet.
vLLM Semantic Router
System Level Intelligent Router for Mixture-of-Models
https://vllm-semantic-router.com/docs/intro
vLLM Semantic Router
vLLM Semantic Router - Manage your AI-powered Intelligent Router
https://play.vllm-semantic-router.com/
White Paper — vLLM Semantic Router
Signal Driven Decision Routing for Mixture-of-Modality Models
https://vllm-semantic-router.com/white-paper/
vLLM Semantic Router
System Level Intelligent Router for Mixture-of-Models
https://vllm-semantic-router.com/