semantic router

By comparing input embeddings against pre-defined route embeddings at millisecond latency, it provides intent classification suitable for real-time routing in production AI systems.

The library is designed as a lightweight layer that sits in front of LLM-powered applications, making coarse-grained decisions about input handling before more expensive processing.

Routes are defined with example utterances that the system uses to build embedding-based classifiers a route for customer support queries, another for technical documentation, another for billing questions and incoming messages are matched to the best route using configurable similarity thresholds.

This reduces LLM API calls for high-volume applications where many inputs can be handled by deterministic rules or lightweight models rather than frontier LLMs.

Semantic Router is open-source under the MIT license and integrates with OpenAI, Cohere, Hugging Face, and local embedding models for the embedding component.

It supports dynamic routes that include LLM-generated responses, static routes that return fixed content, and function routes that call arbitrary Python code, covering the full spectrum of routing needs in production AI systems.

The library is used in chatbots, voice assistants, and multi-modal AI applications where intent classification speed and cost are critical performance factors.