2026 — now · Author, open source
brisvox
A Rust engine for real-time voice agents: hundreds of concurrent calls in one container.
Problem
A voice agent that feels natural needs models in the loop on every call: voice activity, end-of-turn detection, speech-to-text, an LLM and text-to-speech, all while the caller may cut in at any moment. The Python frameworks I compared (Pipecat and LiveKit Agents) spend a container’s CPU or memory on a few dozen calls, and after an interruption the agent’s memory should hold only the words the caller actually heard.
System
One Rust process runs an isolated async session per call: transport in, voice activity and Smart Turn end-of-turn detection (Silero and Smart Turn through ONNX) on a dedicated CPU pool, streaming speech-to-text, an LLM with tools, streaming text-to-speech, and transport out. Every user turn is a numbered epoch, so stale frames are dropped, and after a barge-in the LLM context keeps only the text actually played. Providers (Deepgram, OpenAI, Anthropic, Gemini, Cartesia, ElevenLabs, or local sherpa-onnx and Ollama) are cargo features with per-stage failover. Tools run in your own process through the Python or TypeScript SDK, and side-effect tools never run speculatively or twice for one call id. The server adds admission control, graceful drain and Prometheus metrics.
Hard parts
- Running the real VAD and turn-detection models on every call and still fitting hundreds of calls into 4 CPUs and 4 GiB.
- Correct interruptions: turn epochs, played-text truncation and tools that never run twice, proved with deterministic paused-clock simulation.
- Honest benchmarks: same container, same latency limit, three runs each, with every number in a committed file.
- Turkish as a first-class language, alongside English.
Result
Released as v0.1.1 in October 2026 under MIT or Apache-2.0 on GitHub, PyPI, npm and crates.io, with prebuilt binaries, a Docker image and a docs book. In my Linux benchmark with mock providers (4 CPUs, 4 GiB, p95 voice-to-voice within 1.2 s) it held 325 concurrent calls in about 430 MB, against 20 for Pipecat and 23 for a tuned LiveKit Agents. On real providers over 100 turns, p95 voice-to-voice was 2.88 s against 4.35 s for Pipecat.