End-to-end speech-to-speech processing
Speech recognition, turn-taking, language-model behavior, and voice output are tuned together in a single Realtime model rather than selected independently.
Eleven Realtime
Eleven Realtime is a speech-to-speech model family for building low-latency voice agents in ElevenAgents or through an OpenAI-compatible API. It offers v1 Mini for natural conversation and v1 Max for complex reasoning and multi-step tool use.
What is Eleven Realtime?
Eleven Realtime is a speech-to-speech model family for building low-latency voice agents. It combines speech recognition, turn-taking, an LLM, and text-to-speech in an end-to-end system, so builders select a Realtime model instead of configuring each of those components separately.
The product is available through ElevenAgents and an OpenAI-compatible Realtime API. V1 Mini is designed for natural conversation, while v1 Max is intended for more complex tasks that require reasoning and multi-step tool use. Agents can be equipped with voices, knowledge bases, guardrails, tools, and channel deployments.
Key features
Two conversation modes
V1 Mini is positioned for natural, expressive back-and-forth, while v1 Max reasons before responding for complex instructions and multi-step tool use.
Multi-channel deployment
ElevenAgents can use the same agent across phone, web widget, WhatsApp, and a contact center platform, according to the product page.
Agent controls and monitoring
Agents can be configured with a knowledge base, guardrails, scripted scenario tests, and production conversation monitoring.
OpenAI-compatible API
The API is compatible with OpenAI's Realtime API, including corresponding client and server events, allowing existing handlers and tooling to be retained.
Voice and language selection
The product supports a voice library with more than 17,500 voices and realtime conversation in over 90 languages, as stated on the product page.
Use cases
AI assistants
Build assistants that handle quick questions and interruptions with natural timing and expressive voice output, using v1 Mini for conversation-focused interactions.
Educational tutors
Create spoken tutoring or guided-learning experiences where the agent can explain topics conversationally and respond to student follow-up questions.
Customer support
Use v1 Max for support interactions that require careful responses, access to tools, or multi-step handling of a customer request.
Lead qualification
Qualify prospects through spoken conversations, using a model designed for complex business interactions and tool calls.
Interactive characters
Develop interactive fictional or game characters that maintain voice-based dialogue and react during back-and-forth exchanges.
Pros and Cons
Pros
- Combines speech recognition, turn-taking, language-model behavior, and voice output in one managed model choice.
- Offers separate models for lower-latency natural conversation and more complex reasoning workflows.
- Provides an OpenAI-compatible API that can preserve existing client event handling and tooling.
- Supports agent knowledge bases, guardrails, scripted testing, monitoring, and deployment across multiple channels.
Cons
- Realtime models are intended for voice agents and are not suited to text-only agents.
- Availability is rolling out by workspace, and documentation lists model-specific restrictions for some data residency and compliance configurations.
- Detailed Realtime-specific pricing and quotas are not established by the provided evidence.
FAQ
How do I set up Eleven Realtime?
Eleven Realtime is available in ElevenAgents and through the Realtime API. In ElevenAgents, set an agent's Voice mode to Realtime, choose v1 Mini or v1 Max, configure the voice and agent settings, and publish the agent. API users can use an OpenAI-compatible Realtime client by changing the endpoint and model name.
What is the difference between Realtime v1 Mini and v1 Max?
V1 Mini is the default, lower-cost and lower-latency option for natural conversations such as assistants, tutors, and interactive characters. V1 Max is intended for complex instructions, reasoning, and multi-step tool use, including customer support and lead qualification.
Can I use an existing OpenAI Realtime API client?
The Realtime API is compatible with OpenAI's Realtime API. Existing clients can point their WebSocket transport to ElevenLabs, select a Realtime model and voice, and retain the event handling, logging, and tooling built for the compatible API.
When should I use Custom mode instead of Realtime mode?
Realtime models are intended for voice conversations. Use Custom mode instead when an agent requires a custom LLM, Amazon Bedrock, a Speech Engine, a specific LLM or TTS model, workflow overrides to managed settings, transfers to or from Custom-mode agents, or text-only operation.
Are there workspace or deployment limitations?
Realtime models are rolling out gradually. If Realtime mode is not shown in an agent's dashboard, it may not yet be available for that workspace. Documentation also lists model-specific availability limits for data residency environments and privacy or healthcare requirements.
Quick Facts
- Category
- Voice AI / Speech-to-speech
- Available through
- ElevenAgents and the ElevenLabs Realtime API
- Models
- Eleven Realtime v1 Mini and v1 Max
- Supported conversation languages
- 90+ languages
- Voice library
- 17,500+ voices stated on the product page
- API compatibility
- OpenAI Realtime API-compatible