If you’ve looked at managed voice AI platforms like Vapi, Bland AI, or Retell, you’ll have noticed they’re genuinely impressive and genuinely pricey. The infrastructure, low-latency audio handling, and concurrency they provide add up quickly once you’re past prototyping. Pipecat is the open-source alternative: a Python framework that lets you build the same kind of real-time voice AI pipeline without the per-minute cost or vendor lock-in.
Built originally by Daily.co and now an independent open-source project, Pipecat has become one of the more actively maintained frameworks for voice AI in 2026. Here’s how it works and when it makes sense to use it.
The Pipeline Model
Pipecat structures a voice AI application as a pipeline of processors. Audio comes in from a transport layer — Daily.co WebRTC, a telephony provider, or a local microphone — and flows through a chain of components:
- VAD (voice activity detection) — detects when the user starts and stops speaking
- STT (speech-to-text) — transcribes audio using Whisper, Deepgram, AssemblyAI, or others
- LLM — processes the transcript and generates a response (OpenAI, Anthropic, Gemini, Ollama)
- TTS (text-to-speech) — converts the response to audio using ElevenLabs, Cartesia, Azure, or OpenAI TTS
- Transport — delivers audio back to the caller
Each stage is a processor you compose into a Pipeline and run via a PipelineTask. You’re not writing audio handling code yourself — Pipecat manages the buffer, event loop, and streaming between stages.
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.task import PipelineTask
from pipecat.services.openai import OpenAILLMService, OpenAITTSService
from pipecat.services.deepgram import DeepgramSTTService
pipeline = Pipeline([
transport.input(),
DeepgramSTTService(api_key=DEEPGRAM_API_KEY),
OpenAILLMService(api_key=OPENAI_API_KEY, model="gpt-4o"),
OpenAITTSService(api_key=OPENAI_API_KEY, voice="nova"),
transport.output()
])
task = PipelineTask(pipeline)That’s roughly the shape of a functional voice agent. You’d add a system prompt, wire up turn detection, and handle interruption (when the user speaks mid-response), but the core structure is this simple.
The Latency Problem
Latency is the central challenge in voice AI — anything over roughly 800ms between end of user speech and start of AI speech feels wrong. Pipecat handles this through streaming at every stage: STT sends partial transcripts as the user speaks, the LLM starts generating before transcription is complete, and TTS starts speaking before the LLM finishes. You never wait for a full response before starting the next stage.
Getting this right in your own code is harder than it sounds. It’s one of the main things managed platforms have spent real engineering time on, and it’s also one of the main things Pipecat abstracts away — the pipeline execution model handles the concurrency and streaming plumbing so you don’t have to.
Telephony Integration
One common use case is voice AI for actual phone calls. Pipecat integrates with Twilio, Vonage, and Daily.co’s PSTN gateway. Vonage has strong UK telephony infrastructure, which makes it a natural choice for UK deployments — you can provision a UK number and connect it to a Pipecat application via webhook in an afternoon.
For UK businesses, this opens up some practical applications: automated appointment booking for dental practices or GP surgeries, out-of-hours customer handling for SMEs, or intelligent IVR systems that understand natural language rather than requiring “press 1 for billing.” The NHS has been piloting voice AI for appointment management at some trusts, and commercial deployments in small business contexts are increasingly common.
Supported Integrations
Pipecat supports a wide range of providers at each stage, which is important for avoiding lock-in:
STT: Deepgram, AssemblyAI, Whisper (local), Azure Speech, Google Speech-to-Text, Gladia LLM: OpenAI, Anthropic Claude, Google Gemini, Mistral, Ollama (for local models) TTS: ElevenLabs, Cartesia, Azure TTS, OpenAI TTS, Deepgram Aura, LMNT Transport: Daily.co WebRTC, local audio, Twilio, Vonage
The breadth means you can mix providers to optimise for cost or quality at each stage — for example, Deepgram for fast STT, Claude Haiku for low-latency LLM inference, and Cartesia for natural-sounding TTS — without any one vendor owning your entire stack.
When to Use Pipecat vs. Managed Platforms
To be honest, Vapi and Retell are easier to start with. You’ll have a working demo in a couple of hours without touching infrastructure. If you’re validating an idea or shipping a prototype quickly, managed platforms are the right call.
Pipecat makes sense when the per-minute costs at production volume don’t work, when you have privacy requirements that mean call audio can’t leave your infrastructure, or when you want providers the managed platforms don’t support. Healthcare applications are the obvious UK example — anything touching patient data has constraints that make fully self-hosted pipelines preferable.
The framework is also worth knowing about for teams building voice features into existing applications. Rather than routing through a third-party platform and its data agreements, you host your own Pipecat service alongside your stack.
Getting Started
pip install pipecat-ai[daily,openai,deepgram]The extras in brackets install transport and service integrations. The official examples repository has working demos for phone agents, customer service bots, and multimodal agents. Daily.co’s free tier provides enough WebRTC capacity to build and test without spending anything.
The documentation is thorough and the project has active maintainers. If you’ve been paying for managed voice AI and want to understand what’s under the hood — or bring it in-house — Pipecat is the clearest route in.