llama.cpp v0.8.0 adds /v1/realtime for local voice agents

llama.cpp v0.8.0 shipped on October 28 with a /v1/realtime endpoint that implements the OpenAI realtime protocol. In plain terms: a voice agent you built against a hosted realtime API can now, in principle, point at a server on your own hardware. The release also adds a POST /tts endpoint and support for Liquid AI's LFM3 audio and omni models. For a phone line or a front-desk kiosk, that is speech in, speech out, with no per-minute bill.

What actually happened

Per the v0.8.0 release notes:

• /v1/realtime: a new server endpoint "implementing the OpenAI realtime protocol for conversation and transcription."
• POST /tts: a new endpoint for text-to-speech models.
• LFM3 models: the full LiquidAI LFM3 family (text, vision, audio and omni). LFM3-Audio-2B and LFM3-Omni-2B handle speech recognition, text-to-speech and voice chat from one model, with streaming audio in and out.
• Streaming audio API: a new mtmd_audio_stream API that turns PCM audio into model input as it arrives. The release labels it experimental.
• A breaking change: the CLI now requires --model or -hf. Scripts that relied on a default model will fail.

Why local voice agents matter for your business

Voice is where hosted AI gets expensive fastest. Every minute of a call is billed audio in and audio out, and calls run long. A 2B model on a local box costs electricity. It also keeps call audio, which often holds names, addresses and card details, inside your building.

Protocol compatibility is the real news. You do not need to rewrite your voice agent to try a local model. Change the base URL, keep the client. That makes a side-by-side test cheap. Here is how we would run it:

1. Record 50 real calls (with consent) and replay them through both your hosted model and the local one.
2. Score what matters: did it get the order number right, the address right, the intent right? Latency to first audio counts too. A pause of a second and a half on a phone call feels broken.
3. Keep the hosted API as the fallback. The streaming layer is experimental. Route to local first and fail over when it stalls.
4. Read the model license. LFM3 is under Liquid's own license with a $10M revenue cap on commercial use. The endpoint is open source; the model may not be free for you.

Also pin your llama.cpp version in production. The --model change is a small one, but it shows the CLI is still moving.

Sources: llama.cpp v0.8.0 release notes (GitHub).