AI phone agent
AI Voice
An AI phone agent that talks in real time, books appointments and takes messages.
In development, not in production yet
Live call
LiveChecking availability …
Hi, this is Christoph's AI assistant. How can I help?
I'd like to book a call with Christoph about the phone assistant.
Sure! One moment, let me check availability.
Live calendar
SyncedFriday
Product demo
1:00 – 2:00 PM
Free · 3:00 PM
Agent actions
LiveCaptured the request
Wants a consultation about the phone assistant
Checked availability
Found an open slot
Booked the appointment
Consultation scheduled
Sent confirmation
Email + SMS with calendar invite
Saved the lead
Contact captured for follow-up
A simulated run with timings from real backend logs. No real call, and the people are made up.
Core skills
- Real-time systems
- LLMs and agents
- Security and privacy
- Backend and APIs
- Testing and verification
- Infrastructure and ops
- AI-assisted engineering
Tech stack
- TypeScript
- NestJS
- PostgreSQL
- Prisma
- WebSocket
- Telnyx
- Gemini Live
- bge-m3
- ONNX
- Expo
- React Native
- NativeWind
- TanStack Query
- Linux
- pm2
- nginx
- GitHub Actions
- Vitest
- Playwright
The hardest part
Silence in the middle of a call
On the phone, even waiting is something you have to design.
The whole story
What it looked like
While the AI looks up an appointment in the calendar, nothing happens on the line. To the caller, it sounds like the call has died.
What was really going on
A website can show a spinner. A phone can't, and the model is supposed to keep talking while it waits for the result.
The fix
The AI now bridges the gap like a person would, with something like “one moment, let me check”. If the model stalls, a watchdog nudges it. If the caller talks over it, it goes quiet at once.
- 4,706
- tests in the whole product suite, 3,523 of them in Voice
- 753
- commits in 12 weeks, whole product suite
- ~120
- HTTP routes in the Voice backend
- ~110k
- lines of code, plus ~57k lines of tests, whole product suite
In plain words
Picture a receptionist who is never busy. Someone calls a business and an AI answers, understands the question, checks the calendar and books the appointment. If you interrupt it, it stops talking at once, like a person would.
Small businesses miss calls because everyone is busy working. AI Voice is built for exactly these small businesses.
How it works
Step 1 of 5
Call
Telnyx takes the call, the webhook signature proves where it came from, and the audio streams to the backend.
What was built
- Telnyx phone audio bridged live to a speech model, with its own config and knowledge base per business
- Tools for the calendar, messages, confirmation, orders and hanging up
- A live call view for operators over WebSocket
- A call archive with AI scoring and an improvement loop
- Per-business semantic search with local bge-m3 embeddings and a keyword fallback
- A 3-role dashboard as an Expo web app and Android APK
How those skills were used
- Real-time systems
- Barge-in, voice activity detection and bounded audio queues at PCM16, 16 kHz in and 24 kHz out
- LLMs and agents
- Tool design for a model that talks live on the phone while calling functions
- Security and privacy
- 2FA with TOTP and replay protection, CASL with 3 roles, webhook signatures, GDPR export and erasure, 90-day retention
- Backend and APIs
- About 120 HTTP routes in NestJS, workers with locks, retries and terminal states, 18 Prisma models
- Testing and verification
- A test walks every route and rejects any that is unguarded or wrongly scoped
- Infrastructure and ops
- Linux, pm2, nginx, Let's Encrypt, a deploy script with a health check, monitoring and an independent uptime probe
- AI-assisted engineering
- Spec-first with separate agents for implementing, reviewing, researching and critiquing plans
My role
Concept, architecture, directing the agents and verification. Claude Code wrote most of the code, inside an agent workflow I developed myself.
See the Context Observer project