Inbound Voice Agent

A self-hosted AI voice agent that answers inbound phone calls, holds a natural conversation, and logs a full transcript to Airtable after every call.

Pipeline: Twilio (SIP) → LiveKit Cloud → Silero VAD → Deepgram STT → OpenAI GPT → ElevenLabs TTS → Airtable

---

Environment Variables

Copy .env.example to .env and fill in every value before running.

LiveKit Cloud

Variable	What it is	Where to get it
LIVEKIT_URL	WebSocket URL of your LiveKit project	LiveKit Cloud dashboard → your project → Settings → Keys. Looks like wss://your-project.livekit.cloud
LIVEKIT_API_KEY	LiveKit API key	Same page as above — click Add key
LIVEKIT_API_SECRET	LiveKit API secret	Shown once when you create the key

Deepgram (Speech-to-Text)

Variable	What it is	Where to get it
DEEPGRAM_API_KEY	Deepgram API key	console.deepgram.com → API Keys → Create a Key

OpenAI (LLM)

Variable	What it is	Where to get it
OPENAI_API_KEY	OpenAI API key	platform.openai.com/api-keys
OPENAI_MODEL	Model to use	Optional. Default: gpt-4o-mini. Options: gpt-4o, gpt-4o-mini, gpt-3.5-turbo

ElevenLabs (Text-to-Speech)

Variable	What it is	Where to get it
ELEVENLABS_API_KEY	ElevenLabs API key	elevenlabs.io → Profile → API Key
ELEVENLABS_VOICE_ID	ID of the voice to use	Voice Library → pick any voice → click the </> icon to copy its ID
ELEVENLABS_MODEL	TTS model	Optional. Default: eleven_turbo_v2_5 (lowest latency, recommended for phone)

Airtable (Call Log Storage)

Variable	What it is	Where to get it
AIRTABLE_PAT	Personal Access Token	airtable.com/create/tokens → Create token → grant data.records:write scope on your base
AIRTABLE_BASE_ID	ID of your Airtable base	Open your base in the browser. The URL is airtable.com/appXXXXXXXX/... — the appXXXXXXXX part is the Base ID

Required Airtable table: Create a table named call_logs with these exact fields:

Field name	Field type
caller_number	Single line text
duration_seconds	Number
transcript	Long text
created_at	Date

Agent Behaviour

Variable	What it does	Default
MAX_CALL_SECONDS	Hard cap on call length (seconds). Ends the call and says goodbye if exceeded. Protects against runaway API costs.	600 (10 min)
AGENT_SYSTEM_PROMPT	Instructions sent to the LLM before each call. Keep it short and phone-appropriate.	Friendly assistant

Twilio

Variable	What it is	Where to get it
TWILIO_AUTH_TOKEN	Used to validate incoming webhook signatures	console.twilio.com → Account Info → Auth Token

---

Setup Checklist

Work through each section in order before starting the agent.

1 · LiveKit Cloud account

• Create a free account at livekit.io/cloud
• Create a new project
• Copy LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET
• Under your project → SIP → SIP Trunk → create an Inbound SIP Trunk
  • Set the allowed IP ranges to Twilio's SIP IP list (see Twilio docs: "Elastic SIP Trunking IP addresses")
  • Copy the LiveKit SIP URI (format: sip:<your-project>.sip.livekit.cloud)
• Under SIP → Dispatch Rule → create a rule that routes all inbound SIP calls to a worker dispatch

2 · Deepgram account

• Sign up at deepgram.com
• Create an API key and copy it to DEEPGRAM_API_KEY
• Confirm you have credits (new accounts receive free credits)

3 · OpenAI account

• Sign up at platform.openai.com
• Create an API key and copy it to OPENAI_API_KEY
• Add a payment method or ensure you have credits

4 · ElevenLabs account

• Sign up at elevenlabs.io
• Copy your API key to ELEVENLABS_API_KEY
• Browse the Voice Library, pick a voice, and copy its ID to ELEVENLABS_VOICE_ID
• Confirm the eleven_turbo_v2_5 model is available on your plan (it is on free and paid)

5 · Twilio SIP setup

• Create a Twilio account at twilio.com
• Buy or port an inbound phone number
• Go to Elastic SIP Trunking → Trunks → Create Trunk
• Under the trunk → Origination → add an origination SIP URI pointing to your LiveKit SIP endpoint: sip:<your-project>.sip.livekit.cloud
• Under the trunk → Numbers → associate your inbound phone number
• Copy your Twilio Auth Token to TWILIO_AUTH_TOKEN

6 · Airtable setup

• Create an Airtable account at airtable.com
• Create a new base (any name)
• Create a table named exactly call_logs
• Add the four fields: caller_number (text), duration_seconds (number), transcript (long text), created_at (date)
• Create a Personal Access Token at airtable.com/create/tokens — grant data.records:write on your base
• Copy the PAT to AIRTABLE_PAT and the Base ID to AIRTABLE_BASE_ID

7 · VPS deployment

Install Python 3.11+

python3 --version   # must be 3.11 or newer

Clone the repo and install dependencies

git clone <your-repo-url> /opt/voice-agent
cd /opt/voice-agent
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Create your .env file

cp .env.example .env
nano .env   # fill in every value

Test run (foreground)

source venv/bin/activate
python agent.py dev

You should see Connected to LiveKit Cloud in the logs. Make a test call to your Twilio number.

Run as a persistent service (systemd)

Create /etc/systemd/system/voice-agent.service:

[Unit]
Description=LiveKit Inbound Voice Agent
After=network.target

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/opt/voice-agent
EnvironmentFile=/opt/voice-agent/.env
ExecStart=/opt/voice-agent/venv/bin/python agent.py start
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable voice-agent
sudo systemctl start voice-agent
sudo systemctl status voice-agent
# View live logs:
sudo journalctl -u voice-agent -f

---

Architecture

Incoming call (PSTN)
       │
  Twilio number
       │  (SIP)
LiveKit SIP Trunk
       │
 LiveKit Cloud room created
       │
  agent.py worker dispatched
       │
  ┌────▼────────────────────────────────────┐
  │  Voice Pipeline                          │
  │                                          │
  │  Caller audio → Silero VAD               │
  │                    │ (speech detected)   │
  │               Deepgram STT               │
  │                    │ (text stream)       │
  │                OpenAI LLM               │
  │                    │ (token stream)      │
  │             ElevenLabs TTS               │
  │                    │ (audio stream)      │
  │            Caller hears response         │
  └──────────────────────────────────────────┘
       │ (call ends)
  Airtable call_logs record created

---

VAD Tuning

The Silero VAD parameters in agent.py are commented with explanations. If callers complain about being cut off, increase min_silence_duration (e.g. from 0.4 to 0.6). If there are awkward pauses after they finish speaking, decrease it.

---

Cost protection

Set MAX_CALL_SECONDS to a value appropriate for your use case. The agent will politely say goodbye and disconnect when the limit is reached. This prevents a stuck call from accumulating Deepgram, OpenAI, and ElevenLabs API charges indefinitely.