Vapi Voice Assistant with Google Calendar Integration

A voice assistant powered by Vapi that creates Google Calendar events through natural conversation.

🌐 Deployed URL

Live Demo: https://voice-scheduling-agent-2hsn.onrender.com

📹 Demo Video

Watch the demo: https://www.loom.com/share/600bf0cb30164ed1a7c5eae536916ccb

🎯 Try It Now

Vapi Assistant: https://vapi.ai/?demo=true&shareKey=a75efaa4-3334-4843-89ee-7b9991507859&assistantId=4a5eabd1-8c9a-4573-b931-cbfca500a9e5

This will launch the Vapi demo page where you can talk to the assistant directly.

The integration is working!

The voice assistant successfully:

1. Listens to your voice
2. Collects name, date, time, and meeting title
3. Confirms details with you
4. Creates a Google Calendar event

---

Architecture

User → Vapi Widget → Groq LLM → Tool Call → FastAPI (/webhook/vapi) → Google Calendar

How It Works

1. Frontend: Vapi Widget (HTML/JS) connects to Vapi
2. Voice: Vapi handles speech-to-text and text-to-speech
3. LLM: Groq (llama-3.3-70b-versatile) processes conversation
4. Backend: FastAPI receives tool calls and creates calendar events
5. Calendar: Google Calendar API creates the event

System Flow Diagram

sequenceDiagram
    participant User
    participant Vapi
    participant Groq
    participant FastAPI
    participant GoogleCalendar
    
    User->>Vapi: "Hi, I want to schedule a meeting"
    Vapi->>Groq: Send transcript
    Groq->>Vapi: Ask for name
    Vapi->>User: "What's your name?"
    User->>Vapi: "My name is John"
    Vapi->>Groq: Send transcript
    Groq->>Vapi: Ask for date
    Vapi->>User: "What date?"
    User->>Vapi: "March 20"
    Vapi->>Groq: Send transcript
    Groq->>Vapi: Ask for time
    Vapi->>User: "What time?"
    User->>Vapi: "3 PM"
    Vapi->>Groq: Send transcript
    Groq->>Vapi: Ask for title (optional)
    User->>Vapi: "Team meeting"
    Vapi->>Groq: Send transcript
    Groq->>Vapi: Confirm details
    Vapi->>User: "John, March 20 at 3 PM, Team meeting. Confirm?"
    User->>Vapi: "Yes"
    Vapi->>Groq: Tool call with all data
    Groq->>FastAPI: POST /webhook/vapi
    FastAPI->>GoogleCalendar: Create event
    GoogleCalendar->>FastAPI: Event link
    FastAPI->>Vapi: Success response
    Vapi->>User: "Event created successfully!"

Loading

---

Implementation Status

Error Handling & Robustness

Feature	Status	Implementation	Why Important
Calendar API failures	✅	main.py:288 - Returns "Sorry, I couldn't create the event. Please try again."	Prevents user confusion when Google Calendar is unavailable
Invalid date inputs	✅	main.py:263 - "Could not understand the date..."	Voice input is error-prone; users may say "next Tuesday" or unclear dates
Invalid time inputs	✅	main.py:268 - "Could not understand the time..."	Same as dates - natural speech varies widely

Conversation Design

Feature	Status	Implementation	Why Important
State machine	✅	session_manager.py	Ensures consistent flow: name → date → time → title → confirmation
Confirmation before event	✅	System prompt enforcement	Prevents accidental event creation with wrong details
Skip optional title	✅	session_manager.py:315-317	Not all meetings need titles; gives user flexibility
Cancel/Restart	✅	session_manager.py:345-352	Users may change their mind mid-conversation

Reliability & Safety Improvements

The backend includes several safeguards to ensure reliable and safe event creation:

1. Backend Confirmation Check

Why: LLMs can sometimes call tools prematurely, before collecting all required information.

Implementation:

• Session must be in ready_for_tool state
• confirmation_status must be True
• If not confirmed → returns error message to user

2. Idempotency Check

Why: Users might accidentally confirm multiple times, creating duplicate events.

Implementation:

• Session tracks event_created flag
• If event already created → returns existing event link
• Prevents duplicate bookings

3. Date Validation Layer

Why: LLMs may hallucinate dates or users might say ambiguous dates.

Implementation:

• Validates date is not in the past
• Validates date is within 1 year
• Rejects invalid formats with user-friendly messages

4. Error Handling

Philosophy:

• Internal logs contain full error details (for debugging)
• External responses are user-friendly (no technical jargon)
• Google Calendar API failures are caught gracefully

Trade-offs

Feature	Benefit	Trade-off
Strict validation	Prevents bad data	May reject ambiguous human input
Idempotency	Prevents duplicates	Requires session tracking
Confirmation check	Prevents premature execution	Slightly more complex flow

Production Readiness

Feature	Status	Implementation	Why Important
Comprehensive logging	✅	main.py:31-35	Debug issues in production; trace conversation flow
Environment validation	✅	calendar_service.py:42-45	Fail fast on missing credentials
Health check endpoint	✅	main.py:291-302 - GET /health	Monitor service status; used by load balancers

---

Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Configure Environment Variables

Create .env file:

PORT=8000

# Google Calendar API Credentials
GOOGLE_CLIENT_ID=your_client_id
GOOGLE_CLIENT_SECRET=your_client_secret
GOOGLE_REFRESH_TOKEN=your_refresh_token

# Vapi Configuration
VAPI_API_KEY=your_vapi_api_key

# Groq Configuration
GROQ_API_KEY=your_groq_api_key

3. Start the Backend

python main.py

The server runs at http://localhost:8000

4. Start ngrok (for local development)

ngrok http 8000

Copy the ngrok URL (e.g., https://abc123.ngrok.io)

5. Configure Vapi Dashboard

1. Go to Vapi Dashboard
2. Create or select your assistant
3. Add the Tool:
   • Type: Function
   • Name: function_tool
   • Server URL: https://iconological-clerklier-amelia.ngrok-free.dev/webhook/vapi
   • Timeout: 30 seconds
4. Set Model: Groq (llama-3.3-70b-versatile)
5. Set Voice: Cartesia (or your preferred voice)

6. Open the Frontend

IMPORTANT: Use a local server, not file://

Option A: Use ngrok URL (recommended)

https://iconological-clerklier-amelia.ngrok-free.dev/

Make sure backend is running with ngrok tunnel active.

Option B: Run locally

python main.py

Then open: http://localhost:8000

Option C: Python HTTP server

python -m http.server 3000

Then open: http://localhost:3000

---

Tool Configuration in Vapi

Server URL

https://iconological-clerklier-amelia.ngrok-free.dev/webhook/vapi

Tool Parameters

{
  "name": "function_tool",
  "description": "Create a calendar event",
  "parameters": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "date": { "type": "string" },
      "time": { "type": "string" },
      "title": { "type": "string" }
    },
    "required": ["name", "date", "time"]
  }
}

System Prompt

You are a voice scheduling assistant.
Ask for user's name, then date, then time, then title (optional).
Confirm all details, then call function_tool.
Do NOT skip steps.

---

Testing the Agent

Sample Conversation

Assistant: "Thank you for calling. How may I help you?"
User: "Hi, I want to schedule a meeting"
Assistant: "What's your name?"
User: "My name is John"
Assistant: "What date would you like?"
User: "March 20"
Assistant: "What time?"
User: "3 PM"
Assistant: "Would you like a title?"
User: "Team meeting"
Assistant: "Let me confirm. Your name is John. Meeting is on March 20 at 3 PM, title is Team meeting. Should I create it?"
User: "Yes"
Assistant: "Event created successfully!"

Test Checklist

• Backend starts without errors (python main.py)
• Health check works (GET /health)
• ngrok tunnel is active
• Vapi tool is configured with ngrok URL
• Voice assistant responds to voice
• Calendar event is created in Google Calendar

---

API Endpoints

Method	Endpoint	Description
POST	/webhook/vapi	Vapi webhook (tool calls)
GET	/health	Health check
GET	/api/sessions/{id}	Session info
POST	/api/tools/create_event	Direct API (testing)

---

Project Structure

├── main.py                 # FastAPI backend
├── calendar_service.py    # Google Calendar API
├── session_manager.py      # Session state machine
├── index.html              # Voice widget
├── requirements.txt        # Dependencies
├── .env.example            # Environment template
└── README.md               # This file

---

Troubleshooting

Issue: "Vapi SDK failed to load"

• Check internet connection
• Try refreshing the page

Issue: "The OAuth client was not found"

• Verify GOOGLE_CLIENT_ID is correct in .env

Issue: "Could not parse date/time"

• Make sure the backend has python-dateutil installed

Issue: Tool not called

• Verify ngrok URL is set in Vapi Dashboard tool settings
• Check that Server URL ends with /webhook/vapi

---

Credits

• Vapi: https://vapi.ai
• Groq: https://groq.com

---