Skip to content

ASR4/destinx

BranchesTags

Repository files navigation

Destinx — AI Travel Agent

An AI-powered travel agent that operates entirely through WhatsApp. It plans end-to-end trips, searches flights/hotels/restaurants/experiences, books via API or deep links, remembers preferences across conversations with confidence-weighted memory, validates trip plans, sends proactive notifications, and handles payments through Stripe.

Built with Claude (Anthropic), Fastify, PostgreSQL + pgvector, BullMQ, Twilio, Duffel, and Stripe.

Table of Contents

Features

Core

  • WhatsApp-native — Users interact entirely through WhatsApp with interactive buttons and list messages
  • AI trip planning — Claude generates multi-day itineraries grounded in real-world research (Google Places, Brave Search, weather data)
  • Flight search & booking — Duffel API integration for searching and booking flights across 300+ airlines
  • Payment processing — Stripe Checkout for flight payments with per-booking service fees

Context Engineering

  • Three-tier context windowing — Recent messages kept verbatim, mid-range messages condensed, old messages summarized by Haiku to stay within 80K token budget
  • Trip state injection — Active trip plan appended at the END of context on every turn, preventing "lost in the middle" drift in long planning sessions
  • Stable tool set — All 15 tools registered on every Claude call for consistent tool selection accuracy
  • Concise tool results — Search results condensed to top 3-5 with key fields before entering context

Memory & Personalization

  • Confidence-weighted preferences — Explicit preferences scored at 0.7, inferred at 0.4, with confirmation bumps (+0.2) and decay over time
  • Contradiction detection — Detects when a new preference contradicts an existing one (e.g., vegetarian user asking for steakhouse) and prompts natural clarification
  • Semantic memory — pgvector embeddings for contextual recall with similarity + recency + confidence reranking
  • Confidence-weighted prompting — High-confidence preferences stated as fact, medium as observed tendencies, low as tentative

Onboarding

  • Smart new user detection — Automatically enters onboarding mode for users with fewer than 4 high-confidence preferences
  • Value-first approach — Gives genuinely useful travel insights before extracting preferences
  • Natural extraction — Max 2 questions per message, weaved into conversation naturally
  • Invisible transition — Seamlessly switches to regular prompt once enough preferences are learned

Trip Validation

  • Logistics checking — Validates travel times between consecutive activities, flags overlaps
  • Budget checking — Flags when total costs exceed stated budget by more than 15%
  • Pace checking — Ensures activity count matches user's pace preference (packed/balanced/relaxed)
  • Venue verification — Google Places API lookup to catch closed or non-existent venues
  • Auto-fix — Automatically shifts overlapping activities; flags unfixable issues for Claude to address

Booking

  • Smart deep links — Pre-filled booking links for Marriott, Hilton, Hyatt, Airbnb, Booking.com, OpenTable, Resy, GetYourGuide, Viator, Skyscanner, Google Flights, and Kayak
  • Booking tracking — Deep-link bookings tracked as link_sent, updated to user_confirmed when user reports back
  • Booking confirmation tool — Claude can record user-confirmed bookings with reference numbers
  • Browser automation — Available behind feature flag for hotel/restaurant/experience booking on real websites

Proactive Notifications

  • Price alerts — Price drop and increase notifications for watched bookings
  • Trip countdowns — Reminders at T-7, T-3, and T-1 days before trip start
  • Abandoned plan follow-up — Re-engages users after 48+ hours of silence on an active plan
  • Opt-out — Every notification includes "Reply STOP to turn off alerts"

Web Companion

  • Itinerary page — Mobile-optimized HTML page with day-by-day timeline, booking CTAs, budget breakdown, and map links (GET /trip/:tripId)
  • Travel DNA profile — Shows all learned preferences grouped by category with confidence badges (GET /profile/:userId)
  • Comparison page — Side-by-side hotel/flight comparison with recommended badge and booking buttons (GET /compare/:comparisonId)
  • Secure links — Short-lived, unguessable tokens stored in Redis

Resilience

  • Retry with backoff — All external API calls (Claude, Duffel, Google, Brave, Twilio) wrapped with exponential backoff retry
  • Circuit breaker — 3 failures in 5 minutes trips the circuit, blocking requests for 2 minutes to prevent cascading failures
  • Dead letter queue — Permanently failed conversation jobs send an apology message to the user
  • Graceful degradation — Search failures return helpful messages instead of errors; Claude handles missing data naturally
  • Unsupported media handling — Audio/video messages get a friendly "type that out for me" response

Other

  • Intent classification — Layered system: fast regex for trivial intents, Haiku for complex classification, Redis-cached results
  • Itinerary modification — Natural language plan edits ("swap day 3 dinner for sushi")
  • PDF itineraries — Generates formatted PDFs with booking references
  • Event discovery — Ticketmaster integration for finding concerts, festivals, and events during travel dates
  • Structured logging — Per-turn logging of intent, tools called, token usage, onboarding state, and response time

Architecture

┌──────────────┐     ┌──────────────┐     ┌──────────────────┐
│   WhatsApp   │────▶│   Fastify    │────▶│     BullMQ       │
│   (Twilio)   │◀────│   Server     │     │   Job Queues     │
└──────────────┘     └──────────────┘     └──────────────────┘
                            │                      │
                     ┌──────┴──────┐        ┌──────┴──────┐
                     │  PostgreSQL │        │    Redis     │
                     │  + pgvector │        │  (cache +   │
                     │  (Drizzle)  │        │   queues)   │
                     └─────────────┘        └─────────────┘
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
        ┌──────────┐ ┌──────────┐ ┌──────────────┐
        │  Claude  │ │  Duffel  │ │    Stripe    │
        │ (AI/LLM) │ │ (Flights)│ │  (Payments)  │
        └──────────┘ └──────────┘ └──────────────┘
Layer Technology
Runtime Node.js 20+ / TypeScript
HTTP Server Fastify 5
Database PostgreSQL 16 + pgvector (via Drizzle ORM)
Queue / Cache Redis + BullMQ + ioredis
AI Anthropic Claude (Sonnet for conversation, Haiku for background tasks)
Embeddings Voyage AI (voyage-large-2, 1536-d vectors)
WhatsApp Twilio Business API + Content API (interactive messages)
Flights Duffel API
Payments Stripe Checkout Sessions
Object Storage Cloudflare R2 (S3-compatible)
Search Google Maps Platform, Brave Search
Events Ticketmaster Discovery API

Project Structure

src/
├── ai/                          # LLM layer
│   ├── client.ts                # Anthropic SDK singleton
│   ├── tools.ts                 # 15 tool definitions exposed to Claude
│   └── prompts/                 # System, planning, booking, extraction prompts
│       ├── system.ts            # Main system prompt with confidence-weighted profiles
│       ├── extraction.ts        # Preference extraction with contradiction detection
│       ├── planning.ts          # Trip planning prompt
│       └── booking.ts           # Booking flow prompt
│
├── config/
│   ├── env.ts                   # Zod-validated environment variables
│   └── constants.ts             # Context tiers, models, FSM states, queue config
│
├── db/
│   ├── schema.ts                # Drizzle table definitions
│   ├── client.ts                # Postgres + Drizzle singleton
│   └── migrate.ts               # Startup migration runner
│
├── jobs/
│   ├── queue.ts                 # BullMQ queue definitions + workers + dead letter handling
│   ├── scheduler.ts             # Cron jobs (price checks, memory decay, notifications)
│   └── workers/                 # Job processors (planning, booking, memory, pricing)
│
├── routes/
│   ├── health.ts                # GET /health, /health/detailed
│   ├── whatsapp.ts              # POST /webhook/whatsapp (Twilio)
│   ├── booking.ts               # Booking session management
│   ├── payments.ts              # POST /webhook/stripe + success/cancel pages
│   ├── web.ts                   # Web companion pages (itinerary, profile, comparison)
│   └── dev.ts                   # Dev-only chat endpoint (non-production)
│
├── services/
│   ├── conversation/            # Core conversation engine
│   │   ├── engine.ts            # Three-tier context + Claude tool loop + onboarding detection
│   │   ├── context.ts           # Three-tier context windowing + trip state injection
│   │   ├── state-machine.ts     # Conversation FSM
│   │   ├── tool-executor.ts     # Routes Claude tool calls with retry/fallback
│   │   ├── intent.ts            # Intent classification (regex + Haiku)
│   │   └── clarifier.ts         # Missing-info detection for planning
│   │
│   ├── booking/                 # Booking orchestration
│   │   ├── orchestrator.ts      # Browser session setup + execution
│   │   ├── session.ts           # Session lifecycle management
│   │   ├── search-cache.ts      # Redis-backed search result cache
│   │   └── providers/           # Site-specific automation + Duffel flights
│   │
│   ├── planning/                # Trip planning
│   │   ├── planner.ts           # Plan generation orchestration
│   │   ├── validator.ts         # Post-generation validation (logistics, budget, pace, venues)
│   │   ├── modifier.ts          # AI-powered plan modifications
│   │   ├── research.ts          # Destination research (Brave Search)
│   │   ├── pdf.ts               # PDF itinerary generation
│   │   └── pricing.ts           # Price comparison + drop detection
│   │
│   ├── memory/                  # User preference memory
│   │   ├── store.ts             # Preference CRUD with contradiction detection + confidence scoring
│   │   ├── recall.ts            # Semantic memory retrieval + reranking
│   │   ├── embeddings.ts        # Voyage AI vector embeddings
│   │   ├── extractor.ts         # Extract preferences + detect contradictions from conversation
│   │   └── profile.ts           # Build complete user profile
│   │
│   ├── notifications/           # Proactive outbound messaging
│   │   └── service.ts           # Price alerts, trip countdowns, abandoned plan follow-ups
│   │
│   ├── payments/                # Stripe integration
│   │   ├── stripe.ts            # Checkout session creation
│   │   └── webhook.ts           # Payment completion → booking trigger
│   │
│   ├── search/                  # Search providers
│   │   ├── flights.ts           # Duffel HTTP API
│   │   ├── hotels.ts            # Google Places hotels
│   │   ├── restaurants.ts       # Google Places restaurants
│   │   └── experiences.ts       # Google Places experiences
│   │
│   ├── tools/                   # Utility tools for Claude
│   │   ├── maps.ts              # Google Maps Places + Directions
│   │   ├── weather.ts           # OpenWeatherMap + Brave fallback
│   │   ├── web-search.ts        # Brave Search API
│   │   └── events.ts            # Ticketmaster + Brave fallback
│   │
│   ├── whatsapp/                # WhatsApp messaging
│   │   ├── handler.ts           # Incoming message processing + unsupported media handling
│   │   ├── sender.ts            # Twilio message sending with retry
│   │   ├── templates.ts         # Interactive message templates
│   │   └── formatter.ts         # Message formatting for WhatsApp limits
│   │
│   ├── storage/
│   │   └── r2.ts                # Cloudflare R2 uploads (screenshots, PDFs)
│   │
│   └── rate-limiter.ts          # Rate limiting for Claude + browser sessions
│
├── templates/                   # Server-rendered HTML pages
│   ├── itinerary.ts             # Mobile-optimized trip itinerary page
│   ├── profile.ts               # Travel DNA profile page
│   ├── comparison.ts            # Side-by-side option comparison page
│   └── live-view.html           # Live booking view (browser automation)
│
├── types/                       # Shared TypeScript types
├── utils/
│   ├── errors.ts                # Retry wrapper, circuit breaker, error categories
│   ├── deeplink.ts              # Deep link generators (12 providers)
│   ├── logger.ts                # Pino structured logging with PII redaction
│   ├── redis.ts                 # Shared Redis client
│   ├── correlation.ts           # Request correlation IDs
│   ├── phone.ts                 # Phone number formatting
│   ├── date.ts                  # Date utilities
│   └── currency.ts              # Currency formatting
│
└── index.ts                     # Application entry point

drizzle/                         # SQL migration files

Prerequisites

  • Node.js 20+
  • PostgreSQL 16+ with the pgvector extension
  • Redis 6+

Quick Start

# Clone the repository
git clone https://github.com/ASR4/destinx.git
cd destinx

# Install dependencies
npm install

# Configure environment
cp .env.example .env
# Edit .env with your API keys (see Environment Variables below)

# Run database migrations
npm run db:migrate

# Start development server
npm run dev

Scripts

Command Description
npm run dev Start dev server with hot reload (loads .env automatically)
npm run build Compile TypeScript + copy templates and migrations
npm start Run compiled output (production)
npm test Run tests with Vitest
npm run test:watch Run tests in watch mode
npm run db:generate Generate migration files from schema changes
npm run db:migrate Apply pending migrations
npm run db:push Push schema directly (dev convenience)
npm run db:studio Open Drizzle Studio (visual DB explorer)

Environment Variables

Copy .env.example to .env and fill in your keys. Here's what each one does:

Required

Variable Description
DATABASE_URL PostgreSQL connection string
REDIS_URL Redis connection string
ANTHROPIC_API_KEY Anthropic API key for Claude
TWILIO_ACCOUNT_SID Twilio Account SID
TWILIO_AUTH_TOKEN Twilio Auth Token
TWILIO_WHATSAPP_NUMBER Twilio WhatsApp sandbox or Business number

Search & Booking APIs

Variable Description Required?
GOOGLE_MAPS_API_KEY Google Maps Platform (Places, Directions, venue verification) Recommended
DUFFEL_API_KEY Duffel flight search & booking Recommended
BRAVE_SEARCH_API_KEY Brave Search (web search, weather fallback, research) Recommended
VOYAGE_API_KEY Voyage AI embeddings for semantic memory Optional

Payments

Variable Description Required?
STRIPE_SECRET_KEY Stripe secret key (test or live) Optional
STRIPE_WEBHOOK_SECRET Stripe webhook signing secret With Stripe
STRIPE_SERVICE_FEE_CENTS Service fee per booking in cents (default: 1500) With Stripe
FORCE_STRIPE_FLOW Set true to test Stripe with Duffel test keys Optional

Browser Automation

Variable Description Required?
ENABLE_BROWSER_AUTOMATION Set true to enable browser booking (default: false) Optional
BROWSERBASE_API_KEY Browserbase API key for cloud browsers Optional
BROWSERBASE_PROJECT_ID Browserbase project ID With Browserbase

Storage & Events

Variable Description Required?
CLOUDFLARE_R2_ACCESS_KEY R2 access key for screenshots/PDFs Optional
CLOUDFLARE_R2_SECRET_KEY R2 secret key With R2
CLOUDFLARE_R2_BUCKET R2 bucket name With R2
CLOUDFLARE_R2_ENDPOINT R2 S3-compatible endpoint With R2
TICKETMASTER_API_KEY Ticketmaster Discovery API for events Optional
OPENWEATHERMAP_API_KEY OpenWeatherMap 3.0 (Brave Search is fallback) Optional

Server

Variable Description Default
PORT Server port 3000
APP_URL Public URL (used for web pages, Stripe redirects) http://localhost:3000
HEALTH_CHECK_API_KEY Protects /health/detailed endpoint None
LOG_LEVEL Pino log level info

Database Setup

The application uses PostgreSQL with the pgvector extension for semantic memory. Migrations run automatically at startup via Drizzle ORM.

Tables:

  • users — Phone number, name, active/opt-out status
  • user_preferences — Key/value preferences with confidence scores, contradiction tracking, and decay
  • user_memory_embeddings — 1536-dimensional vector embeddings for semantic recall
  • trips — Destinations, dates, JSON itinerary plans, budget
  • conversations — Conversation state, context, and cached conversation summaries
  • messages — Full message history including tool calls/results
  • bookings — Booking records with provider, status (planned/link_sent/user_confirmed/booked), Stripe session, payment status
  • automation_scripts — Browser automation step tracking and success rates

Deployment

Railway (recommended)

  1. Connect your GitHub repo to Railway
  2. Add PostgreSQL and Redis services
  3. Set environment variables in the Railway dashboard
  4. Set APP_URL to your Railway public domain
  5. Railway auto-deploys on push to main

The build runs tsc and copies templates/migrations. The start command runs node dist/index.js, which automatically executes pending database migrations.

Twilio WhatsApp Setup

  1. Go to Twilio Console → Messaging → Try it out → Send a WhatsApp message
  2. In the sandbox settings, set:
    • When a message comes in: https://YOUR-DOMAIN/webhook/whatsapp (POST)
    • Status callback URL: https://YOUR-DOMAIN/webhook/whatsapp (POST)
  3. Join the sandbox by sending the join code from your phone

Stripe Webhook Setup

  1. Go to Stripe Dashboard → Webhooks → Add endpoint
  2. URL: https://YOUR-DOMAIN/webhook/stripe
  3. Events: checkout.session.completed
  4. Copy the signing secret to STRIPE_WEBHOOK_SECRET

API Endpoints

Method Path Description
GET /health Basic health check (always 200)
GET /health/detailed Detailed health with dependency status (requires API key)
POST /webhook/whatsapp Twilio WhatsApp incoming messages
GET /webhook/whatsapp Twilio webhook verification
POST /webhook/stripe Stripe payment webhook
GET /payment/success Post-payment success page
GET /payment/cancel Post-payment cancel page
POST /booking/start Start a browser booking session
GET /booking/live/:sessionId Live booking view (embedded browser)
POST /booking/live/:sessionId/cancel Cancel a booking session
GET /trip/:tripId Web companion: full trip itinerary page
GET /profile/:userId Web companion: Travel DNA profile page
GET /compare/:comparisonId Web companion: side-by-side comparison page
POST /dev/chat Dev-only: test conversation without WhatsApp
GET /dev/conversations/:userId Dev-only: view conversation history

How It Works

Conversation Flow

  1. User sends a WhatsApp message
  2. Twilio webhook hits /webhook/whatsapp
  3. Message is queued via BullMQ for async processing
  4. Intent classifier determines the user's goal (regex fast-path → Haiku LLM)
  5. Three-tier context window is assembled: recent messages verbatim, mid-range condensed, old messages summarized
  6. Active trip state is injected at the END of context to prevent drift
  7. Claude receives the message with all 15 tools and the user's confidence-weighted profile
  8. Claude may call tools (search flights, check weather, create trip plan, book flight, etc.)
  9. Tool results are fed back to Claude in a loop (up to 10 iterations)
  10. Final response is sent back via WhatsApp (with interactive buttons when appropriate)
  11. Preferences are extracted asynchronously via Haiku and stored with confidence scores

Booking Flow (Flights)

  1. User asks to search flights → Claude calls search_flights → Duffel API
  2. Results are cached in Redis with a short searchId (prevents Claude from hallucinating long IDs)
  3. User picks a flight → Claude collects passenger details → calls book_flight
  4. With Stripe: Creates a Checkout Session → user pays → webhook triggers Duffel booking
  5. Without Stripe: Books directly via Duffel API
  6. Confirmation sent via WhatsApp

Booking Flow (Hotels/Restaurants/Experiences)

  1. Claude calls initiate_booking with venue details
  2. System generates pre-filled deep links for multiple providers (Marriott, Booking.com, OpenTable, etc.)
  3. Links are sent to the user via WhatsApp with provider labels
  4. Booking is tracked as link_sent in the database
  5. When user confirms they booked, Claude calls confirm_booking to update the record
  6. Browser automation path available behind ENABLE_BROWSER_AUTOMATION=true flag

Memory System

  • Preferences are extracted from every conversation by Claude Haiku (background task)
  • Stored with confidence scores: explicit = 0.7, inferred = 0.4, with bumps for confirmation
  • Contradictions detected and flagged — agent asks naturally instead of silently overwriting
  • Confidence decays by 0.1 for preferences not referenced in 6+ months
  • Semantic embeddings enable recall of relevant memories with similarity + recency + confidence reranking
  • Profile injected into system prompt with confidence-weighted language

Proactive Notifications

  • Price monitoring runs every 6 hours for active bookings
  • Trip countdown reminders sent at T-7, T-3, and T-1 days
  • Abandoned plan follow-ups sent after 48+ hours of silence
  • All notifications include opt-out footer

Plan Validation

After generating an itinerary, the validator checks:

  • Logistics: travel time between consecutive activities, scheduling overlaps
  • Budget: total costs vs. stated budget (flags >15% overage)
  • Pace: activity count vs. user preference (packed/balanced/relaxed)
  • Venues: Google Places API verification for named restaurants/hotels/experiences
  • Auto-fixable issues (time overlaps) are corrected automatically; others are flagged for Claude

Testing

Automated Tests

# Run all tests
npm test

# Run specific test suites
npx vitest run src/__tests__/deep-links.test.ts
npx vitest run src/__tests__/state-machine.test.ts
npx vitest run src/__tests__/unit/search-cache.test.ts

Manual Testing via Dev Endpoint

The /dev/chat endpoint lets you test the full conversation loop without WhatsApp:

# Start a conversation
curl -X POST http://localhost:3000/dev/chat \
  -H 'Content-Type: application/json' \
  -d '{"message": "Plan a trip to Tokyo in October for 2 people"}'

# Continue with the same user (default phone: +15550001234)
curl -X POST http://localhost:3000/dev/chat \
  -H 'Content-Type: application/json' \
  -d '{"message": "Budget is around $4000, we love food and history"}'

# Test with a different user
curl -X POST http://localhost:3000/dev/chat \
  -H 'Content-Type: application/json' \
  -d '{"phone": "+15550009999", "message": "Hi there!"}'

Web Companion Pages

After creating a trip, test the web pages:

# Get a trip ID from the database
psql $DATABASE_URL -c "SELECT id, destination FROM trips ORDER BY created_at DESC LIMIT 1;"

# View in browser
open "http://localhost:3000/trip/<TRIP_ID>"
open "http://localhost:3000/profile/<USER_ID>"

Database Verification

# Check preference confidence scoring
psql $DATABASE_URL -c "SELECT category, key, value, confidence, source FROM user_preferences ORDER BY created_at DESC LIMIT 10;"

# Check booking tracking
psql $DATABASE_URL -c "SELECT type, provider, status, booking_reference FROM bookings ORDER BY created_at DESC LIMIT 5;"

# Check conversation summaries
psql $DATABASE_URL -c "SELECT id, context->>'conversationSummary' as summary FROM conversations WHERE context->>'conversationSummary' IS NOT NULL LIMIT 3;"

External Services

Service Free Tier Sign Up
Anthropic Pay-as-you-go Required
Twilio Free trial credits Required
Duffel Free test mode Recommended
Google Maps Platform $200/month free Recommended
Brave Search 2,000 queries/month free Recommended
Stripe No monthly fee (2.9% + 30¢ per transaction) Optional
Cloudflare R2 10GB free, zero egress Optional
Voyage AI Free tier available Optional
Ticketmaster 5,000 requests/day free Optional

Contributing

Contributions are welcome! Here's how to get started:

  1. Fork the repository
  2. Create a branch for your feature (git checkout -b feature/my-feature)
  3. Make your changes and ensure they build (npm run build)
  4. Run tests (npm test)
  5. Commit with a descriptive message
  6. Open a Pull Request

Areas Where Help Is Needed

  • Testing — Integration and unit test coverage (see src/__tests__/)
  • WhatsApp templates — Twilio Content API template registration for interactive buttons/lists
  • Multi-currency — Currency conversion and display for international users
  • Localization — Multi-language support for WhatsApp messages
  • Observability — Prometheus metrics, Sentry integration

Development Tips

  • Use npm run dev for hot-reload development
  • Use /dev/chat endpoint to test without WhatsApp (non-production only)
  • Use npm run db:studio to inspect the database visually
  • Duffel test keys only work with "Duffel Airways" (fictional airline) — real airline bookings require a live key
  • Browser automation is behind ENABLE_BROWSER_AUTOMATION=true — without it, the system uses deep links

License

MIT

About

WhatsApp-first AI travel agent powered by Claude, Browserbase + Stagehand, with memory-driven planning and browser automation. Built with Node.js, TypeScript, Fastify, PostgreSQL + pgvector, and BullMQ.

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages