feat: migrate from Supabase to direct PostgreSQL connection

.env.example

@@ -204,11 +204,10 @@ CONFIDENCE_THRESHOLD=0.0
 # DATABASE CONFIGURATION (Optional)
 # ======================================================================
 
-# Supabase URL for cloud database storage
-# SUPABASE_URL=https://your-project.supabase.co
-
-# Supabase Anonymous Key
-# SUPABASE_ANON_KEY=your_supabase_anon_key_here
+# PostgreSQL connection URL for database storage
+# Format: postgresql://user:password@host:port/database
+# Example: postgresql://postgres:mypassword@localhost:5432/ymemo
+POSTGRES_URL=postgresql://user:password@host:5432/database_name
 
 # ======================================================================
 # LOGGING & DEBUGGING

CLAUDE.md

@@ -6,6 +6,8 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 YMemo is a real-time voice meeting transcription application built with Gradio and multiple transcription services (AWS Transcribe and Azure Speech Service). The application captures audio from microphones, sends it to your chosen transcription provider for real-time speech-to-text conversion, and displays the results in a responsive web interface with speaker diarization support.
 
+**Database Architecture**: YMemo uses direct PostgreSQL connection (migrated from Supabase in 2025) for meeting and persona storage with full DDL permissions and automatic timestamp tracking.
+
 ## Development Setup
 
 **IMPORTANT: Always use the virtual environment (.venv) for all Python operations:**
@@ -63,6 +65,19 @@ source .venv/bin/activate && python tests/create_test_audio.py
 source .venv/bin/activate && python test_azure_speech_provider.py
 ```
 
+### Database Operations
+
+```bash
+# Run pending database migrations
+source .venv/bin/activate && python -c "from src.utils.database import run_database_migrations; successful, failed = run_database_migrations(); print(f'✅ Successful: {successful}'); print(f'❌ Failed: {failed}')"
+
+# Check migration status
+source .venv/bin/activate && python -c "from src.utils.database import get_migration_status; status = get_migration_status(); print(f'Applied: {status[\"applied_count\"]}, Pending: {status[\"pending_count\"]}')"
+
+# Test database connection
+source .venv/bin/activate && python -c "from src.utils.database import test_database_connection; print('✅ Connected' if test_database_connection() else '❌ Failed')"
+```
+
 ## Architecture Overview
 
 ### Core Components
@@ -196,6 +211,18 @@ print_config_summary()  # Shows current configuration
 - `AZURE_SPEECH_TIMEOUT` - Connection timeout in seconds (default: 30)
 - Requires `azure-cognitiveservices-speech>=1.45.0` dependency
 
+**Database Configuration (PostgreSQL):**
+
+- `POSTGRES_URL` - PostgreSQL connection URL (required)
+  - Format: `postgresql://user:password@host:port/database`
+  - Example: `postgresql://postgres:mypassword@localhost:5432/ymemo`
+- Direct PostgreSQL connection with full DDL permissions (migrated from Supabase 2025)
+- Supports automatic schema migrations via `db/migrations/` directory
+- Uses `psycopg2-binary>=2.9.10` for database connectivity (replaces supabase>=2.0.0)
+- Tables: `ymemo` (meetings with created_at/updated_at), `ymemo_persona` (personas)
+- Automatic timestamp triggers maintain `updated_at` fields on record modifications
+- Backward-compatible deprecated API available for gradual migration of client code
+
 ## Testing Strategy
 
 **MIGRATED PYTEST INFRASTRUCTURE (2024):**
@@ -244,8 +271,9 @@ tests/
 **Key Test Principles:**
 
 - **Hardware Independence**: All PyAudio calls mocked, no AWS credentials needed, no device access
+- **Database Independence**: All database operations mocked, no PostgreSQL connection required for tests
 - **Consistent Patterns**: All tests inherit from base classes with standard fixtures
-- **Comprehensive Mocking**: Centralized mock factories for AudioProcessor, Providers, AWS services
+- **Comprehensive Mocking**: Centralized mock factories for AudioProcessor, Providers, AWS services, Database clients
 - **Performance Focus**: Fast execution through effective mocking and parallel-safe design
 
 **Base Test Classes:**
@@ -259,25 +287,27 @@ tests/
 - `MockAudioProcessorFactory` - Standardized AudioProcessor mocks
 - `MockProviderFactory` - Provider mocks with proper interface compliance
 - `MockSessionManagerFactory` - Session manager mocks with state management
+- `MockDatabaseFactory` - PostgreSQL client mocks for database operations without connections
 - AWS mocking patterns for transcription without actual service calls
 
-**Recent Test Infrastructure Improvements (2024):**
+**Recent Test Infrastructure Improvements (2024-2025):**
 
-- **Provider System Refactoring**: Added 88 comprehensive tests for new registry/service architecture
-- **Workspace Migration**: Successfully migrated 7 root directory test files to organized pytest structure
-- **Azure Provider Testing**: Complete Azure Speech Service provider test coverage with comprehensive mocking
-- **Dual Provider System**: Full test coverage for AWS dual-channel architecture with channel splitting
-- **Configuration Validation**: Robust testing of environment variable parsing and configuration validation
-- **Device Selection**: Enhanced device selection testing with unicode support and edge case handling
-- **Async Testing**: Proper async test infrastructure with event loop management and resource cleanup
-- **Performance Optimization**: Test execution time reduced from ~8s to ~4.3s with better mocking strategies
+- **Provider System Refactoring (2024)**: Added 88 comprehensive tests for new registry/service architecture
+- **Workspace Migration (2024)**: Successfully migrated 7 root directory test files to organized pytest structure
+- **Azure Provider Testing (2024)**: Complete Azure Speech Service provider test coverage with comprehensive mocking
+- **Dual Provider System (2024)**: Full test coverage for AWS dual-channel architecture with channel splitting
+- **Configuration Validation (2024)**: Robust testing of environment variable parsing and configuration validation
+- **Device Selection (2024)**: Enhanced device selection testing with unicode support and edge case handling
+- **Async Testing (2024)**: Proper async test infrastructure with event loop management and resource cleanup
+- **Performance Optimization (2024)**: Test execution time reduced from ~8s to ~4.3s with better mocking strategies
+- **Database Migration Testing (2025)**: PostgreSQL migration from Supabase with zero hardware dependencies and comprehensive repository mocking
 
 **Legacy Test Files (Deprecated):**
 
 - Use migrated pytest versions instead of legacy unittest files
 - `test_core_functionality.py` - Use `tests/unit/` instead
 - `test_file_audio_capture.py` - Use `tests/audio/test_device_selection.py` instead
-- Always follow these rules: Tests should NOT require hardware devices, AWS credentials, or network connectivity
+- Always follow these rules: Tests should NOT require hardware devices, AWS credentials, PostgreSQL database, or network connectivity
 
 ## File Structure
 
@@ -296,34 +326,59 @@ src/
 ├── core/                    # Core business logic and interfaces
 │   ├── interfaces.py        # Abstract interfaces for all providers
 │   ├── factory.py           # Provider factory with registries
+│   ├── models.py            # Data models (Meeting, Persona, RecordingSession)
 │   └── processor.py         # Main audio processing pipeline
 ├── exceptions/              # Custom exceptions (NEW 2024)
 │   └── provider_exceptions.py  # Provider-specific exception hierarchy
 ├── managers/                # Management classes
-│   └── session_manager.py   # Singleton session management
+│   ├── session_manager.py   # Singleton session management
+│   ├── meeting_repository.py   # Meeting database operations (MIGRATED 2025)
+│   └── persona_repository.py   # Persona database operations (NEW 2024)
 ├── services/                # Service layer (NEW 2024)
 │   └── provider_service.py     # High-level provider service facade
 ├── ui/                      # User interface
 │   ├── interface.py         # Gradio web interface with responsive design
 │   └── provider_handlers.py    # Provider selection UI handlers (REFACTORED 2024)
 └── utils/                   # Utility modules
+    ├── database.py          # PostgreSQL database client (MIGRATED FROM SUPABASE 2025)
     ├── device_utils.py      # Audio device utilities
     ├── exceptions.py        # Custom exceptions
     └── status_manager.py    # Status management
+
+db/
+├── migrations/              # Database schema migrations (NEW 2025)
+│   ├── 001_baseline_ymemo_table.sql
+│   ├── 002_create_persona_table.sql
+│   └── 003_add_updated_at_to_ymemo.sql
+└── tools/
+    └── migration_runner.py  # PostgreSQL migration runner (MIGRATED 2025)
 ```
 
 ## Key Files
 
+**Core Application:**
 - `main.py` - Application entry point with CLI arguments
 - `src/core/interfaces.py` - Abstract interfaces for all providers
 - `src/core/factory.py` - Provider factory with registries
 - `src/core/processor.py` - Main audio processing pipeline
+- `src/core/models.py` - Data models (Meeting with updated_at, Persona, RecordingSession)
+
+**Database Layer (MIGRATED FROM SUPABASE 2025):**
+- `src/utils/database.py` - PostgreSQL client with connection management and DDL support
+- `src/managers/meeting_repository.py` - Meeting CRUD operations with updated_at support
+- `src/managers/persona_repository.py` - Persona management and database operations
+- `db/tools/migration_runner.py` - PostgreSQL migration system with baseline/incremental support
+- `db/migrations/003_add_updated_at_to_ymemo.sql` - Latest migration adding updated_at field
+
+**Session and UI Management:**
 - `src/managers/session_manager.py` - Singleton session management
 - `src/ui/interface.py` - Gradio web interface with responsive design
+- `src/ui/provider_handlers.py` - Provider selection UI handlers (REFACTORED 2024)
+
+**Configuration System:**
 - `src/config/audio_config.py` - Audio system configuration classes
 - `src/config/provider_registry.py` - Provider registry with dataclasses (NEW 2024)
 - `src/services/provider_service.py` - High-level provider service facade (NEW 2024)
-- `src/ui/provider_handlers.py` - Provider selection UI handlers (REFACTORED 2024)
 
 ## Threading and Async
 

db/__init__.py

@@ -0,0 +1 @@
+"""Database migration system for YMemo."""

db/migrations/001_baseline_ymemo_table.sql

@@ -0,0 +1,28 @@
+-- Migration: 001_baseline_ymemo_table.sql
+-- Description: Create baseline ymemo table (idempotent for universal deployment)
+-- Author: system
+-- Created: 2025-01-02
+-- Status: EXECUTABLE (works on fresh or existing databases)
+
+-- Create ymemo table matching current production schema
+-- Uses IF NOT EXISTS to be safe for existing databases
+CREATE TABLE IF NOT EXISTS public.ymemo (
+  id integer generated by default as identity not null,
+  name text null,
+  duration numeric null,
+  transcription text null,
+  created_at timestamp with time zone not null default now(),
+  audio_file_path text null,
+  constraint ymemo_pkey primary key (id)
+) TABLESPACE pg_default;
+
+-- Add any missing indexes that should exist on the baseline table
+-- (Currently no additional indexes defined for ymemo table)
+
+-- This migration creates the foundational ymemo table structure.
+-- It uses CREATE TABLE IF NOT EXISTS to be safe on databases where
+-- the table already exists, making it suitable for:
+-- - Fresh database deployments (dev/staging/test environments)
+-- - Re-deployment scenarios
+-- - Schema reconstruction
+-- - Environment consistency

db/migrations/002_create_persona_table.sql

@@ -0,0 +1,34 @@
+-- Migration: 002_create_persona_table.sql
+-- Description: Add persona management functionality
+-- Author: system
+-- Created: 2025-01-02
+-- Dependencies: 001_baseline_ymemo_table.sql
+
+-- Create simplified persona table following ymemo table patterns
+CREATE TABLE public.ymemo_persona (
+  id integer generated by default as identity not null,
+  name text not null,
+  description text null,
+  created_at timestamp with time zone not null default now(),
+  updated_at timestamp with time zone not null default now(),
+
+  constraint ymemo_persona_pkey primary key (id),
+  constraint ymemo_persona_name_check check (length(trim(name)) > 0)
+) TABLESPACE pg_default;
+
+-- Create indexes for performance
+CREATE INDEX idx_ymemo_persona_created_at ON public.ymemo_persona (created_at DESC);
+
+-- Add trigger for updated_at auto-update
+CREATE OR REPLACE FUNCTION update_ymemo_persona_updated_at()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = NOW();
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER trigger_ymemo_persona_updated_at
+    BEFORE UPDATE ON public.ymemo_persona
+    FOR EACH ROW
+    EXECUTE FUNCTION update_ymemo_persona_updated_at();

db/migrations/003_add_updated_at_to_ymemo.sql

@@ -0,0 +1,72 @@
+-- Migration: 003_add_updated_at_to_ymemo.sql
+-- Description: Add updated_at field to ymemo table for consistency with persona table
+-- Author: system
+-- Created: 2025-08-02
+-- Dependencies: 001_baseline_ymemo_table.sql
+--
+-- This migration adds the updated_at timestamp field to the existing ymemo table,
+-- following the same pattern as the ymemo_persona table. It includes:
+-- 1. Adding the updated_at column with default value (safe for existing records)
+-- 2. Backfilling existing records with created_at timestamp
+-- 3. Creating trigger function for automatic updates
+-- 4. Creating trigger for automatic timestamp updates on UPDATE operations
+--
+-- ROLLBACK: To rollback this migration:
+-- DROP TRIGGER IF EXISTS trigger_ymemo_updated_at ON public.ymemo;
+-- DROP FUNCTION IF EXISTS update_ymemo_updated_at();
+-- ALTER TABLE public.ymemo DROP COLUMN IF EXISTS updated_at;
+
+-- Step 1: Add updated_at column with default value
+-- This is safe for existing data - all current records will get NOW() as default
+-- Handle potential ownership issues by using IF NOT EXISTS pattern
+DO $$
+BEGIN
+    -- Check if column already exists
+    IF NOT EXISTS (SELECT 1 FROM information_schema.columns
+                   WHERE table_name='ymemo' AND column_name='updated_at') THEN
+        ALTER TABLE public.ymemo
+        ADD COLUMN updated_at timestamp with time zone NOT NULL DEFAULT NOW();
+    END IF;
+END $$;
+
+-- Step 2: Backfill existing records to set updated_at = created_at
+-- This gives existing records a logical updated_at value
+DO $$
+BEGIN
+    -- Only update if the column was just added (updated_at equals default NOW())
+    UPDATE public.ymemo
+    SET updated_at = created_at
+    WHERE ABS(EXTRACT(EPOCH FROM (updated_at - NOW()))) < 5;
+END $$;
+
+-- Step 3: Create trigger function for automatic updated_at updates
+-- This function will be called on every UPDATE to set updated_at = NOW()
+CREATE OR REPLACE FUNCTION update_ymemo_updated_at()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = NOW();
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+-- Step 4: Create trigger to automatically update updated_at on record updates
+-- This ensures updated_at is always current when a record is modified
+DO $$
+BEGIN
+    -- Drop trigger if it exists, then create it
+    DROP TRIGGER IF EXISTS trigger_ymemo_updated_at ON public.ymemo;
+    CREATE TRIGGER trigger_ymemo_updated_at
+        BEFORE UPDATE ON public.ymemo
+        FOR EACH ROW
+        EXECUTE FUNCTION update_ymemo_updated_at();
+END $$;
+
+-- Step 5: Add index for performance (optional but recommended)
+-- This helps with queries that filter or sort by updated_at
+DO $$
+BEGIN
+    -- Create index if it doesn't exist
+    IF NOT EXISTS (SELECT 1 FROM pg_class WHERE relname = 'idx_ymemo_updated_at') THEN
+        CREATE INDEX idx_ymemo_updated_at ON public.ymemo (updated_at DESC);
+    END IF;
+END $$;

db/migrations/__init__.py

@@ -0,0 +1 @@
+"""Database migration files for YMemo schema changes."""

db/seeds/__init__.py

@@ -0,0 +1 @@
+"""Database seed files for YMemo default data."""

db/seeds/default_personas.sql

@@ -0,0 +1,12 @@
+-- Seed data: Default persona configurations
+-- Description: Insert default persona profiles for YMemo
+-- Author: system
+-- Created: 2025-01-02
+
+-- Insert default personas (will be executed after persona table creation)
+INSERT INTO public.ymemo_persona (name, description) VALUES
+(
+    'Michael Wei',
+    'xxxxxx'
+)
+ON CONFLICT DO NOTHING;

db/tools/__init__.py

@@ -0,0 +1 @@
+"""Database migration tools and utilities for YMemo."""