Skip to content

implementation_summary.md

Latest commit

 

History

History
265 lines (214 loc) · 7.72 KB

implementation_summary.md

File metadata and controls

265 lines (214 loc) · 7.72 KB
Raw

LocalPulse Action Engine - Real Data V0 Implementation Summary

Project Overview

This implementation adds real-time capacity monitoring and action recommendations to the LocalPulse Action Engine by processing traffic and weather/air quality CSV data sources.

Key Components

1. Data Processing Pipeline

  • CSV Ingestion: Parse and normalize traffic and weather CSV files
  • Data Aggregation: Aggregate data to hourly intervals with proper weighting
  • Scoring Algorithm: Compute flow, environment, and capacity scores (0-100)
  • Action Generation: Rule-based action recommendations with overrides

2. API Endpoints

  • GET /signals/current: Current capacity index with detailed metrics
  • GET /actions?limit=N: Top N action recommendations
  • GET /demo: Static demo page showcasing the functionality

3. Testing Strategy

  • Unit Tests: Core logic functions with edge cases
  • Contract Tests: API endpoint validation with temporary CSV fixtures
  • Coverage Target: ≥70% across all new code

Implementation Architecture

graph TB
    subgraph "Data Sources"
        A[Traffic CSV] --> D[ingest_csv.py]
        B[Weather CSV] --> D
    end
    
    subgraph "Processing"
        D --> E[index_v0.py]
        E --> F[Scoring Functions]
        E --> G[Action Rules]
    end
    
    subgraph "API Layer"
        F --> H[routes_signals.py]
        G --> H
        H --> I[/signals/current]
        H --> J[/actions]
        H --> K[/demo]
    end
    
    subgraph "Frontend"
        I --> L[demo.html]
        J --> L
    end
    
    subgraph "Testing"
        M[Unit Tests] --> E
        N[Contract Tests] --> H
        O[Fixtures] --> M
        O --> N
    end
Loading

File Structure

LocalPulse/
├── packages/
│   └── signals/
│       ├── __init__.py
│       ├── ingest_csv.py
│       └── index_v0.py
├── packages/datasets/
│   ├── traffic_sample.csv
│   └── weather_sample.csv
├── apps/action-engine-api/
│   ├── app/
│   │   ├── routes_signals.py
│   │   └── static/
│   │       └── demo.html
│   └── tests/
│       ├── unit/
│       │   └── test_index_v0_logic.py
│       └── contract/
│           └── test_signals_and_actions_contract.py
├── examples/
│   └── requests.http (updated)
├── scripts/
│   └── demo.sh (updated)
└── README.md (updated)

Data Flow

  1. CSV Loading: Load and normalize CSV files with timezone awareness
  2. Data Aggregation: Aggregate to hourly intervals with traffic weighting
  3. Score Calculation:
    • Flow score: Based on historical traffic patterns (5th/95th percentile)
    • Environment score: Combined weather factors (temp, humidity, wind, air quality)
    • Capacity index: 65% flow + 35% environment
  4. Action Generation: Rule-based recommendations with emergency overrides
  5. API Response: JSON responses with rate limiting headers
  6. Caching: TTL-based caching to optimize performance

Key Algorithms

Traffic Weighting

weights = {
    "PEOPLE": 1.0,
    "MOTORCYCLE": 0.6,
    "BIKE": 0.5,
    "AUTOCAR": 0.7,
    "BUS": 0.8,
    "TRUCK": 0.3
}

Capacity Index Formula

Capacity Index = 0.65 × Flow Score + 0.35 × Environment Score

Environment Score Components

  • Temperature: Peak scoring (optimal at 24°C)
  • Humidity: Peak scoring (optimal at 55%)
  • Wind: Linear penalty (higher wind = lower score)
  • Air Quality: Linear penalty (higher PM2.5 = lower score)

Action Rules

Base Rules by Capacity Band

Capacity Band Range Actions
Peak 80-100 加派人手與備貨, 排隊動線/流量管制, 限時優惠拉高轉單
High 60-79 彈性排班, 社群預告檔期/活動, 現場加價購或加值服務
Mid 40-59 提高外送權重或店內活動, 看板文案改成吸客/避雨熱飲, 聯合鄰店做導流
Low 0-39 主打線上/外送, 調整營運時段/人力, 社群觸達暖身

Emergency Overrides

  • PM2.5 ≥ 55: Insert "提供口罩/室內主打/空氣清淨"
  • Wind ≥ 10 m/s: Insert "固定戶外物件/改室內動線"

Implementation Checklist

Phase 1: Core Infrastructure

  • Create packages/signals module structure
  • Implement CSV ingestion functions
  • Implement scoring and action logic
  • Create sample CSV files

Phase 2: API Integration

  • Create API routes with caching
  • Integrate with existing rate limiting
  • Add proper error handling
  • Update main app to include new routes

Phase 3: Testing

  • Implement comprehensive unit tests
  • Implement contract tests with temp files
  • Verify ≥70% code coverage
  • Test error conditions and edge cases

Phase 4: Demo & Documentation

  • Create static demo page
  • Update documentation
  • Update demo script
  • Add examples to requests.http

Environment Variables

# Required for production
TRAFFIC_CSV_PATH=/path/to/traffic.csv
WEATHER_CSV_PATH=/path/to/weather.csv

# Optional
TIMEZONE=Asia/Taipei  # Default: Asia/Taipei

Performance Considerations

Caching Strategy

  • Cache key namespace: signals:*
  • TTL: 300 seconds (configurable via existing TTL_SEC)
  • Separate cache entries for different action limits

Error Handling

  • Graceful degradation when CSV files are missing
  • Proper HTTP status codes (404, 503, 422)
  • Detailed error messages for debugging

Rate Limiting

  • Reuse existing rate limiting middleware
  • Same limits as other endpoints (configurable via RATE_LIMIT_PER_MIN)
  • Consistent rate limit headers

Security Considerations

Input Validation

  • Validate all query parameters (limit range)
  • Sanitize file paths to prevent directory traversal
  • Proper CSV parsing to prevent injection attacks

Error Information

  • Don't expose file system details in error messages
  • Use generic error codes for client responses
  • Log detailed errors for debugging

Deployment Notes

File Permissions

  • Ensure CSV files are readable by the application
  • Set appropriate permissions for datasets directory

Monitoring

  • Monitor API response times for signals endpoints
  • Track cache hit/miss ratios
  • Alert on CSV file parsing errors

Scaling

  • CSV processing is CPU-bound for large files
  • Consider file size limits for production
  • Monitor memory usage during aggregation

Future Enhancements

V1 Potential Features

  • Real-time data streaming instead of CSV polling
  • Machine learning-based action recommendations
  • Historical trend analysis
  • Multi-location support
  • User feedback integration

Performance Optimizations

  • Incremental CSV processing
  • More efficient caching strategies
  • Background processing for large datasets
  • Database integration for historical data

Success Criteria

  1. Functional Requirements

    • Successfully parse and process CSV files
    • Compute accurate capacity indices
    • Generate relevant action recommendations
    • Provide responsive API endpoints

  2. Non-Functional Requirements

    • ≥70% test coverage
    • Response time < 500ms for cached requests
    • Proper error handling and status codes
    • Consistent rate limiting

  3. Documentation Requirements

    • Updated README with new endpoints
    • Example requests and responses
    • Clear setup instructions
    • Working demo page

Next Steps

  1. Review and approve this implementation plan
  2. Switch to Code mode to begin implementation
  3. Follow the implementation checklist in order
  4. Run tests after each major component
  5. Deploy and verify functionality

This implementation provides a solid foundation for real-time capacity monitoring while maintaining compatibility with the existing LocalPulse Action Engine architecture.