Skip to content

Changelog

Jeremy Peterson edited this page Mar 31, 2026 · 5 revisions

Changelog

This document tracks significant changes to PERTI across versions.


Version 19 (Current)

Released: March 2026 — 503 commits

Note: System re-entered hibernation mode March 30, 2026 with SWIM exemption — all SWIM daemons and API endpoints remain active. Previously hibernated March 22, un-hibernated briefly, then re-entered.

SWIM Platform Expansion

SWIM Data Isolation (Phase 4 — Migration 026)

All SWIM API endpoints now query exclusively from the SWIM_API database. Internal databases (VATSIM_TMI, VATSIM_ADL, VATSIM_REF, perti_site MySQL) are never accessed directly by API request handlers.

  • swim_flights expansion: +34 new columns, row_hash BINARY(20) for change detection, 14 unused indexes dropped (36 → 22)
  • 25 new mirror tables: 10 TMI (swim_ntml, swim_tmi_programs, _entries, _advisories, _reroutes, _reroute_routes, _reroute_flights, _reroute_compliance_log, _public_routes, _flight_control) + 4 flow (swim_tmi_flow_*) + 4 CDM (swim_cdm_*) + 4 reference (swim_airports, swim_airport_taxi_reference, _detail, swim_playbook_route_throughput) + 3 infrastructure (swim_change_feed, swim_sync_watermarks, swim_sync_state)
  • 14 SWIM views (8 active/recent TMI + 3 CDM + 3 existing)
  • sp_Swim_BulkUpsert updated: row-hash skip eliminates ~60-70% of no-op updates, change feed emission, +60 columns in OPENJSON/MERGE
  • 48 SWIM endpoints migrated to SWIM_API-only queries
  • PERTI_SWIM_ONLY optimization added to connect.php — skips MySQL/ADL/TMI/REF/GIS connections (~500-1000ms saved per request)
  • TMI endpoints switched from $conn_tmi to $conn_swim with swim_tmi_* mirror tables (10 files)
  • CDM endpoints switched to SWIM mirror reads with CDMService v2.0 (5 files + CDMService)

GUFI UUID Migration (PR #263 — Migration 032)

Migrates swim_flights.gufi from NVARCHAR to UNIQUEIDENTIFIER (UUID v4) per ICAO Doc 9965 / FIXM 4.3 standards.

  • gufi_legacy column preserves backward-compatible CID-based identifiers
  • sp_Swim_BulkUpsert v4: UUID generated by database, legacy format passed from sync layer
  • Ingest endpoints (CDM, SimTraffic, VNAS track/tags/handoff, metering) use swim_gufi_lookup_sql() for transparent UUID/legacy auto-detection; ADL ingest uses swim_generate_gufi_legacy() with flight_uid primary lookup
  • All 5 SDKs (PHP, Python, JS/TS, C#, Java) updated with gufi_legacy + gufi_created_utc fields
  • OpenAPI spec updated with GufiMetadata schema and oneOf format documentation

TMI Operations & Analytics Platform (PR #274 — Migration 055)

10 new tables in VATSIM_TMI across 3 architectural layers:

  • Layer 1 — TMI Unified Log: tmi_log_core, _scope, _parameters, _impact, _references — action-level audit trail with airspace/facility scope, operational values, metrics snapshots, and entity/Discord linkage
  • Layer 2 — Delay Attribution: tmi_cause_taxonomy (25-row taxonomy: TMI, WEATHER, VOLUME, etc.), tmi_delay_attribution (per-flight delay attribution)
  • Layer 3 — Facility Statistics: tmi_facility_stats_hourly, _daily, tmi_ops_performance (ASPM-style airport on-time metrics)
  • log_tmi_action() helper wired to 15 GDT endpoints (activate, blanket, cancel, compress, create, ECR, extend, power_run, publish, purge, reoptimize, revise, simulate, submit_proposal, transition)
  • WebSocket IPC broadcast via _tmi_log_ws_broadcast()/tmp/swim_ws_events.json
  • 8 SWIM mirror tables (swim_tmi_*) synced by swim_tmi_sync_daemon.php (5-min watermark cycle)
  • 3 SWIM API endpoints: event-log.php, delay-attribution.php, facility-stats.php

VATSWIM Splits Bridge (PR #275)

Bidirectional sector split configuration sync through the SWIM mirror architecture.

  • Inbound: POST /api/swim/v1/splits/ingest — idempotent upsert from external facility tools with source/source_id tracking (Migration 012)
  • Outbound: 6 REST endpoints (active, facility, configs, presets, areas, history) with strata filtering (superhigh/high/low)
  • Sync pipeline: Tier 1 (5-min delta via watermark) for configs+positions; Tier 2 (daily full replace) for presets+areas; 10-min dedup window for history detection
  • WebSocket: splits.activated/deactivated/updated/ingested channels with ARTCC-based subscriber filtering

VATSWIM Client Bridges (PR #276 — Migration 056)

5 client bridges for bidirectional data flow between PERTI/SWIM and external ATC/pilot tools:

  • Bridge 1 — HoppieWriter (PHP): EDCT delivery pipeline wired into ADL daemon's 60s TMI sync cycle. Change detection (new/amended/cancelled EDCT, GS hold/release) dispatches through EDCTDelivery multi-channel delivery (CPDLC, plugin, WebSocket, Discord). 20+ ICAO CPDLC message type formatters. Generic deliverMessage() with dedup and delivery logging. Migration 056 adds delivery_mode/delivery_freq to tmi_reroutes and gs_release_followon to tmi_programs. vIFF CTOT delivery wired into CDM poll daemon.
  • Bridge 2 — FSD Bridge (Go): integrations/fsd-bridge/ — SWIM consumer binary for FSD protocol injection
  • Bridge 3 — EuroScope Plugin (C++): integrations/euroscope-plugin/ — SWIM tag enrichment plugin with i18n (en-US, fr-CA)
  • Bridge 4 — Pilot Portal (Vue 3): integrations/pilot-portal/ — TOS filing and i18n
  • Bridge 5 — AOC Client (C/C++): integrations/aoc-client/ — Bidirectional sim telemetry daemon

VATSWIM Route Query API

Multi-source route suggestion engine for the SWIM API:

  • RouteQueryService — orchestrator querying playbook routes, CDRs, and historical route data
  • RouteFilterParser — PHP port of the playbook compound boolean filter parser
  • POST/GET /api/swim/v1/routes/query — unified route suggestions endpoint
  • GET /api/swim/v1/routes/resolve — route resolution with batch mode support
  • swim_route_stats table (Migration 034) for route usage statistics
  • Route stats sync added to Tier 2 refdata_sync_daemon.php

SimTraffic-VATSWIM Webhook Bridge (PR #277 — Migration 034)

Bidirectional webhook event bridge between PERTI/SWIM and SimTraffic:

  • Inbound: POST /api/swim/v1/webhooks/simtraffic — HMAC-verified event receiver with idempotency
  • Outbound: WebhookSender with HMAC signing, exponential backoff retry (10/30/90s), circuit breaker, batch delivery
  • WebhookEventBuilder — TMI event envelope construction for 8 event types (EDCT assign/revise/cancel, GS, reroute, gate ops)
  • webhook_delivery_daemon.php — 10s outbound queue processor, hibernation-aware
  • POST /api/swim/v1/webhooks/register — webhook subscription registration
  • Schema: swim_webhook_subscriptions, swim_webhook_events, swim_webhook_delivery_log

SWIM Ingest & WebSocket Additions

  • POST /api/swim/v1/ingest/aman — AMAN arrival sequence ingest endpoint
  • POST/GET /api/swim/v1/tos/file, /tos/status — TOS filing endpoints (Migration 033 tos_options)
  • WebSocket: cdm.* and aman.* channels added, message type passthrough fix

SWIM Daemons

  • swim_tmi_sync_daemon.php — 22 tables synced every 5 minutes (watermark-based delta + OPENJSON MERGE)
  • refdata_sync_daemon.php — CDRs (~41K), preferred routes (~28K), playbook routes (~55K), airports, taxi reference, route stats daily at 06:00Z
  • simtraffic_swim_poll.php — SimTraffic time data polling every 2 minutes
  • swim_adl_reverse_sync_daemon.php — SimTraffic data back to ADL every 2 minutes

FAA Preferred Routes Import Pipeline

New Step 2 in the refdata_sync_daemon.php daily pipeline for FAA preferred routes (prefroutes_db.csv):

  • ~28K routes imported with ICAO normalization, TRACON/Center enrichment from apts.csv, and endpoint-stripped route strings
  • Batch MERGE into VATSIM_REF dbo.preferred_routes with origin/dest airport detection and center/TRACON classification
  • REF→ADL sync via sync_ref_to_adl.php/.sql and airac_update.py
  • SWIM mirror: swim_preferred_routes table in SWIM_API with automated sync via swim_refdata_sync.php
  • Migration 023: preferred_routes table schema (REF + ADL) with 6 indexes
  • Migration 035: swim_preferred_routes mirror + updated sync status view

GDT Feature Parity & TFMS Time Management

GDT Feature Parity (PRs #255, #257, #259, #260 — Migrations 051-053)

Closes major FAA TFMS feature gaps across 8 areas:

  • Phase 1 — Quick Wins: Auto-reoptimization daemon cycle (~240s for active GDP programs), EDCT compliance monitoring with FAA ±5min window (EARLY/COMPLIANT/LATE/NO_SHOW), distance-based scope UI toggle (TIER/DISTANCE radio), GS ETD rule fix per FAA FSM Ch.7 (Migration 051)
  • Phase 2 — Core Features: ECR backend API (SWAP/DELAY/ADVANCE operations) with audit columns (Migration 052), Data Graph enhanced statistics (affected_flights, slot_utilization, on_time_pct, delay_variance, delay_p95, stack_peak)
  • Phase 3 — Modeling Power: Power Run multi-scenario parameter sweep endpoint + comparison table UI, Edit 15 per-15-minute rate editor with variable-rate slot generation (Migration 053)
  • Phase 4 — Blanket & Filters: Blanket adjustment endpoint for uniform delay offset, include/exclude carrier/type filters, slot bridging for GS→GDP transitions
  • Variable Rate Editor (#260): Unified hourly + Edit 15 tier rate editor, DD/HHMMZ time header format, rates_quarter_json in API and rendering
  • GS Scope UI Redesign (#257): Tier numbering, EDCT timing corrections, UTF-8 safety
  • Hard-delete ephemeral MODELING programs (#255) instead of soft-delete

TFMS Time Hierarchy (PRs #268, #269)

Demand charts now use FAA TFMS time priority: Actual > Controlled (GDP/GS) > Estimated > Scheduled.

  • OETA/OETD Freeze (#269): Write-once oeta_utc/oetd_utc columns frozen on first TMI assignment via CASE guards in executeDeferredTMISync(). Reverted when GDP/GS cancelled or purged (#272)
  • Time Source Prefixing (#269): A/C/E/S prefix on all displayed ETA/ETD values — TIME_ARR_SOURCE/TIME_DEP_SOURCE constants in AdlQueryHelper, new formatTimeWithSource() utility
  • Variable Rate GDP (#269): Quarter-hour rates (rates_quarter_json with DD_HH:MM keys) rendered in demand charts, falling back to hourly/4 or flat/4
  • Canonical Time Constants (#268): TIME_ARR/TIME_DEP/TIME_ARR_METER COALESCE expressions applied to facility.php, summary.php, facility_summary.php

Advisory Generation Improvements

  • Variable rate advisory display with rate tables in NTML format (#259)
  • Recalculate delay metrics before advisory generation in publish (#259)
  • Add missing /HR unit marker to program rate in advisories
  • Power Run internal loopback optimization (localhost:8080 instead of HTTPS roundtrip)

Demand Page Enhancements (PRs #213, #215, #264, #267, #270)

Facility-Level Demand Charts (#213, #215)

  • Facility Demand Mode: New TRACON/ARTCC/FIR/Group demand charts alongside existing airport mode
  • 951 facility tiers configured (421 US, 24 Canadian, 37 Caribbean, 469 global)
  • Measurement modes: airport counts vs boundary crossings
  • facility_summary.php backend for facility breakdown data
  • Facility demand parity with airport mode (#215): matching chart types, overlays, and drill-down

Major Features (#264)

  • TMI Overlay: GS/GDP vertical marker lines with collision avoidance on demand charts
  • Enhanced Filters: Client-side Select2 filters with proportional scaling, URL hash persistence, direction-aware behavior
  • Summary Cards: Breakdown views with flight phase statistics
  • Comparison Mode: Side-by-side chart comparison with "no data" handling
  • Parallel Fetch: Summary added as 7th concurrent fetch in Promise.allSettled (was sequential)

TFMS Integration (#268, #270)

  • All demand queries use TFMS time hierarchy (Actual > Controlled > Estimated > Scheduled)
  • Time source globalization across demand and GDT views
  • Controlled phase checkbox, legend entry, and event handler

Bug Fixes (#267, #270)

  • Fix gray "undefined" bars in comparison and DemandChartCore charts
  • Group chart phases to match sidebar filter categories
  • Match Select2 filter dropdowns to light sidebar theme
  • Fix summary card rendering bugs and rate table styling
  • Fix label overlap and filter styling in comparison mode

Route Analysis System (PRs #172-#180)

Below-map route analysis panel for SWIM playbook routes and filed flight routes.

  • SWIM CDR & Playbook Endpoints (#172): CDR and playbook route data isolation with GIS geometry support, ?name= lookup, daily refdata_sync_daemon.php
  • Route Analysis Panel (#174-#175): UTC timeline, facility filter dropdowns, fix disambiguation, loading indicators, error states
  • Traversal Accuracy (#178): Segment table with per-ARTCC distance/time, route picker dropdown for multi-route comparison
  • Polish (#179): Expanded route display, improved segment columns, export format, picker UX
  • Client Geometry (#180): Analysis uses client-computed geometry directly (eliminates server round-trip)
  • 4-Layer GeoJSON Export: Route lines, ARTCC boundaries, fix markers, sector overlays
  • Shared route-analysis-panel.js module with Mode 1 (client waypoints) / Mode 2 (server PostGIS) resolution

Playbook Enhancements (PRs #185-#196, #235-#242, #246, #265)

Spatial & Geometry (PRs #185-#194)

  • Frozen Route Geometry (#186): GeoJSON stored per-route for AIRAC resilience — geometry survives navdata updates
  • Traversal & TRACON (#185): Fix playbook traversal computation, US sector data, TRACON disambiguation
  • TRACON Import (#187): Correct data mapping with schema migration
  • Copy Routes & Changelog (#188-#189): Copy Routes dropdown menu, changelog diff readability with word-level route diffs
  • Shared RouteAnalysisPanel (#190-#191): Replace custom analysis with shared module, spatial coherence validation for route expansion and airway import
  • Pseudo-Fix Filtering & Coordinate Parsing (#192): 7 fixes including UNKN/VARIOUS filter, amber highlight, traversed column layout, frozen geometry waypoint extraction, distance rounding, UTC time format
  • Antimeridian Fix (#193): Resolve crossing artifacts in route/playbook maps
  • ARTCC Code Misresolution (#194): Fix causing playbook geometry spikes

Filtering & Analysis (PRs #195-#196, #235-#242)

  • Facility Filter Symbology (#195): Endpoint icons and fan connectors for origin/destination markers
  • Compound Boolean Filter (#196): Full query builder with drag-and-drop AND/OR groups, chip removal, blur race condition fixes
  • Per-Play Facility Counts (#235): Route counts with sector coverage computation
  • Dark Theme (#238): Facility count styling for dark mode
  • Multi-Strata Coverage (#239): Sector coverage across superhigh/high/low strata
  • Unique/All Toggle (#240, #242): Toggle between unique and all route counts in facility sidebar

Performance (#265)

  • Fix 3 critical bottlenecks: bulk endpoint optimization, gzip CSV compression, PERTI_MYSQL_ONLY for playbook endpoints

CTP & NAT Operations

CTP Enhanced Demand Charts (#210, #212)

  • Simulated demand charts for CTP sessions with map toggle (#210)
  • Resolution toggle, ocean exit point markers, and constraint overlays (#212)

NAT Track Plotting (#220, #221)

  • PB.NATS live NAT track plotting with labels and timestamps on route map (#220)
  • NAT track label layer toggle with black outline for visibility (#221)

NATOTs Advisory Parser (PR #199)

  • Fetch and parse NAT Organized Track System advisories from FAA fly.faa.gov
  • Floating draggable search panel in route plotter with URL and date inputs
  • Routes grouped by departure facility (BOS, JFK, EWR, PHL, DC METRO, overflights)
  • One-click add/plot individual or bulk routes
  • MySQL nat_track_cache for parsed advisory storage

CTP NAT Track Throughput Pipeline (Migration 048, #244)

  • NAT Track Resolution: Hybrid token + sequence match resolver (NATTrackResolver)
  • CTP Planning Simulator: Compute engine with scenario CRUD, block assignments, track capacity
  • Throughput Config: CRUD API for capacity limits per track (list, create, update, delete, preview)
  • SWIM Sync: CTP-to-SWIM per-flight sync, metrics recompute, throughput utilization
  • SWIM Schema: Migration 030-031 — NAT track metrics, throughput tables, sp_Swim_BulkUpsert v3 with NAT columns
  • Pull-Based Playbook Sync (#244): CTPPlaybookSync service + CTPApiClient with content hash, pull sync state table (Migration 015)
  • VATSWIM CTP Endpoints (#216): Read-only CTP endpoints, WebSocket filtering, API key migration

PostGIS Spatial Enhancements

Antimeridian Handling (PR #256 — Migration 018)

Full antimeridian support across all spatial functions:

  • Helper functions: crosses_antimeridian(), normalize_lon() with ST_ShiftLongitude detection
  • Updated functions: analyze_route_traversal, 4 trajectory crossing functions, get_route_boundaries, analyze_tmi_route, get_route_tracons
  • Routes crossing 180° detected via densification, shifted to [0,360] range, boundaries conditionally shifted, output normalized back to [-180,180]

Two-Pass Route Disambiguation (PR #266 — Migration 020)

New disambiguation strategy for ambiguous waypoint names:

  • Pass 1: Resolve unambiguous waypoints to establish route geometry
  • Pass 2: Re-resolve ambiguous waypoints using spatial proximity to established route line
  • Dropped stale 5-parameter resolve_waypoint() overload

International CIFP Procedure Expansion (#250, #253)

  • Include CIFP source in procedure expansion for international routes (#250)
  • Handle ARINC 424 truncation mismatch for international CIFP procedures (#253)

STAR/DP Procedure Expansion

  • Add STAR and DP procedure expansion to expand_route() — procedures resolved to their constituent waypoints during route parsing

Spatial Safety

  • ST_IsValid pre-filters added to playbook TRACON/sector queries, batch route expansion, trajectory crossing functions, and analyze_route_traversal
  • ST_MakeValid() in analyze_route_traversal for invalid sector geometries
  • Deduplicate consecutive waypoints at procedure boundaries
  • Graceful fallback for cross-variant airway expansion

AIRAC Pipeline & Navigation Data (PRs #198-#202)

Supersession Tracking (PR #198)

  • is_superseded, superseded_cycle, superseded_reason columns on 5 tables across REF, ADL, and PostGIS
  • Changelog generation with _old_ suffix parsing for replaced fixes/airways/procedures
  • PostGIS sync improvements with supersession column support
  • AIRAC 2603 deployed with 179K changelog entries

AIRAC 2603 Update (#200-#202)

  • Update playbook routes to AIRAC 2603 (#200)
  • Org-agnostic ADVZY parsing — supports CANOC, vATCSCC, etc. (#201)
  • Playbook route changelog support with word-level route diffs and i18n (#202)
  • AIRAC historical changelog generation script (2601→2602)

Route Help Dialog Rewrite (#198)

  • Complete rewrite with accurate documentation of all parser features
  • Covers facility types, FIR groups, coordinate formats, mandatory markers, filters, route groups, CDR/playbook expansion
  • Real route examples and 28 new i18n keys

CDM & TMI Coordination (PRs #181-#184)

  • Discord Embed Messages (#181): Convert TMI coordination messages to rich Discord embeds
  • TMI Publish Enhancements (#182): 5 fixes to TMI publishing and coordination workflow
  • Org Scope Fix (#183): Resolve $org_scope scoping bug causing empty active TMI queries
  • ATFCM Sub-fields & ASRT (#184): CDM v2.2.8.25 with ATFCM sub-fields and ASRT support
  • vIFF ATFCM CDM Connector: EU CDM milestone ingestion from vIFF instances
  • CDM Sector Boundaries: Import canonical GeoJSON into both databases

ARTCC Hierarchy Visualization (PRs #138-#139)

  • Boundary Classification (#138): Per-level ARTCC visualization (ARTCC/sub-area/sector)
  • Hierarchy Layers across all map pages (NOD, Route, JATOC, SUA, Playbook) with blue color scheme, dashed sub-areas, per-level line widths, z-ordering
  • VATSpy Boundary Refresh (#139): Upstream data refresh with vertical hierarchy UI
  • Labels rendering fix and low visibility correction

Historical Routes (PRs #229, #237, #246-#248)

New route history analysis page with search, visualization, and comparison tools.

Initial Release (#229)

  • Star schema: dim_route, dim_aircraft_type, dim_operator, dim_time, route_history_facts (RANGE-partitioned by month)
  • ICAO-aware route normalization function
  • Nightly aggregation job for historical route data
  • 3-panel layout: filters + list, map, detail panel
  • Tag-based location filter (Airport/TRACON/ARTCC/FIR modes)
  • Route list with frequency, distance, ETE, last filed — URL state for deep linking
  • Aircraft, operator, and time filter sections
  • Spectral frequency colors with click-to-toggle selection (#237)
  • Map layer toggles, enhanced info dialog with sort/select/minimize
  • Flight detail columns, dep/arr fix stats, airport markers

Enhancements (#237, #246-#248)

  • Alliance, DCC region, and time period quick-selects (#237)
  • Collapsible filters, map tier selector, aircraft family filter
  • Rename nav link from /routes to /historical-routes for disambiguation
  • Green→yellow→red frequency color map (based on % share, replacing blue tiers) (#246)
  • Auto-grouping by OD pair, origin, or destination with collapsible sections and combined stats (#246)
  • Multi-select toolbar: Compare (max 3, side-by-side metrics) and Group (combined stats with contribution bar) (#246)
  • Select all/none with master checkbox, full data suite in group & compare dialogs (#247)
  • Grouped header overflow and stat overlap fixes (#248)
  • Map boundary layer loading refactored to Promise.all() for deterministic render order

JATOC Enhancements

  • Wildcard facilities, multi-lifecycle filter, and UI/date formatting redesign
  • Checkbox lifecycle filter replacing dropdown, dark theme fixes
  • POTUS schedule: ET + UTC dual display, date+time descending sort
  • "Other" option added to INC TYPE filter dropdown
  • Facility type enum and report Name label corrections

Hibernation Mode (PR #118)

Comprehensive operational pause system for off-event periods:

  • HIBERNATION_MODE config flag (config.php + Azure App Setting)
  • Centralized page redirect (12 pages → /hibernation info page)
  • Conditional daemon startup in startup.sh (4 core vs 15 total daemons)
  • Adaptive resource metering for background backfill tasks (#130)
  • Status page hibernation effects display

FAA Advisory Fetch Integration (#208, #209)

  • FAA advisory fetch integration for playbook and route pages
  • ARTCC origin styling in playbook display

Internationalization Updates

  • fr-CA translations brought to 100% coverage (7,560 keys)
  • i18n keys for VATSWIM bridges delivery UI, TOS, and AMAN
  • Remaining hardcoded strings replaced in CTP, routes, navdata, initiative_timeline modules

Pipeline Alignment & Code Quality

ArtccNormalizer (Sitewide)

  • New PERTI\Lib\ArtccNormalizer class replacing 6 inline ARTCC normalizers
  • PostGIS normalize_artcc_code() function
  • Applied in: GISService, analysis API output, expand_route_with_artccs(), analyze_route_traversal()

CORS & Constants

  • Add perti_constants.php include to 74 API endpoints missing perti_set_cors

Connection Reliability

  • MySQL connection timeouts and persistent session storage
  • PostGIS heartbeat added to monitoring daemon

Performance Improvements

PERTI_MYSQL_ONLY Expansion (PR #249)

  • Applied to 8 additional pages (demand, health timeout, demand cache)
  • Playbook endpoints converted to MySQL-only (#265)

Playbook Bottlenecks (PR #265)

  • Bulk endpoint optimization
  • Gzip CSV compression for large dataset responses
  • PERTI_MYSQL_ONLY for playbook API endpoints

Archival Fix

  • sp_Archive_CompletedFlights: Fix flight_status column + trigger FK constraint issue

GDP & GDT Bug Fixes (PRs #271-#273)

  • GDP Auto-Extend (#271): Auto-extend slots to eliminate NO_SLOT overflow for flights beyond program end time
  • ETA/ETD Revert (#272): Revert ETA/ETD from OETA/OETD when GDP/GS is cancelled or purged
  • GS→GDP Auto-Simulate (#273): Auto-simulate GDP after Ground Stop to GDP transition activation
  • Fix GDP mode, quarter rates, Power Run, TMI handoff & demand overlay
  • Resolve duplicate current variable declaration in wrapAdvisoryLabelValue
  • Add missing rates_quarter_json to active.php UNION ALL query

SWIM Bug Fixes

  • WebSocket position timestamp: pos.updated_atpos.position_updated_utc
  • TMI sync advisory_number type mismatch: INTNVARCHAR(16) (source data is 'ADVZY 001' etc.)
  • TMI sync flight_control watermark: modified_utcupdated_at (actual source column name)
  • Correct column names in TMI analytics SWIM API endpoints
  • Resolve 3 critical 500 errors found during SWIM API audit (#219)
  • Capture response_code and response_time_ms in SWIM audit log (#243)
  • Parsedown installation for proper SWIM documentation markdown rendering (#261)

Documentation

  • Wiki Audit: 18 files updated across 4 rounds — API response standardization, jQuery→fetch migration plan, CSS audit, SDK reconciliation, styling guide
  • SWIM API Wiki: Comprehensive wiki page with SWIM vs PERTI distinction, 19 missing endpoints documented
  • Ground Stop Reference: Comprehensive reference from TFMS FSM User's Guide (#257)
  • ICAO SWIM/GUFI Standards Analysis (#262): Standards alignment review
  • Design Specs: TMI Analytics Platform, VATSWIM Splits Bridge, SimTraffic webhook bridge, VATSWIM route query API, demand page enhancements, two-pass disambiguation, CTP NAT track throughput, pipeline alignment
  • Hibernation Runbook: Updated with actual Azure resource tiers and procedures

Version 18

Released: February 2026

Note: System entered hibernation mode on March 9, 2026, then re-entered March 13, 2026 with SWIM exemption. All features below are deployed but many non-SWIM features are inactive during hibernation.

Recent Enhancements (February-March 2026, PRs #142-163)

GDP Algorithm Redesign (Phases 1-4 Complete)

  • Phase 1 (Migration 037): Bug fixes, compression endpoint, batch optimization
  • Phase 2 (Migration 038): FPFS+RBD slot assignment algorithm, adaptive reserves, FlightListType TVP rebuild
  • Phase 3 (Migration 039): sp_TMI_ReoptimizeProgram orchestrator, reoptimize.php endpoint
  • Phase 4 (Migration 041): Reversal metrics, anti-gaming flags, GDT UI (compress/reopt/observability)
  • TMI→ADL sync via executeDeferredTMISync() in ADL daemon (60s cycle)

Playbook Route Enhancements (PRs #143-163)

  • Route Grouping & Coloring (#143) - Auto-group by DCC Region with canonical colors
  • Facility Filter Dropdowns (#144) - Playbook list optimization
  • ATC-Zero Import + Daily Backup Daemon (#146) - Automated playbook backup
  • Forward Group Colors to Route Page (#147) - Consistent coloring
  • Multi-line Remarks & Descriptions (#150) - Rich text support
  • ICAO Prefix FIR Pattern Expansion (#151) - International facility matching
  • Route Editor UX Polish (#152) - Editor improvements
  • FIR Pattern Expansion in Route Rendering (#153) - Dynamic areaCenters from GeoJSON
  • Collapsible Edit Sections + Advisory Parser (#154) - UX improvements
  • FIR Pattern Expansion in Playbook (#155) - Global FIR code registry
  • Token-Type Splitting for Route Detection (#157) - Improved origin/route/dest detection
  • Expand FIR Patterns in Bulk Paste (#159) - Individual ARTCC code expansion
  • Normalize US ICAO ARTCC Codes (#160) - KZAB→ZAB, PAZA→ZAN sitewide
  • FIR Exclusions + Pseudo-Fix Audit + ARTCC Scope (#162) - Member-based scope filtering
  • Route Analysis Tools (#163) - Consolidation, compaction, auto-filters

PostGIS Boundary Fix (#142)

  • Added is_subsector filter to 11 PostGIS ARTCC boundary functions
  • Prevents subsector boundaries from contaminating ARTCC-level queries

TRACON Pseudo-Fixes

  • Added 218 TRACON pseudo-fixes (205 US + 13 Canadian)
  • Enables TRACON-level route filtering in playbook and demand analysis

New Features

Traffic Management Review (TMR) Report System (PR #18)

  • TMR Report - Guided review workflow based on NTMO Guide structure
  • Sidebar navigation with sections: triggers, overview, airport conditions, weather, TMIs, equipment, personnel, findings
  • TMR report CRUD API with auto-save (api/data/review/tmr_report.php)
  • Historical TMI lookup from VATSIM_TMI database (api/data/review/tmr_tmis.php)
  • Bulk NTML paste parser for rapid TMI entry
  • Discord-formatted TMR export (api/data/review/tmr_export.php)
  • Embedded demand charts (DemandChartCore) per plan airport
  • Database migration: r_tmr_reports table

NOD TMI Enhancements & Facility Flows (PRs #19-21)

  • Phase 1: Enhanced TMI sidebar with rich data cards
    • GS cards: countdown timer, flights held, prob extension, origin centers
    • GDP cards: controlled/exempt counts, avg/max delay, compliance bar, GDT link
    • Reroute cards: assigned/compliant counts, compliance bar
    • MIT/AFP section with restriction details and fix coordinates
    • Delay Reports section with severity coloring and trend indicators
    • Map TMI status layer: airport rings by severity, delay glow circles, MIT fix markers, GS pulse animation
  • Phase 2: Facility flow configuration system
    • 3 new tables: facility_flow_configs, facility_flow_elements, facility_flow_gates
    • CRUD APIs for configs, elements, gates, and suggestions
    • Flows tab in NOD sidebar with facility/config selectors
    • Flow element management (fixes, procedures, routes, gates)
    • Inline color picker, visibility toggle, FEA toggle per element
    • Fix/procedure autocomplete from nav_fixes/nav_procedures
    • 8 map layers: boundary, procedure/route lines, fix markers
  • Phase 3-4: Flow map rendering & FEA integration
    • Per-element line weight selector for PROCEDURE/ROUTE elements
    • FEA bridge API (api/nod/fea.php): demand monitor toggle, bulk create/clear
    • Demand count feedback on sidebar and map labels
    • Route GeoJSON LineString support in demand layer

Internationalization (i18n) System (PR #18)

  • Core translation module (PERTII18n) with t(), tp(), formatNumber(), formatDate()
  • Locale loader with auto-detection (URL param, localStorage, browser language)
  • SweetAlert2 dialog wrapper (PERTIDialog) with i18n key resolution
  • 450+ translation keys in assets/locales/en-US.json
  • Integrated across 13 JS modules: demand, gdt, jatoc, nod, plan, review, schedule, sheet, splits, sua, weather_impact, reroute, tmi-publish

Performance Improvements

PERTI_MYSQL_ONLY Optimization (PR #17)

  • define('PERTI_MYSQL_ONLY', true) before include connect.php skips 5 Azure SQL connections
  • Applied to ~98 plan/sheet/review PHP endpoints, saving ~500-1000ms per request
  • Lazy-loaded database connection getters (get_conn_adl(), get_conn_tmi(), etc.)
  • Removed closing ?> tag from connect.php (PSR-12)

Frontend Parallelization (PR #17)

  • plan.js: 16 sequential AJAX calls replaced with Promise.all() batch
  • sheet.js: 5 calls parallelized
  • review.js: 3 calls parallelized

Codebase Cleanup (PR #16)

  • Removed 13 unused files (4,829 lines deleted):
    • reroutes.php (replaced by route.php + tmi-publish.php)
    • advisory-builder.php and advisory-builder.js
    • reroute.js (orphaned, replaced by new module)
    • Legacy admin migration scripts (migrate_public_routes.php, migrate_reroutes.php, etc.)
    • test_star_parsing.php

vATCSCC Playbook System (PRs #75-84)

  • Playbook Page (/playbook.php) - Pre-coordinated route play catalog and management
  • Play CRUD with FAA and DCC source categories (standard/split route formats)
  • MapLibre GL route visualization with sector boundary overlays
  • Shareable playbook links with ?play=NAME URL parameter
  • Play duplication with _MODIFIED suffix for creating variants
  • Bulk paste with ECFMP/CANOC source auto-detection
  • DCC play expansion on route.php with GIS route geometry
  • Client-side filtering by region, category, and status
  • Route remarks field for TMU annotations
  • Playbook changelog with full audit trail
  • Database: playbook_plays, playbook_routes, playbook_changelog tables in perti_site MySQL

Canadian FIR Sector Expansion (PRs #79-84)

  • CZYZ (Toronto) - Low, high, superhigh sector boundaries
  • CZWG (Winnipeg) - Sector boundaries with ESE conversion
  • CZEG (Edmonton) - Sector boundaries
  • CZUL (Montreal) - Sector boundaries
  • CZVR (Vancouver) - Sector boundaries
  • CZQM (Moncton) - 5 low + 27 high = 32 sectors
  • CZQX (Gander) - 3 low + 39 high = 42 sectors
  • Generalized ESE-to-GeoJSON converter handling terminal keywords (RADIO, UNICOM, NO-CONTROL, TRANSITION) and LF-prefix airports
  • Total sector boundaries: 1,379 (1,002 US + 377 Canadian)

Splits Enhancements (PRs #73, #84)

  • Scheduled splits layer with low/high/superhigh strata filtering
  • Sector map visualization on splits.php and plan page Splits tab
  • Personnel tables for staffing assignments
  • Region grouping for multi-ARTCC operations
  • Splits tab added to PERTI plan pages

Ops Plan & Plan Page Enhancements (PRs #67-73)

  • Structured FAA-format Ops Plan with formatted output sections
  • Sortable columns in plan page tables (staffing, configs, initiatives)
  • ARTCC grouping in plan tables for multi-facility events
  • Ops Plan tab with structured output for DCC operations
  • Initiative timeline improvements (facility word-wrap, rotated time axis labels)
  • Auto-select inferred config modifiers, stacked ARR/DEP display

Multi-Organization Support (PRs #68-72)

  • Org-scoped facility authorization for TMI and JATOC endpoints
  • Multi-org Discord posting via tmi_discord_posts queue
  • CANOC and ECFMP flow measure sources integrated
  • Org-aware i18n locale with {commandCenter} template resolution
  • PERTI_ORG config loaded before locale system

TMI Publisher Enhancements (PR #69)

  • TMI Publisher visible in nav for all authenticated users
  • Compact layout with monospace fonts for NTML formatting
  • Org-scoped Discord notification with role mentions

Additional i18n Coverage

  • fr-CA locale (near-complete French Canadian translation)
  • en-CA and en-EU locale overlays
  • i18n integrated across 28 PHP pages and 13+ JS modules
  • BC Canadian airports added to demand page

Bug Fixes

  • Fix: PERTI_MYSQL_ONLY removed from config_data/ endpoints that use Azure SQL (PR #17 hotfix)
  • Fix: GS compliance analysis shows human-readable phase labels
  • Fix: i18n locale loader loads full JSON instead of partial
  • Fix: Missing MapLibre GL and i18n dependencies on review and demand pages
  • Fix: NOD loads i18n scripts required by merged main branch code
  • Fix: i18n key inconsistencies resolved in en-US.json
  • Fix: jQuery .toggle() on <tr> sets display:block — use explicit display:table-row (PR #76)
  • Fix: Playbook login/session handling and navbar visibility
  • Fix: Canadian FIR misclassification (oceanic vs domestic) for CZQM/CZQX
  • Fix: Initiative timeline facility word-wrap on / characters
  • Fix: Homepage table column alignment with table-layout:fixed
  • Fix: Duplicate i18n PHP include causing class redeclaration

Infrastructure Changes

  • VATSIM_ADL migrated from General Purpose Serverless to Hyperscale Serverless (16 vCores, min 3)
  • PostgreSQL GIS database (vatcscc-gis) deployed: Burstable B2s, PostGIS, PostgreSQL 16
  • MySQL upgraded from Burstable to General Purpose D2ds_v4
  • Geo-replica server and VATSIM_Data database decommissioned
  • Monthly costs increased from ~$670 to ~$3,500 (primarily Hyperscale compute)

Version 17

Released: January 2026

New Features

ATFM Training Simulator

  • Simulator Page (/simulator.php) - TMU training environment
  • Node.js flight engine with realistic physics simulation
  • Practice GS/GDP/AFP/MIT/Reroute TMI decisions
  • Reference data: 3,989 O-D route patterns, 107 airports, 17 carriers
  • Web-based interface with MapLibre visualization

Airspace Element Demand

  • Query traffic demand at navigation fixes, airway segments, and route segments
  • Table-valued SQL functions for efficient demand analysis
  • Support for both airway-based and direct (DCT) route queries
  • TRACON-based filtering for facility-specific queries

Config Modifiers System

  • Structured modifier categories for runway configurations
  • Categories: PARALLEL_OPS, APPROACH_TYPE, TRAFFIC_BIAS, VISIBILITY_CAT, SPECIAL_OPS, TIME_RESTRICT, WEATHER_OPS, NAMED
  • Migrated from free-text modifiers to normalized schema

ATIS Type Priority Logic

  • Enhanced ATIS source selection: ARR+DEP > COMB > single
  • Views for effective ATIS determination across split ATIS scenarios
  • Improved weather data sourcing for rate suggestions

API Additions (v17)

  • GET /api/simulator/navdata.php - Navigation data for routing
  • GET/POST /api/simulator/engine.php - Engine control
  • GET /api/simulator/routes.php - Route pattern data
  • GET/POST /api/simulator/traffic.php - Traffic generation
  • GET /api/adl/demand/fix.php - Flights at a navigation fix
  • GET /api/adl/demand/airway.php - Flights on an airway segment
  • GET /api/adl/demand/segment.php - Flights between two fixes (airway or DCT)

Database Changes (v17)

  • New tables: sim_ref_carrier_lookup, sim_ref_route_patterns, sim_ref_airport_demand
  • New tables: modifier_category, modifier_type, config_modifier
  • New functions: fn_FixDemand, fn_AirwaySegmentDemand, fn_RouteSegmentDemand
  • New views: vw_current_atis_by_type, vw_effective_atis, vw_config_with_modifiers, vw_runway_with_modifiers
  • New indexes: IX_waypoint_fix_eta, IX_waypoint_airway_eta (filtered indexes for demand queries)
  • Migrations 092-095: Config modifiers, ATIS type priority
  • Migrations demand/001-004: Airspace demand indexes and functions
  • Data sourced from BTS On-Time Performance (20.6M flight records)

Version 16

Released: January 2026

New Features

Demand Analysis System

  • Demand Page (/demand.php) - Airport demand visualization
  • Weather-aware rate suggestions with confidence scoring
  • Manual rate override support with time windows
  • Multi-level rate suggestion algorithm

Airport Configuration & ATIS

  • Normalized runway configuration schema
  • VATSIM ATIS import with weather extraction
  • Runway-in-use detection from ATIS parsing
  • Flight-track-based runway detection as fallback
  • Rate change audit trail

API Additions

  • GET /api/demand/airports.php - Airport list
  • GET /api/demand/summary.php - Demand summary
  • GET /api/demand/rates.php - Rate data
  • POST /api/demand/override.php - Manual override

Database Changes

  • Migrations 079-091 (Airport config, ATIS, rates)
  • New tables: airport_config, airport_config_runway, airport_config_rate
  • New tables: runway_in_use, manual_rate_override, rate_history

Version 15

Released: December 2025

New Features

GDT Ground Stop NTML Architecture

  • Complete program lifecycle management
  • Stored procedure-based workflow
  • Pop-up flight detection
  • EDCT issuance and management

Weather Radar Integration

  • IEM NEXRAD/MRMS tile integration
  • Multiple color table options
  • Configurable opacity and layers

SUA/TFR Display

  • Special Use Airspace boundaries on map
  • TFR visualization
  • Active/inactive filtering

Initiative Timeline

  • Gantt-style TMI visualization
  • Interactive timeline navigation
  • Multiple initiative tracking

API Changes

  • New GS API endpoints (/api/tmi/gs/*)
  • Enhanced flight data responses
  • WebSocket support preparation

Database Changes

  • NTML schema (tmi/001_ntml_schema.sql)
  • GS procedures (tmi/002_gs_procedures.sql)
  • GDT views (tmi/003_gdt_views.sql)
  • Phase column unification in ADL

Version 14

Released: October 2025

New Features

  • ETA calculation with aircraft performance
  • Trajectory logging and visualization
  • Zone detection (OOOI) implementation
  • Boundary crossing detection

Improvements

  • Route parsing accuracy enhancements
  • SimBrief flight plan integration
  • Performance optimizations for large events

Version 13

Released: August 2025

New Features

  • JATOC incident management
  • NOD dashboard
  • Public route sharing
  • Discord webhook integration

API Changes

  • JATOC CRUD endpoints
  • NOD data endpoints
  • Public routes API

Version 12

Released: June 2025

New Features

  • Splits sector configuration
  • Route Plotter TSD interface
  • Weather alert integration

Improvements

  • MapLibre GL JS migration
  • Performance improvements
  • Mobile responsiveness

Version 11

Released: April 2025

New Features

  • Ground Delay Program (GDP) support
  • EDCT management
  • Enhanced planning worksheets

Version 10

Released: February 2025

New Features

  • Initial PERTI platform release
  • VATSIM OAuth integration
  • Basic planning tools
  • Ground Stop prototype

Migration Notes

Upgrading to v19

  1. Apply database/migrations/swim/026_swim_data_isolation.sql to SWIM_API (DDL admin: jpeterson)
  2. Run initial data load for all mirror tables (TMI sync daemon handles this automatically on first run)
  3. Deploy updated scripts/swim_sync.php (60 new columns in ADL SELECT/JSON mapping)
  4. Deploy scripts/swim_tmi_sync_daemon.php and scripts/refdata_sync_daemon.php
  5. Update scripts/startup.sh to start new sync daemons
  6. Verify all SWIM endpoints return identical data post-migration
  7. Confirm DTU stays below 80% peak on Azure portal

Upgrading to v18

  1. Apply NOD flow migrations: database/migrations/nod/001_facility_flow_tables.sql, 002_flow_element_fea_linkage.sql
  2. Apply TMR migration: r_tmr_reports table in perti_site MySQL
  3. Apply playbook migrations: database/migrations/playbook/001-004 in perti_site MySQL
  4. Import Canadian sector GeoJSON files to assets/data/ directory
  5. Deploy new API endpoints: api/nod/flows/*, api/nod/fea.php, api/data/review/tmr_*.php, api/data/playbook/*, api/mgt/playbook/*
  6. Ensure assets/locales/en-US.json, fr-CA.json, and assets/js/lib/i18n.js are deployed
  7. Verify PERTI_MYSQL_ONLY is NOT set in any files that use Azure SQL connections

Upgrading to v17

  1. Apply migrations 092-095 to Azure SQL (config modifiers, ATIS priority)
  2. Apply migrations demand/001-004 to Azure SQL (airspace demand functions)
  3. Deploy new API endpoints (api/adl/demand/*)
  4. Install Node.js simulator engine if using ATFM Simulator

Upgrading to v16

  1. Apply migrations 079-091 to Azure SQL
  2. Update ATIS daemon to latest version
  3. Configure demand page access
  4. Review rate suggestion settings

Upgrading to v15

  1. Apply TMI migrations (001-003)
  2. Update GDT frontend components
  3. Configure weather tile sources
  4. Test GS workflow end-to-end

Deprecations

v18

  • reroutes.php removed (use route.php + tmi-publish.php)
  • advisory-builder.php / advisory-builder.js removed (functionality in tmi-publish)
  • Legacy admin migration scripts removed
  • Hardcoded English strings in JS (use PERTII18n.t() for new strings)

v16

  • Legacy GDP endpoints (use new GDT API)
  • Old demand calculation views (replaced by procedures)

v15

  • flight_status column (replaced by phase)
  • Legacy ground stop tables (migrated to NTML)

Upcoming (Planned)

v20 (Planned - Post-Hibernation)

  • CDM Adaptation (hybrid FAA/EUROCONTROL/AMNAC model, 6 phases)
  • Daemon reoptimization cycle for GDP (2-5 min automatic reopt)
  • TMI Historical Import (NTML compact + ADVZY parsers)
  • Change feed delta endpoint (GET /api/swim/v1/changes?since_seq=<n>&limit=<m>) for external consumers
  • Reroute compliance automation
  • StatSim v2 integration
  • Additional i18n locale support

See Also

Clone this wiki locally