Skip to content

plan.md

Latest commit

 

History

History
249 lines (236 loc) · 13.2 KB

plan.md

File metadata and controls

249 lines (236 loc) · 13.2 KB

Execution Checklist: PostgreSQL + ORM + Containers Only

Phase 1: Cleanup legacy imports

  • Remove modules: plexpy/plexwatch_import.py, plexpy/plexivity_import.py
  • Update API endpoint imports and logic: plexpy/webserve.py (remove PlexWatch/Plexivity paths)
  • Remove template/JS references if any: plexpy/web/assets/interfaces/** (search for import UI)
  • Update docs/API references: README.md, API.md, CHANGELOG.md
  • Success criteria: import endpoints removed; UI has no legacy import entry points
  • Tests: app starts; import endpoints removed without 500s

Phase 2: Project organization (no behavior change)

  • Add package directories under plexpy/:
    • plexpy/app/
    • plexpy/config/
    • plexpy/db/
    • plexpy/web/
    • plexpy/services/
    • plexpy/integrations/
    • plexpy/util/
    • plexpy/platform/
  • Move files (no logic changes), update imports:
    • plexpy/__init__.py -> plexpy/app/bootstrap.py (keep a minimal plexpy/__init__.py for globals if needed)
    • plexpy/webserve.py -> plexpy/web/webserve.py
    • plexpy/api2.py -> plexpy/web/api2.py
    • plexpy/webauth.py -> plexpy/web/webauth.py
    • plexpy/session.py -> plexpy/web/session.py
    • plexpy/config.py -> plexpy/config/core.py
    • plexpy/database.py -> plexpy/db/sqlite_legacy.py (temporary until removed)
    • plexpy/helpers.py -> plexpy/util/helpers.py
    • plexpy/logger.py -> plexpy/util/logger.py
    • plexpy/request.py -> plexpy/util/request.py
    • plexpy/lock.py -> plexpy/util/lock.py
    • plexpy/exceptions.py -> plexpy/util/exceptions.py
    • plexpy/webstart.py -> plexpy/web/webstart.py
    • plexpy/web_socket.py -> plexpy/web/web_socket.py
    • plexpy/plextv.py -> plexpy/integrations/plextv.py
    • plexpy/plex.py -> plexpy/integrations/plex.py
    • plexpy/pmsconnect.py -> plexpy/integrations/pmsconnect.py
    • plexpy/http_handler.py -> plexpy/integrations/http_handler.py
    • plexpy/common.py -> plexpy/app/common.py
    • plexpy/version.py -> plexpy/app/version.py
    • plexpy/activity_* -> plexpy/services/
    • plexpy/notification_* -> plexpy/services/
    • plexpy/newsletter_* -> plexpy/services/
    • plexpy/users.py -> plexpy/services/users.py
    • plexpy/libraries.py -> plexpy/services/libraries.py
    • plexpy/datafactory.py -> plexpy/db/datafactory.py
    • plexpy/datatables.py -> plexpy/db/datatables.py
    • plexpy/graphs.py -> plexpy/services/graphs.py
    • plexpy/exporter.py -> plexpy/services/exporter.py
    • plexpy/log_reader.py -> plexpy/services/log_reader.py
    • plexpy/mobile_app.py -> plexpy/services/mobile_app.py
    • plexpy/notifiers.py -> plexpy/services/notifiers.py
    • plexpy/newsletters.py -> plexpy/services/newsletters.py
    • plexpy/versioncheck.py -> plexpy/services/versioncheck.py
    • plexpy/macos.py -> plexpy/platform/macos.py
    • plexpy/windows.py -> plexpy/platform/windows.py
  • Keep entrypoint shim: Tautulli.py -> call plexpy.app.main
  • Add backward-compatible import shims or re-export modules for moved files
  • Add architecture doc: docs/architecture.md
  • Success criteria: imports resolve with new paths; shims prevent breakage
  • Tests: app starts; core routes/UI load with new import paths -> you can test the app by running bash test_start.sh

Phase 2.5: Docker-only cleanup + asset relocation

  • Remove non-Docker root folders and workflows:
    • snap/
    • init-scripts/
    • installer packaging assets in package/
    • .github/workflows/publish-installers.yml
    • .github/workflows/publish-snap.yml
  • Update docs to reflect Docker-only support:
    • README.md
    • CONTRIBUTING.md
    • CHANGELOG.md
  • Relocate UI assets:
    • Move data/ -> plexpy/web/assets/
    • Update asset paths in templates/JS/Python
    • Update Dockerfile/runtime paths that reference /app/data
  • Success criteria: assets load from new root; Docker image runs without old paths
  • Tests: Docker build succeeds; UI assets served from new paths

Phase 2.6: UI templates + static asset wiring

  • Update CherryPy static mounts to new asset root:
    • plexpy/web/webstart.py (paths for /css, /js, /images, /fonts, /interfaces)
  • Update Mako template asset URLs:
    • plexpy/web/assets/interfaces/default/base.html (CSS/JS/image includes)
  • Confirm template lookup path for interfaces:
    • plexpy/web/webserve.py (TemplateLookup root)
  • Success criteria: template lookup uses new root; assets resolve under new URLs
  • Tests: login page renders; CSS/JS/images load without 404s

Phase 3: ORM 2.0 foundation

  • Add DB core:
    • plexpy/db/engine.py (Postgres engine + pool config)
    • plexpy/db/session.py (SessionLocal + context manager)
  • Add ORM models:
    • plexpy/db/models/__init__.py
    • plexpy/db/models/*.py (one per table or grouped)
  • Add repository layer:
    • plexpy/db/repository/*.py
  • Define ORM conventions:
    • SQLAlchemy 2.0 style, naming convention for constraints/indexes
    • UTC timezone handling; explicit sequence behavior for integer PKs
  • Add config keys for Postgres connection:
    • plexpy/config/core.py
  • Phase 3 completion plan (SQLite is source of truth):
    • Ensure SQLAlchemy 2.x is pinned in requirements.txt
    • Build a schema comparison test that:
      • Creates a SQLite in-memory DB using the CREATE TABLE SQL in plexpy/app/bootstrap.py
      • Introspects table/column/PK/index definitions via SQLAlchemy inspector
      • Compares against ORM metadata and reports mismatches (names, nullability, defaults, indexes)
    • Add a Postgres smoke test that creates the ORM metadata and verifies engine/session
  • Success criteria: models map to current schema without destructive diffs
  • Tests: engine/session creation works; model metadata loads

Phase 4: Alembic migrations

  • Add Alembic config:
    • alembic.ini
    • plexpy/db/migrations/ (env + versions)
  • Generate initial migration from ORM models with schema diff review
  • Remove schema creation from app startup:
    • plexpy/app/bootstrap.py (remove dbcheck() schema creation)
  • Add migration version check at startup
  • Add migration entrypoint for fresh installs (auto-init empty DB)
  • Add explicit migrate command for existing installs (no auto-destructive changes)
  • Success criteria: fresh container initializes DB; existing DB requires explicit migrate
  • Tests: alembic upgrade head works on empty DB; version check blocks mismatches

Phase 5: Setup wizard migration flow (manual trigger)

  • Add wizard UI step for migration:
    • plexpy/web/assets/interfaces/default/*.html (setup wizard templates)
    • plexpy/web/assets/interfaces/default/js/* (wizard JS if present)
  • Add upload handler and migration kickoff:
    • plexpy/web/webserve.py
  • Add confirmation dialog:
    • Warn about overwriting existing Postgres DB
  • Success criteria: wizard exposes migration flow and protects existing Postgres
  • Tests: wizard flow triggers migration endpoint; confirmation required

Phase 6: One-time migration tool (SQLite -> Postgres)

  • Add migration runner module:
    • plexpy/db/migrate_sqlite.py
  • Implement:
    • SQLite file validation
    • Postgres truncation in dependency order
    • Bulk inserts via ORM
    • Type normalization (bools/timestamps/text)
    • Row count and integrity checks
    • Index/constraint verification
    • Sequence alignment after inserts
    • Migration report logging
  • Wire migration runner to wizard endpoint
  • Success criteria: migrated DB passes data integrity and schema verification
  • Tests: migrate sample SQLite -> Postgres; row counts + key integrity checks pass

Phase 7: Postgres-only cleanup

  • Remove sqlite3 imports and SQLite code paths:
    • plexpy/db/sqlite_legacy.py (delete when no longer used)
    • plexpy/app/bootstrap.py
    • plexpy/services/* (replace SQL calls with ORM)
  • Remove SQLite settings from config
  • Update backup tooling to Postgres (pg_dump backups, dump download, integrity check)
  • Update export tooling to Postgres
  • Success criteria: no sqlite3 imports remain; backups/exports work on Postgres
  • Tests: app runs without sqlite3; key features operate via ORM
  • Next step: Ensure pg_dump is available in the container and verify backup_db + download_database in the UI/API

Phase 8: Linux containers only + Python 3.15

  • Update container base:
    • Dockerfile (Python 3.15, gated by dependency compatibility)
  • Simplify entry script:
    • Docker entrypoint runs via CMD (no start.sh)
  • Verify non-container CI workflows removed in Phase 2.5
  • Update docs to container-only + Python 3.15
  • Success criteria: container runs on target Python; docs match runtime support
  • Tests: container build + start; runtime smoke on Python 3.15

Phase 9: Type safety rollout

  • Add typing config:
    • pyproject.toml
  • Start with DB layer:
    • plexpy/db/engine.py, plexpy/db/session.py, plexpy/db/models/*.py
  • Expand to services and web layer:
    • plexpy/services/*, plexpy/web/*, plexpy/config/*
  • Incrementally raise strictness in config
  • Success criteria: typing gate passes for targeted layers
  • Tests: type checker passes at current strictness; no runtime regressions

Phase 10: Linux-only runtime + de-vendor libs (Docker builds)

  • Remove Windows/macOS/Snap runtime code:
    • Delete OS modules: plexpy/platform/windows.py, plexpy/platform/macos.py
    • Remove platform-specific imports/branches:
      • Tautulli.py (Windows/macOS imports, Snap env/migration)
      • plexpy/web/webserve.py (Windows/macOS imports)
      • plexpy/services/versioncheck.py (install type = snap/windows/macos)
      • plexpy/__init__.py (tray globals, platform-specific restart/alerts)
      • plexpy/services/notification_handler.py (update assets for .exe/.pkg)
      • plexpy/services/notifiers.py (remove OSX notifier agent)
    • Update update-bar messaging in plexpy/web/assets/interfaces/default/base.html for Docker-only installs
  • Remove Windows/macOS/Snap packaging assets (if not already removed in Phase 2.5/8):
    • snap/, package/, init-scripts/init.osx, contrib/clean_pyc.bat
    • .github/workflows/publish-snap.yml, .github/workflows/publish-installers.yml, .github/workflows/submit-winget.yml
    • Clean docs: README.md, CHANGELOG.md, .gitignore, .dockerignore
  • De-vendor lib/ so deps are installed via pip:
    • Move lib/hashing_passwords.py -> plexpy/util/hashing_passwords.py
    • Move lib/certgen.py -> plexpy/util/certgen.py
    • Update imports in plexpy/web/webserve.py, plexpy/web/webauth.py, plexpy/web/api2.py, plexpy/config/core.py, plexpy/util/helpers.py
    • Remove sys.path manipulation in Tautulli.py
    • Delete lib/ after all imports are resolved
    • Ensure all runtime deps are in requirements.txt and Docker build runs pip install -r requirements.txt
  • Keep client platform icons/mappings (Windows/macOS Plex clients) intact
  • Success criteria: platform-specific code removed without breaking UI mappings
  • Tests: app runs without vendored libs; pip deps install cleanly in Docker build

Phase 11: ORM/Core migration (Postgres runtime, SQLite migration-only)

  • Establish a query layer for Core/ORM results:
    • plexpy/db/queries/__init__.py
    • helpers for consistent mappings() results, pagination, and error handling
  • Add Core time helpers for Postgres-only functions:
    • timezone, to_char, extract, epoch utilities using sqlalchemy.func
    • centralize in plexpy/db/queries/time.py
  • Convert low-risk CRUD to ORM/Core (remove MonitorDatabase usage):
    • plexpy/services/activity_processor.py
    • plexpy/services/activity_pinger.py
    • plexpy/services/notifiers.py
    • plexpy/services/users.py (simple reads/updates)
  • Convert reporting queries to SQLAlchemy Core (Postgres-specific allowed):
    • plexpy/services/graphs.py (replace string-built SQL)
    • plexpy/services/users.py LATERAL datatable stats
    • plexpy/services/libraries.py LATERAL datatable stats
    • plexpy/services/notifiers.py LATERAL last-notify lookup
  • Convert plexpy/db/datafactory.py stats queries to Core:
    • move total-duration raw SQL into plexpy/db/queries/raw_pg.py
    • use lateral() and Postgres distinct on (SQLAlchemy supports PG dialect)
    • keep any remaining raw SQL isolated in plexpy/db/queries/raw_pg.py
  • Keep plexpy/db/datatables.py as raw SQL, but:
    • tighten parameter binding and consolidate SQL construction utilities
    • document that this module is intentionally raw SQL
  • Add/extend FK constraints and relationships where safe:
    • session history <-> metadata/media_info, users, notifiers/newsletters
    • add Alembic migrations for new constraints
    • add preflight checks for orphans before enabling FKs
  • Ensure SQLite -> Postgres migration remains compatible:
    • update plexpy/db/migrate_sqlite.py if new constraints require ordering or cleanup
    • validate Base.metadata.sorted_tables order is still acyclic
  • Update docs to state: Postgres-only runtime, SQLite is migration-only
    • README.md, docs/architecture.md, plan.md
  • Success criteria: >=80% of raw SQL in services moved to ORM/Core; remaining raw SQL isolated
  • Tests: parity checks for key stats queries; migration still completes on sample SQLite DB