Rc/v5.16.0

CHANGELOG.md

@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [5.16.0] - 2026-09-18
+
+### Added
+
+- **Client–team messaging** — New `ClientMessage` model, message service, and UI for bidirectional in-portal messaging between the internal team and client contacts; threads visible from both the client view and the client portal (migration **194**).
+- **Email thread sync (Gmail & Outlook)** — `GmailConnector` and `OutlookEmailConnector` pull email threads from Gmail API and Microsoft Graph and link them to CRM clients, leads, and deals; `EmailSyncService` handles OAuth token refresh and incremental sync; dedicated setup wizards for both providers.
+- **Payroll sync (Gusto & ADP)** — `GustoConnector` and `AdpConnector` aggregate time entries into payroll batches and push them to Gusto's Partner API and ADP Workforce Now; `PayrollSyncService` builds period-scoped batches and `PayrollSyncLog` tracks sync history (migration **194**).
+- **DATEV accounting export** — `DatevConnector` generates EXTF Buchungsstapel CSV for direct import into DATEV; `datev_export.py` utility handles the format spec.
+- **Sage Business Cloud integration** — `SageConnector` syncs invoices, contacts, and payments with Sage Business Cloud Accounting via OAuth2.
+- **Integration setup wizards** — Guided step-by-step wizards for ADP, DATEV, Gmail, Gusto, Outlook Email, and Sage make credential configuration consistent with existing integrations.
+- **Visual workflow builder** — Drag-and-drop canvas (`visual_builder.html`) for constructing automation workflows without editing JSON; accessible alongside the existing form editor.
+- **Portal custom domain resolution** — `portal_domain.py` utility resolves white-label client portal hosts to the correct `Client` record, enabling custom-domain client portals when `portal_allowed_custom_domains` is enabled.
+- **Client Portal REST API** — `api_v1_client_portal.py` blueprint exposes authenticated REST endpoints for portal sessions and client data access.
+
+### Documentation
+
+- **Version** — Bumped `setup.py` to **5.16.0** (single source of truth for the application version).
+
 ## [5.15.0] - 2026-09-16
 
 ### Added

README.md

@@ -105,6 +105,10 @@ TimeTracker has been continuously enhanced with powerful new features! Here's wh
 
 **Current version** is defined in `setup.py` (single source of truth). See [CHANGELOG.md](CHANGELOG.md) for versioned release history.
 
+### ✨ Highlights of v5.16.0
+
+**Minor (5.16.0):** **Client–team messaging** — bidirectional in-portal messaging between team and client contacts with thread UI on both sides. **Gmail & Outlook sync** — email threads pulled from Gmail API and Microsoft Graph and linked to CRM clients, leads, and deals. **Payroll sync (Gusto & ADP)** — time entries aggregated into payroll batches and pushed to Gusto and ADP Workforce Now. **DATEV export** — EXTF Buchungsstapel CSV generator for direct DATEV import. **Sage integration** — invoices, contacts, and payments synced with Sage Business Cloud. **Integration wizards** — guided setup wizards for ADP, DATEV, Gmail, Gusto, Outlook Email, and Sage. **Visual workflow builder** — drag-and-drop canvas for building automation workflows. **Portal custom domains** — white-label client portal host resolution. **Client Portal API** — new authenticated REST blueprint for portal sessions and data access. See [CHANGELOG.md](CHANGELOG.md#5160---2026-09-18).
+
 ### ✨ Highlights of v5.15.0
 
 **Minor (5.15.0):** **Timer start override (#760)** — start or adjust a running timer at a custom time. **Pomodoro sessions** — focus blocks tracked via timer API and UI. **Expense lifecycle** — full mobile/desktop expense CRUD. **Payroll templates** — configurable export templates. **QuickBooks & Xero** — deeper accounting sync. **ActivityWatch inbox** — rules, merge filters, and sync-error review. **Gamification** — badges, leaderboards, and award hooks. **Calendar DnD** — drag-to-move and resize events. **Recurring costs** — automatic recurring project cost engine. **Shareable reports** — public tokenized report links. **Geofencing** — location-based attendance clock-in policies. See [CHANGELOG.md](CHANGELOG.md#5150---2026-09-16).

app/__init__.py

@@ -524,6 +524,25 @@ def _generate_csp_nonce():
 
         g.csp_nonce = secrets.token_urlsafe(16)
 
+    # White-label client portal: resolve Host to a Client.custom_domain when enabled.
+    @app.before_request
+    def _resolve_portal_custom_domain():
+        try:
+            from app.utils.portal_domain import bind_portal_client_to_g
+
+            bind_portal_client_to_g()
+        except Exception:
+            g.portal_client = None
+
+        # On a custom portal hostname, send bare "/" to the client portal.
+        try:
+            if getattr(g, "portal_client", None) is not None and request.path in ("/", ""):
+                from flask import redirect, url_for
+
+                return redirect(url_for("client_portal.login"))
+        except Exception:
+            pass
+
     # Remember the public base URL from real requests so background jobs can build
     # absolute links without SERVER_NAME (see app.utils.urls).
     @app.before_request

app/blueprint_registry.py

@@ -72,6 +72,7 @@ def register_all_blueprints(app, logger=None):
     from app.routes.api_v1_projects import api_v1_projects_bp
     from app.routes.api_v1_tasks import api_v1_tasks_bp
     from app.routes.api_v1_time_entries import api_v1_time_entries_bp
+    from app.routes.api_v1_client_portal import api_v1_client_portal_bp
     from app.routes.auth import auth_bp
     from app.routes.budget_alerts import budget_alerts_bp
     from app.routes.calendar import calendar_bp
@@ -182,6 +183,7 @@ def register_all_blueprints(app, logger=None):
     app.register_blueprint(api_v1_leads_bp)
     app.register_blueprint(api_v1_contacts_bp)
     app.register_blueprint(api_v1_issues_bp)
+    app.register_blueprint(api_v1_client_portal_bp)
     app.register_blueprint(api_docs_bp)
     app.register_blueprint(swaggerui_blueprint)
     app.register_blueprint(analytics_bp)

app/integrations/adp.py

@@ -0,0 +1,168 @@
+"""ADP Workforce Now payroll connector."""
+
+import logging
+import os
+from datetime import datetime, timedelta
+from typing import Any, Dict
+from urllib.parse import urlencode
+
+import requests
+
+from app.integrations.base import BaseConnector
+
+logger = logging.getLogger(__name__)
+
+
+class AdpConnector(BaseConnector):
+    """ADP Workforce Now — OAuth2 client credentials / auth code + payroll batches."""
+
+    display_name = "ADP Workforce Now"
+    description = "Push payroll batches to ADP Workforce Now"
+    icon = "adp"
+
+    AUTH_URL = "https://accounts.adp.com/auth/oauth/v2/authorize"
+    TOKEN_URL = "https://accounts.adp.com/auth/oauth/v2/token"
+    API_BASE = "https://api.adp.com"
+
+    @property
+    def provider_name(self) -> str:
+        return "adp"
+
+    def _creds(self):
+        from app.models import Settings
+
+        settings = Settings.get_settings()
+        c = settings.get_integration_credentials("adp")
+        return {
+            "client_id": c.get("client_id") or os.getenv("ADP_CLIENT_ID"),
+            "client_secret": c.get("client_secret") or os.getenv("ADP_CLIENT_SECRET"),
+        }
+
+    def get_authorization_url(self, redirect_uri: str, state: str = None) -> str:
+        c = self._creds()
+        if not c["client_id"]:
+            raise ValueError("ADP_CLIENT_ID not configured")
+        params = {
+            "client_id": c["client_id"],
+            "response_type": "code",
+            "redirect_uri": redirect_uri,
+            "scope": "openid",
+            "state": state or "",
+        }
+        return f"{self.AUTH_URL}?{urlencode(params)}"
+
+    def exchange_code_for_tokens(self, code: str, redirect_uri: str) -> Dict[str, Any]:
+        c = self._creds()
+        r = requests.post(
+            self.TOKEN_URL,
+            data={
+                "grant_type": "authorization_code",
+                "code": code,
+                "redirect_uri": redirect_uri,
+                "client_id": c["client_id"],
+                "client_secret": c["client_secret"],
+            },
+            timeout=30,
+        )
+        r.raise_for_status()
+        data = r.json()
+        expires_at = datetime.utcnow() + timedelta(seconds=int(data.get("expires_in", 3600)))
+        return {
+            "access_token": data.get("access_token"),
+            "refresh_token": data.get("refresh_token"),
+            "expires_at": expires_at.isoformat(),
+            "token_type": data.get("token_type", "Bearer"),
+        }
+
+    def refresh_access_token(self) -> Dict[str, Any]:
+        c = self._creds()
+        # Prefer refresh_token; fall back to client_credentials for server apps
+        if self.credentials and self.credentials.refresh_token:
+            data_body = {
+                "grant_type": "refresh_token",
+                "refresh_token": self.credentials.refresh_token,
+                "client_id": c["client_id"],
+                "client_secret": c["client_secret"],
+            }
+        else:
+            data_body = {
+                "grant_type": "client_credentials",
+                "client_id": c["client_id"],
+                "client_secret": c["client_secret"],
+            }
+        r = requests.post(self.TOKEN_URL, data=data_body, timeout=30)
+        r.raise_for_status()
+        data = r.json()
+        expires_at = datetime.utcnow() + timedelta(seconds=int(data.get("expires_in", 3600)))
+        if self.credentials:
+            self.credentials.access_token = data["access_token"]
+            if data.get("refresh_token"):
+                self.credentials.refresh_token = data["refresh_token"]
+            self.credentials.expires_at = expires_at
+            from app import db
+
+            db.session.commit()
+        return {"access_token": data["access_token"], "expires_at": expires_at.isoformat()}
+
+    def test_connection(self) -> Dict[str, Any]:
+        token = self.get_access_token()
+        if not token:
+            return {"success": False, "message": "Not authenticated"}
+        r = requests.get(
+            f"{self.API_BASE}/hr/v2/workers",
+            headers={"Authorization": f"Bearer {token}", "Accept": "application/json"},
+            params={"$top": 1},
+            timeout=20,
+        )
+        if r.status_code in (200, 401, 403):
+            # 401/403 still prove reachability; partnership scopes vary
+            if r.status_code == 200:
+                return {"success": True, "message": "ADP connection OK"}
+            return {"success": True, "message": f"ADP reachable (HTTP {r.status_code} — check scopes)"}
+        return {"success": False, "message": f"HTTP {r.status_code}: {r.text[:200]}"}
+
+    def sync_data(self, sync_type: str = "full") -> Dict[str, Any]:
+        from app.services.payroll_sync_service import PayrollSyncService
+
+        return PayrollSyncService().push_period(
+            provider="adp",
+            integration=self.integration,
+            connector=self,
+        )
+
+    def push_payroll_batch(self, batch: Dict[str, Any]) -> Dict[str, Any]:
+        token = self.get_access_token()
+        if not token:
+            return {"success": False, "message": "Missing access token"}
+        payload = {
+            "events": [
+                {
+                    "data": {
+                        "eventContext": {
+                            "payrollGroupCode": (self.integration.config or {}).get("payroll_group_code", "DEFAULT"),
+                        },
+                        "transform": {
+                            "payDataInput": {
+                                "payrollPeriodStartDate": batch["period_start"],
+                                "payrollPeriodEndDate": batch["period_end"],
+                                "workers": batch.get("employees", []),
+                            }
+                        },
+                    }
+                }
+            ]
+        }
+        r = requests.post(
+            f"{self.API_BASE}/events/payroll/v1/pay-data-input.modify",
+            headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"},
+            json=payload,
+            timeout=60,
+        )
+        if r.status_code in (200, 201, 202):
+            data = r.json() if r.content else {}
+            return {
+                "success": True,
+                "external_batch_id": str(data.get("events", [{}])[0].get("eventID") or ""),
+                "raw": data,
+            }
+        return {"success": False, "message": f"HTTP {r.status_code}: {r.text[:300]}"}

app/integrations/datev.py

@@ -0,0 +1,82 @@
+"""DATEV accounting connector (file-based EXTF / Buchungsstapel export)."""
+
+import logging
+from datetime import datetime
+from typing import Any, Dict, List
+
+from app.integrations.base import BaseConnector
+from app.utils.datev_export import build_datev_buchungsstapel
+
+logger = logging.getLogger(__name__)
+
+
+class DatevConnector(BaseConnector):
+    """DATEV export connector — generates EXTF CSV for Buchungsstapel import."""
+
+    display_name = "DATEV"
+    description = "Export invoices as DATEV EXTF Buchungsstapel CSV"
+    icon = "datev"
+
+    @property
+    def provider_name(self) -> str:
+        return "datev"
+
+    def get_authorization_url(self, redirect_uri: str, state: str = None) -> str:
+        # File-based — no OAuth; return a stub that integrations UI can skip
+        return redirect_uri or "/"
+
+    def exchange_code_for_tokens(self, code: str, redirect_uri: str) -> Dict[str, Any]:
+        return {
+            "access_token": "datev-file-export",
+            "refresh_token": None,
+            "expires_at": None,
+            "token_type": "none",
+        }
+
+    def refresh_access_token(self) -> Dict[str, Any]:
+        return {"access_token": "datev-file-export", "expires_at": None}
+
+    def test_connection(self) -> Dict[str, Any]:
+        config = (self.integration.config if self.integration else {}) or {}
+        consultant = config.get("consultant_number") or config.get("berater_nr")
+        client_nr = config.get("client_number") or config.get("mandant_nr")
+        if not consultant or not client_nr:
+            return {
+                "success": False,
+                "message": "Configure consultant number (Berater-Nr) and client number (Mandanten-Nr)",
+            }
+        return {"success": True, "message": f"DATEV ready (Berater {consultant}, Mandant {client_nr})"}
+
+    def sync_data(self, sync_type: str = "full") -> Dict[str, Any]:
+        """Generate DATEV export content and store path/summary on the integration."""
+        from app.models import Invoice
+
+        config = (self.integration.config if self.integration else {}) or {}
+        invoices: List = Invoice.query.filter(Invoice.status.in_(["sent", "paid"])).limit(500).all()
+        csv_content = build_datev_buchungsstapel(
+            invoices,
+            consultant_number=str(config.get("consultant_number") or config.get("berater_nr") or "00000"),
+            client_number=str(config.get("client_number") or config.get("mandant_nr") or "00000"),
+            account_revenue=str(config.get("account_revenue") or "8400"),
+            account_receivable=str(config.get("account_receivable") or "10000"),
+        )
+        filename = f"EXTF_Buchungsstapel_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}.csv"
+        # Persist last export in config for download via integrations UI
+        if self.integration:
+            cfg = dict(self.integration.config or {})
+            cfg["last_export_filename"] = filename
+            cfg["last_export_at"] = datetime.utcnow().isoformat()
+            cfg["last_export_preview"] = csv_content[:2000]
+            cfg["last_export_content"] = csv_content
+            self.integration.config = cfg
+            from app import db
+
+            db.session.commit()
+
+        return {
+            "success": True,
+            "synced": len(invoices),
+            "filename": filename,
+            "message": f"Generated DATEV export with {len(invoices)} invoices",
+            "content": csv_content,
+        }