Feat/encrypted connector secrets

.env.example

@@ -58,6 +58,10 @@ AZURE_CLIENT_ID=
 AZURE_CLIENT_SECRET=
 AZURE_TENANT_ID=
 
+# Field encryption (connector secrets)
+# Generate with: python -c "import secrets; print(secrets.token_urlsafe(32))"
+FIELD_ENCRYPTION_KEY=
+
 # Confluence (optional)
 CONFLUENCE_URL=
 CONFLUENCE_USERNAME=

CHANGELOG.md

@@ -8,6 +8,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- Per-tenant encrypted connector secrets: users can now enter API keys/secrets directly in the connector creation UI, stored encrypted in the DB using Fernet with HKDF-derived per-tenant keys
+- `FIELD_ENCRYPTION_KEY` setting for dedicated encryption key (falls back to `SECRET_KEY`)
+- `connectors/crypto.py` module with `encrypt_secret()` / `decrypt_secret()` utilities
+- `encrypted_secret` field on `ConnectorConfig` model with `set_secret()` / `get_secret()` methods
+- Password input field in connector creation form (both standalone page and modal)
+- `migrate_connector_secrets` management command to migrate existing env-var-based credentials into encrypted secrets
+- `cryptography>=42.0` dependency
 - Apache 2.0 LICENSE file
 - `CONTRIBUTING.md`, `CHANGELOG.md`, `SECURITY.md`
 - `SECRET_KEY` startup validation (refuses to start with placeholder key in production)

README.md

@@ -196,9 +196,13 @@ AZURE_MISTRAL_DEPLOYMENT_NAME=...
 # Rate limits
 LLM_REQUESTS_PER_MINUTE=60
 EMBEDDING_BATCH_SIZE=100
+
+# Field encryption (connector secrets)
+# Generate with: python -c "import secrets; print(secrets.token_urlsafe(32))"
+FIELD_ENCRYPTION_KEY=
 ```
 
-> **Security**: In production (`DEBUG=False`), Django will refuse to start if `SECRET_KEY` is set to a placeholder value.
+> **Security**: In production (`DEBUG=False`), Django will refuse to start if `SECRET_KEY` is set to a placeholder value. A dedicated `FIELD_ENCRYPTION_KEY` is recommended for encrypting connector secrets (falls back to `SECRET_KEY` if not set).
 
 For analysis tuning via `config.yaml`, see [docs/configuration.md](docs/configuration.md).
 
@@ -255,7 +259,7 @@ The test suite covers all major modules: analysis views, audit views, chat, chun
 
 ## Dependencies
 
-**Core:** Django 5.1, Celery 5.4, sqlite-vec 0.1.6, OpenAI SDK, tiktoken, django-allauth, whitenoise, gunicorn
+**Core:** Django 5.1, Celery 5.4, sqlite-vec 0.1.6, OpenAI SDK, tiktoken, django-allauth, cryptography (Fernet encryption), whitenoise, gunicorn
 
 **ML/Analysis:** scikit-learn, HDBSCAN, datasketch (MinHash), numpy, NLTK, langid, rank-bm25
 

SECURITY.md

@@ -31,8 +31,40 @@ Include the following in your report:
 
 - Never use the default `SECRET_KEY` in production
 - Generate a proper key: `python -c "from django.core.management.utils import get_random_secret_key; print(get_random_secret_key())"`
+- Set a dedicated `FIELD_ENCRYPTION_KEY` for connector secret encryption: `python -c "import secrets; print(secrets.token_urlsafe(32))"`
 - Keep `.env` files out of version control (already in `.gitignore`)
 - Use HTTPS in production (HSTS is enabled when `DEBUG=False`)
 - Rotate API keys regularly
 - Change default credentials immediately after setup
 - Use a production-grade database (PostgreSQL) for concurrent deployments
+
+## Connector Secret Encryption
+
+Connector credentials (API keys, client secrets) can be stored encrypted in the database using per-tenant Fernet encryption:
+
+- **Key derivation:** HKDF (SHA-256) derives a unique Fernet key per tenant from the master `FIELD_ENCRYPTION_KEY` (or `SECRET_KEY` as fallback)
+- **Info prefix:** `docuscore-connector-secret-v1:` followed by the tenant UUID, enabling future key rotation
+- **Tenant isolation:** Each tenant's secrets are encrypted with a different derived key — a secret encrypted for tenant A cannot be decrypted by tenant B
+- **Graceful fallback:** Connectors with only a `credential_ref` (env var name) continue working without encryption
+- **Failure mode:** Decryption errors return an empty string and log a warning — the application does not crash
+- **Migration:** Use `python manage.py migrate_connector_secrets --apply` to encrypt existing env-var-based credentials (see below)
+
+### Migrating Existing Credentials
+
+If you already have connectors configured with `credential_ref` pointing to environment variables, use the management command to encrypt them:
+
+```bash
+# 1. Set FIELD_ENCRYPTION_KEY in .env (or rely on SECRET_KEY fallback)
+# 2. Ensure the env vars referenced by credential_ref are set
+
+# Dry run — preview what would be migrated
+python manage.py migrate_connector_secrets
+
+# Apply — encrypt env var values into encrypted_secret
+python manage.py migrate_connector_secrets --apply
+
+# Optional: clear credential_ref after verifying encryption works
+python manage.py migrate_connector_secrets --apply --clear-ref
+```
+
+The command is safe to run multiple times — it skips connectors that already have an `encrypted_secret` value.

connectors/confluence.py

@@ -23,7 +23,7 @@ def __init__(self, config: dict, credential: str = ""):
         self._url = config.get("url", os.environ.get("CONFLUENCE_URL", ""))
         self._space_key = config.get("space_key", "")
         self._username = config.get("username", os.environ.get("CONFLUENCE_USERNAME", ""))
-        self._api_token = os.environ.get(credential, os.environ.get("CONFLUENCE_API_TOKEN", ""))
+        self._api_token = credential or os.environ.get("CONFLUENCE_API_TOKEN", "")
         self._client = None
 
     def _get_client(self):

connectors/crypto.py

@@ -0,0 +1,58 @@
+"""
+Per-tenant encryption for connector secrets.
+
+Uses HKDF key derivation to produce a unique Fernet key per tenant from a
+single master key, so each tenant's secrets are cryptographically isolated.
+"""
+
+import base64
+import logging
+
+from cryptography.fernet import Fernet, InvalidToken
+from cryptography.hazmat.primitives.hashes import SHA256
+from cryptography.hazmat.primitives.kdf.hkdf import HKDF
+from django.conf import settings
+
+logger = logging.getLogger(__name__)
+
+_HKDF_INFO_PREFIX = b"docuscore-connector-secret-v1:"
+
+
+def _get_master_key() -> bytes:
+    """Return the master key bytes used for HKDF derivation."""
+    key = getattr(settings, "FIELD_ENCRYPTION_KEY", "") or settings.SECRET_KEY
+    return key.encode("utf-8") if isinstance(key, str) else key
+
+
+def _derive_fernet_key(tenant_id: str) -> bytes:
+    """Derive a tenant-specific Fernet key via HKDF."""
+    hkdf = HKDF(
+        algorithm=SHA256(),
+        length=32,
+        salt=None,
+        info=_HKDF_INFO_PREFIX + tenant_id.encode("utf-8"),
+    )
+    derived = hkdf.derive(_get_master_key())
+    return base64.urlsafe_b64encode(derived)
+
+
+def encrypt_secret(plain_text: str, tenant_id: str) -> str:
+    """Encrypt a secret string for a specific tenant. Returns base64 ciphertext."""
+    if not plain_text:
+        return ""
+    key = _derive_fernet_key(tenant_id)
+    f = Fernet(key)
+    return f.encrypt(plain_text.encode("utf-8")).decode("utf-8")
+
+
+def decrypt_secret(encrypted_text: str, tenant_id: str) -> str:
+    """Decrypt a tenant-specific secret. Returns empty string on failure."""
+    if not encrypted_text:
+        return ""
+    try:
+        key = _derive_fernet_key(tenant_id)
+        f = Fernet(key)
+        return f.decrypt(encrypted_text.encode("utf-8")).decode("utf-8")
+    except (InvalidToken, Exception):
+        logger.warning("Failed to decrypt secret for tenant %s", tenant_id)
+        return ""

connectors/management/commands/migrate_connector_secrets.py

@@ -0,0 +1,124 @@
+"""
+Management command to migrate existing connector credentials from environment
+variables into per-tenant encrypted secrets.
+
+For production deployments that were using credential_ref (env var names),
+this reads each env var value and encrypts it into the encrypted_secret field.
+
+Usage:
+    # Dry run (default) — shows what would be migrated without changing anything
+    python manage.py migrate_connector_secrets
+
+    # Actually perform the migration
+    python manage.py migrate_connector_secrets --apply
+
+    # Also clear credential_ref after migration (env vars no longer needed)
+    python manage.py migrate_connector_secrets --apply --clear-ref
+"""
+
+import logging
+import os
+
+from django.core.management.base import BaseCommand
+
+from connectors.models import ConnectorConfig
+
+logger = logging.getLogger(__name__)
+
+
+class Command(BaseCommand):
+    help = (
+        "Migrate connector credentials from environment variables "
+        "into per-tenant encrypted secrets."
+    )
+
+    def add_arguments(self, parser):
+        parser.add_argument(
+            "--apply",
+            action="store_true",
+            help="Actually perform the migration. Without this flag, runs in dry-run mode.",
+        )
+        parser.add_argument(
+            "--clear-ref",
+            action="store_true",
+            help="Clear credential_ref after successful encryption (env var no longer needed).",
+        )
+
+    def handle(self, *args, **options):
+        apply = options["apply"]
+        clear_ref = options["clear_ref"]
+
+        if not apply:
+            self.stdout.write(
+                self.style.WARNING("DRY RUN — pass --apply to perform the migration.\n")
+            )
+
+        connectors = ConnectorConfig.objects.filter(
+            credential_ref__gt="",
+        ).exclude(
+            encrypted_secret__gt="",
+        )
+
+        total = connectors.count()
+        if total == 0:
+            self.stdout.write(
+                self.style.SUCCESS(
+                    "Nothing to migrate: no connectors with credential_ref and empty encrypted_secret."
+                )
+            )
+            return
+
+        self.stdout.write(f"Found {total} connector(s) to migrate.\n")
+
+        migrated = 0
+        skipped = 0
+        errors = 0
+
+        for connector in connectors:
+            ref = connector.credential_ref
+            env_value = os.environ.get(ref, "")
+
+            tenant_name = connector.tenant.name if connector.tenant else "?"
+            label = f"  [{connector.name}] tenant={tenant_name} ref={ref}"
+
+            if not env_value:
+                self.stdout.write(
+                    self.style.WARNING(f"{label} — SKIPPED (env var '{ref}' is empty or unset)")
+                )
+                skipped += 1
+                continue
+
+            if apply:
+                try:
+                    connector.set_secret(env_value)
+                    if clear_ref:
+                        connector.credential_ref = ""
+                    connector.save()
+                    self.stdout.write(self.style.SUCCESS(f"{label} — MIGRATED"))
+                    migrated += 1
+                except Exception as e:
+                    self.stdout.write(self.style.ERROR(f"{label} — ERROR: {e}"))
+                    errors += 1
+            else:
+                masked = env_value[:3] + "***" if len(env_value) > 3 else "***"
+                self.stdout.write(f"{label} — WOULD MIGRATE (value: {masked})")
+                migrated += 1
+
+        self.stdout.write("")
+        if apply:
+            self.stdout.write(
+                self.style.SUCCESS(
+                    f"Done. Migrated: {migrated}, Skipped: {skipped}, Errors: {errors}"
+                )
+            )
+            if migrated > 0 and not clear_ref:
+                self.stdout.write(
+                    self.style.NOTICE(
+                        "Tip: re-run with --clear-ref to remove credential_ref values "
+                        "once you've verified the encrypted secrets work correctly."
+                    )
+                )
+        else:
+            self.stdout.write(
+                f"Would migrate: {migrated}, Would skip: {skipped}\nPass --apply to execute."
+            )

connectors/migrations/0003_add_encrypted_secret.py

@@ -0,0 +1,21 @@
+# Generated by Django 5.1.15 on 2026-03-23 08:21
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("connectors", "0002_add_project"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="connectorconfig",
+            name="encrypted_secret",
+            field=models.TextField(
+                blank=True,
+                default="",
+                help_text="Fernet-encrypted secret (set via set_secret, read via get_secret)",
+            ),
+        ),
+    ]

connectors/models.py

@@ -5,12 +5,16 @@
 and stores credentials securely.
 """
 
+import logging
+import os
 import uuid
 
 from django.db import models
 from django.utils.translation import gettext_lazy as _
 from tenants.models import ProjectScopedModel
 
+logger = logging.getLogger(__name__)
+
 
 class ConnectorConfig(ProjectScopedModel):
     """Configuration for a document source connector."""
@@ -36,6 +40,12 @@ class ConnectorType(models.TextChoices):
         blank=True,
         help_text="Reference to credential store (env var name or secret manager path)",
     )
+    # Encrypted secret — per-tenant Fernet-encrypted credential value
+    encrypted_secret = models.TextField(
+        blank=True,
+        default="",
+        help_text="Fernet-encrypted secret (set via set_secret, read via get_secret)",
+    )
 
     schedule_cron = models.CharField(
         max_length=100,
@@ -54,3 +64,24 @@ class Meta:
 
     def __str__(self):
         return f"{self.name} ({self.connector_type})"
+
+    def set_secret(self, plain_text: str) -> None:
+        """Encrypt and store a secret value for this connector's tenant."""
+        from connectors.crypto import encrypt_secret
+
+        self.encrypted_secret = encrypt_secret(plain_text, str(self.tenant_id))
+
+    def get_secret(self) -> str:
+        """Return the decrypted secret, falling back to credential_ref env var lookup."""
+        if self.encrypted_secret:
+            from connectors.crypto import decrypt_secret
+
+            decrypted = decrypt_secret(self.encrypted_secret, str(self.tenant_id))
+            if decrypted:
+                return decrypted
+
+        # Fallback: treat credential_ref as an env var name
+        if self.credential_ref:
+            return os.environ.get(self.credential_ref, "")
+
+        return ""

connectors/sharepoint.py

@@ -26,7 +26,7 @@ def __init__(self, config: dict, credential: str = ""):
         self._folder_path = config.get("folder_path", "/")
         self._client_id = config.get("client_id", os.environ.get("SHAREPOINT_CLIENT_ID", ""))
         self._tenant_id = config.get("tenant_id", os.environ.get("SHAREPOINT_TENANT_ID", ""))
-        self._client_secret = os.environ.get(credential, "") if credential else ""
+        self._client_secret = credential or ""
         self._access_token: str | None = None
 
     def _authenticate(self):