feat: Azure Key Vault native Certificate-object storage mode (clean rebase of #118)

[rocogamer]

Replaces #118 — same feature, rebased clean on top of current main so the diff only contains Azure-KV-related changes. The original PR's git merge main had pulled in ~1200 lines of unrelated work from our fork; this PR contains only the 12 Azure-KV commits cherry-picked onto current upstream/main.

Also addresses all three non-blocker suggestions from @fabriziosalmi's full review on #118.

Summary

Adds a configurable storage_mode to the Azure Key Vault storage backend so it can persist certificates as native Certificate objects (PKCS12, issuer_name="Unknown") in addition to — or instead of — the existing per-PEM Secrets layout. Native Certificate objects are consumed directly by App Service, Application Gateway, Front Door, API Management and AKS Ingress without manual PFX export.

• storage_mode: secrets (default) — current per-PEM Secrets layout, fully backwards-compatible.
• storage_mode: certificate — single Certificate object, metadata stored as tags.
• storage_mode: both — write to both surfaces; reads compare updated_on and return the freshest (stale-read detection).

A new admin endpoint POST /api/storage/azure-keyvault/backfill-certificates imports Certificate objects for domains already stored as Secrets; restricted to storage_mode='both' so the legacy Secrets are still listed during the walk. Accepts ?limit=N for paginating large vaults.

Changes since #118 review

Suggestions addressed

1. Documented Secrets/Get private-key exposure (docs/architecture.md): pulled the by-design Azure behaviour (importing a Certificate object auto-creates a Secret containing the full PFX) out of the permissions-table cell into a prominent Security note under the storage-modes section, so operators sizing RBAC scopes for certificate/both modes actually see it.
2. ?limit=N query parameter on the backfill endpoint (modules/api/resources.py): caps how many domains are processed per call to avoid Azure Key Vault transaction throttling on vaults with hundreds of legacy certs. Response now includes a remaining count so operators can paginate by calling repeatedly; each call is idempotent because already-imported domains are reported as skipped.
3. cryptography>=41.0.0 floor in storage extras (requirements-azure-storage.txt, requirements-storage-all.txt): the Certificate-object code path uses cryptography.x509.load_pem_x509_certificates (plural) and the PKCS12 builder. Main requirements.txt already pins cryptography==46.0.7 via certbot, but the optional extras were silently relying on that transitive constraint. Making the floor explicit makes a layered install fail loudly with a pip resolver error instead of an opaque runtime ImportError.

Commits in this PR (12)

e0a6d26 deps(azure-storage): pin cryptography>=41.0.0 in storage extras
248fb9c feat(azure-kv): ?limit=N query parameter on backfill endpoint
634b79e docs: callout the Secrets/Get private-key exposure in Azure KV
9ede12d fix(azure-kv): stale read detection + CRC-aware list fix + release notes
09afe7d test: cover the symmetric and double-failure cases in store_certificate
2e0410f fix: apply surface-independence contract to delete_certificate
0165779 fix: don't let Secrets failure abort Certificate import in 'both' mode
bae2126 refactor: make _sanitize_secret_name a staticmethod
d60b973 fix: hide Azure Key Vault backfill button outside 'both' mode
994c174 fix: sanitize backfill per-domain error messages
550a932 docs: clarify metadata source in retrieve_certificate docstring
456a80d feat: store Azure Key Vault certificates as native Certificate objects

Diff stat (was 3413 / 202 across 30+ files on #118)

 RELEASE_NOTES.md                                 |   11 +
 docs/architecture.md                             |   44 +-
 modules/api/models.py                            |   12 +-
 modules/api/resources.py                         |  105 +-
 modules/core/settings.py                         |   17 +-
 modules/core/storage_backends.py                 |  709 +++++++++++++++--
 requirements-azure-storage.txt                   |    9 +
 requirements-storage-all.txt                     |    5 +
 static/js/settings.js                            |   86 ++
 templates/partials/settings_storage.html         |   32 +-
 tests/test_azure_keyvault_certificate_storage.py |  974 ++++++++++++++++++++++
 11 files changed, 1878 insertions(+), 126 deletions(-)

Test plan

• pytest tests/test_azure_keyvault_certificate_storage.py -v — 40/40 pass (2 skip cleanly when azure-keyvault-certificates is not installed).
• AST parse on every modified Python file.
• Backwards-compat: load a settings.json without storage_mode, confirm it migrates to secrets silently.
• Smoke test against a real Azure Key Vault (requires a Service Principal with Certificates + Secrets permissions):
  1. Configure SP in UI with storage_mode=both; press Test connection — both Secrets and Certificate API access verified.
  2. Issue a new cert from the dashboard — confirm in Azure Portal that Secrets cert-…-{cert,chain,fullchain,privkey}-pem-{crc} AND a cert-{domain}-{crc} Certificate object (with domain/dns_provider/staging tags) both appear.
  3. Renew the cert — confirm a new version of the same Certificate object (not a new object).
  4. Switch to storage_mode=certificate; issue a new cert — only the Certificate object is created, no Secrets.
  5. GET /api/certificates/{domain} — returns the four PEMs reconstructed from the PFX export.
  6. DELETE /api/certificates/{domain} — both surfaces removed in both mode.
  7. Switch back to both with legacy Secrets-only domains, press Backfill Certificate objects — every legacy domain gets an imported Certificate object; already-imported ones are reported as skipped. Pass ?limit=10 on a large vault — first 10 get processed, response reports remaining: N-10.
  8. Bind the Certificate object from an Azure App Service / Application Gateway with managed identity — TLS works without manual PFX export.

Notes for reviewers

• Default storage_mode='secrets' means existing installs see no behaviour change after upgrade.
• The azure-keyvault-certificates SDK is in the optional storage extras (requirements-azure-storage.txt); CI does not install it by default, so the two SDK-touching tests skip cleanly under pytest.importorskip.

🤖 Generated with Claude Code

[fabriziosalmi]

Heads-up: v2.4.12 (PR #147) landed a few minutes ago and touched modules/api/resources.py, modules/core/settings.py, and static/js/settings.js, which puts this PR in a CONFLICTING state.

Could you rebase onto current main when you have a moment? The v2.4.12 changes that intersect with what this PR adds:

• PUBLIC_SETTINGS_WRITABLE_KEYS in modules/core/settings.py — if your new storage_mode setting lives at the top level (alongside certificate_storage), it needs to be added there so POST /api/settings accepts it. If it's nested under certificate_storage (which from a quick read of the diff looks like the case), the existing entry in the whitelist covers it and no change is needed.
• modules/api/resources.py got a new _check_domain_scope helper and a Settings.post handler reshape. Your /api/storage/azure-keyvault/backfill-certificates endpoint shouldn't intersect (it's admin-only, not per-domain), but please double-check the merge of the resource registration.
• modules/api/models.py and static/js/settings.js were both touched lightly — likely trivial conflicts.

Everything else (Azure-KV-specific code, the 12 commits you cherry-picked, the suggestions-addressed pass on #118) is untouched by v2.4.12, so the rebase shouldn't surface anything substantive.

Thanks again for the patient cleanup work going from #118 to this — it's a substantially easier review with the unrelated 1200 lines stripped out.

[rocogamer]

Hi @fabriziosalmi,

Merge conflicts resolved — I merged the latest upstream/main (now at v2.4.15) into clean/keyvault-certificate-objects and updated the PR branch.

All 40 Azure Key Vault certificate-object tests pass, plus the upstream sprint security audit suites (Sprint 1 + 1.5). Zero test regressions.

Thanks again for your patience and for the super clear pointers in your review comment — they made the conflict resolution straightforward. No trouble at all, and really appreciate you reviewing this!

[fabriziosalmi]

Heads-up on the rebase surface: since the earlier rebase ask, v2.4.13 through v2.4.17 have landed on main and touched modules/api/resources.py, modules/core/factory.py, modules/web/settings_routes.py, modules/web/misc_routes.py, and modules/web/auth_routes.py. Most of it is additive (new endpoints, audit-log wiring, the scope-check helper, the diagnostic-snapshot resource) and shouldn't structurally conflict with the Azure-KV storage-backend code path in this PR, but the resource registration in setup_api() may need a re-stitch.

No urgency — the PR stays open until you have time. When you do rebase, the Sprint 1.5 / 1.7 changes most likely to matter for this one are:

• modules/api/resources.py got _check_domain_scope helper + DiagnosticsSnapshot resource + extended DownloadCertificate (privkey role split). None of these intersect with the storage-backend code.
• modules/core/factory.py added ns_diagnostics to the namespace list. If your PR added an admin endpoint, register it in the namespace list following the same pattern.

Thanks again for the patient cleanup work from #118 → here.

[rocogamer]

Rebased cleanly on top of upstream/main (v2.4.17).

All four sprint patches integrated with zero structural conflicts:

• v2.4.13 – audit follow-up (auth refactor, download role split, audit wiring)
• v2.4.15 – audit polish (ReDoc self-host, per-username rate limit, viewer GET on settings)
• v2.4.16 – + one-click bug-report UI
• v2.4.17 – bare-list deploy history + EXTRA_REQUIREMENTS build-arg + certbot-dns-azure baked in

What was manually stitched:

• – accepted upstream's new class, kept our resource + probe in .
• – adopted upstream's namespace + wiring.
• – bare-list response shape accepted cleanly.
• – combined upstream notes with our Unreleased section.

Test results on the merge commit (7c9a41c):

• ============================= test session starts ==============================
  platform linux -- Python 3.12.3, pytest-9.0.3, pluggy-1.6.0 -- /usr/bin/python3
  cachedir: .pytest_cache
  rootdir: /home/imartinezgr/Proyectos/certmate
  configfile: pytest.ini
  plugins: requests-mock-1.12.1
  collecting ... collected 40 items

tests/test_azure_keyvault_certificate_storage.py::TestBuildPfx::test_round_trip PASSED [ 2%]
tests/test_azure_keyvault_certificate_storage.py::TestBuildPfx::test_missing_cert_pem_raises PASSED [ 5%]
tests/test_azure_keyvault_certificate_storage.py::TestStorageModeValidation::test_default_is_secrets PASSED [ 7%]
tests/test_azure_keyvault_certificate_storage.py::TestStorageModeValidation::test_explicit_certificate_mode PASSED [ 10%]
tests/test_azure_keyvault_certificate_storage.py::TestStorageModeValidation::test_both_mode PASSED [ 12%]
tests/test_azure_keyvault_certificate_storage.py::TestStorageModeValidation::test_invalid_mode_raises PASSED [ 15%]
tests/test_azure_keyvault_certificate_storage.py::TestStorageModeValidation::test_uppercase_mode_normalised PASSED [ 17%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_secrets_mode_does_not_call_certificate_api PASSED [ 20%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_certificate_mode_only_imports_certificate PASSED [ 22%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_both_mode_invokes_both_paths PASSED [ 25%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_certificate_mode_failure_returns_false PASSED [ 27%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_both_mode_secrets_exception_does_not_skip_certificate_import PASSED [ 30%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_both_mode_certificate_exception_does_not_skip_secrets_write PASSED [ 32%]
tests/test_azure_keyvault_certificate_storage.py::TestStoreCertificateRouting::test_both_mode_double_store_failure_returns_false PASSED [ 35%]
tests/test_azure_keyvault_certificate_storage.py::TestImportCertificateSdkCall::test_import_certificate_uses_pkcs12_and_unknown_issuer PASSED [ 37%]
tests/test_azure_keyvault_certificate_storage.py::TestImportCertificateSdkCall::test_import_returns_false_when_inputs_missing PASSED [ 40%]
tests/test_azure_keyvault_certificate_storage.py::TestTagTruncation::test_oversize_san_domains_truncated_with_marker PASSED [ 42%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveCertificateMode::test_export_certificate_reconstructs_pem_files PASSED [ 45%]
tests/test_azure_keyvault_certificate_storage.py::TestListCertificatesRouting::test_certificate_only_lists_from_certificate_api PASSED [ 47%]
tests/test_azure_keyvault_certificate_storage.py::TestListCertificatesRouting::test_both_mode_unions_and_dedupes PASSED [ 50%]
tests/test_azure_keyvault_certificate_storage.py::TestDeleteCertificateRouting::test_both_mode_deletes_in_both_apis PASSED [ 52%]
tests/test_azure_keyvault_certificate_storage.py::TestDeleteCertificateRouting::test_certificate_only_skips_secret_deletes PASSED [ 55%]
tests/test_azure_keyvault_certificate_storage.py::TestDeleteCertificateRouting::test_both_mode_certificate_exception_does_not_skip_secrets_delete PASSED [ 57%]
tests/test_azure_keyvault_certificate_storage.py::TestDeleteCertificateRouting::test_both_mode_double_failure_returns_false PASSED [ 60%]
tests/test_azure_keyvault_certificate_storage.py::TestDeleteCertificateRouting::test_both_mode_partial_secret_failure_returns_false PASSED [ 62%]
tests/test_azure_keyvault_certificate_storage.py::TestCertificateExists::test_certificate_only_uses_importer PASSED [ 65%]
tests/test_azure_keyvault_certificate_storage.py::TestTagsMetadataRoundTrip::test_round_trip_preserves_known_keys PASSED [ 67%]
tests/test_azure_keyvault_certificate_storage.py::TestTagsMetadataRoundTrip::test_staging_none_is_omitted_not_falsified PASSED [ 70%]
tests/test_azure_keyvault_certificate_storage.py::TestTagsMetadataRoundTrip::test_truncated_san_marker_is_stripped_on_rehydrate PASSED [ 72%]
tests/test_azure_keyvault_certificate_storage.py::TestTagsMetadataRoundTrip::test_external_tags_are_filtered_out_on_rehydrate PASSED [ 75%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveBothMode::test_secrets_present_certificate_api_not_touched PASSED [ 77%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveBothMode::test_secrets_present_metadata_missing_falls_back_to_tags PASSED [ 80%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveBothMode::test_secrets_absent_falls_through_to_certificate_api PASSED [ 82%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveBothMode::test_both_mode_certificate_newer_than_secrets_returns_cert PASSED [ 85%]
tests/test_azure_keyvault_certificate_storage.py::TestRetrieveBothMode::test_both_mode_secrets_newer_than_certificate_returns_secrets PASSED [ 87%]
tests/test_azure_keyvault_certificate_storage.py::TestBackfillFlow::test_certificate_only_mode_lists_only_existing_certs PASSED [ 90%]
tests/test_azure_keyvault_certificate_storage.py::TestBackfillFlow::test_both_mode_imports_only_missing PASSED [ 92%]
tests/test_azure_keyvault_certificate_storage.py::TestVerifyCertificateApiAccess::test_calls_list_properties_without_extra_kwargs PASSED [ 95%]
tests/test_azure_keyvault_certificate_storage.py::TestVerifyCertificateApiAccess::test_propagates_sdk_failure_to_caller PASSED [ 97%]
tests/test_azure_keyvault_certificate_storage.py::TestSettingsMigration::test_storage_mode_backfilled_when_missing PASSED [100%]

============================== 40 passed in 0.56s ============================== → 40 passed

• ============================= test session starts ==============================
  platform linux -- Python 3.12.3, pytest-9.0.3, pluggy-1.6.0 -- /usr/bin/python3
  cachedir: .pytest_cache
  rootdir: /home/imartinezgr/Proyectos/certmate
  configfile: pytest.ini
  plugins: requests-mock-1.12.1
  collecting ... collected 11 items

tests/test_diagnostics_snapshot.py::TestSnapshotShape::test_basic_shape PASSED [ 9%]
tests/test_diagnostics_snapshot.py::TestSnapshotShape::test_certificate_count_matches_manager PASSED [ 18%]
tests/test_diagnostics_snapshot.py::TestSnapshotShape::test_settings_scalars_propagated PASSED [ 27%]
tests/test_diagnostics_snapshot.py::TestSnapshotSanitization::test_response_has_no_secret_values PASSED [ 36%]
tests/test_diagnostics_snapshot.py::TestSnapshotSanitization::test_audit_entries_stripped_of_identifiers PASSED [ 45%]
tests/test_diagnostics_snapshot.py::TestSnapshotSanitization::test_audit_entries_capped_at_five PASSED [ 54%]
tests/test_diagnostics_snapshot.py::TestSnapshotSanitization::test_full_settings_dict_not_exposed PASSED [ 63%]
tests/test_diagnostics_snapshot.py::TestSnapshotPartialFailure::test_certificate_count_failure PASSED [ 72%]
tests/test_diagnostics_snapshot.py::TestSnapshotPartialFailure::test_audit_failure PASSED [ 81%]
tests/test_diagnostics_snapshot.py::TestSnapshotPartialFailure::test_disk_usage_failure PASSED [ 90%]
tests/test_diagnostics_snapshot.py::TestSnapshotPartialFailure::test_no_audit_logger_wired PASSED [100%]

============================== 11 passed in 0.68s ============================== → 11 passed

• ============================= test session starts ==============================
  platform linux -- Python 3.12.3, pytest-9.0.3, pluggy-1.6.0 -- /usr/bin/python3
  cachedir: .pytest_cache
  rootdir: /home/imartinezgr/Proyectos/certmate
  configfile: pytest.ini
  plugins: requests-mock-1.12.1
  collecting ... collected 4 items

tests/test_issue152_deploy_history.py::test_deploy_history_returns_bare_array PASSED [ 25%]
tests/test_issue152_deploy_history.py::test_deploy_history_empty_returns_empty_array_not_envelope PASSED [ 50%]
tests/test_issue152_deploy_history.py::test_deploy_history_forwards_limit_and_domain_filters PASSED [ 75%]
tests/test_issue152_deploy_history.py::test_deploy_history_503_when_deploy_manager_missing PASSED [100%]

============================== 4 passed in 0.16s =============================== → 4 passed

Should be good to go whenever you have review bandwidth.

[anthonysomerset]

Subscribing as an interested party in seeing this functionality

question from me - what is the benefit of using the storage_mode: both ? i understand that secrets is cheaper but how does that play out in this context?

my personal use case is we only use certificate objects for certs, so am curious to understand if there will be any benefit to using both mode

[fabriziosalmi]

Queued for review + merge tonight, in the same window as the broader v3 UI fix pass we have batched. Plan is to run the smoke test plan from the PR body against a real Azure Key Vault (Service Principal with Certificates + Secrets permissions, all three storage modes exercised end-to-end including the ?limit=N backfill path) before pressing merge — won't ship a storage-backend change without that signal, even though the SDK-touching tests skip cleanly in CI.

Thanks for the clean rebase off the original #118 and for addressing the three non-blocker notes from that review; the diff is exactly the surface area I want to review now.