feat: configurable cert key shape + APScheduler app context + EXTRA_REQUIREMENTS build-arg

[rocogamer]

Summary

Three independent quality-of-life items that were originally bundled into the contaminated #118 and have been rebased clean on top of current main here. They share no code paths and could be split into separate PRs if you prefer — flagged as such in each section. One bug found while testing the key-shape feature is fixed in passing.

• feat — Configurable certificate key type/size (RSA 2048/3072/4096 or ECDSA P-256/P-384), as a global default and as an optional per-cert override.
• fix — APScheduler background jobs now push the Flask app_context explicitly, so scheduler-driven renewals can use anything that touches current_app / Flask config.
• build — EXTRA_REQUIREMENTS build-arg lets a single Dockerfile layer in optional storage backends (Azure KV / AWS SM / Vault / Infisical) without forking the image; certbot-dns-azure==2.5.0 baked into the main requirements.txt so default builds ship a working Azure DNS plugin.

Diff stat (14 commits)

 Dockerfile                               |  26 +++-
 RELEASE_NOTES.md                         |   7 +
 docker-compose.yml                       |   9 ++
 docs/architecture.md                     |  19 ++-
 modules/api/models.py                    |  31 +++-
 modules/api/resources.py                 |  35 ++++-
 modules/core/ca_manager.py               |  22 ++-
 modules/core/certificates.py             |  49 ++++++-
 modules/core/factory.py                  |  60 ++++++--
 modules/core/settings.py                 |  54 ++++++-
 modules/core/utils.py                    |  61 ++++++++
 modules/web/settings_routes.py           |  14 ++
 requirements.txt                         |   4 +
 static/js/dashboard.js                   |  25 ++++
 static/js/settings.js                    |  52 ++++++-
 templates/index.html                     |  41 ++++++
 templates/partials/settings_general.html |  41 +++++-
 tests/test_key_options.py                | 241 +++++++++++++++++++++++++++++++
 tests/test_settings_masking_allowlist.py | 135 +++++++++++++++++
 19 files changed, 895 insertions(+), 31 deletions(-)

---

1. Configurable certificate key type/size

Motivation

Every cert issued by CertMate was hardcoded to RSA-2048, because CAManager.build_certbot_command never passed --key-type or --rsa-key-size to certbot. Two real-world asks made this awkward:

• Compliance: some operators are required to use RSA-3072 / RSA-4096.
• Modern stacks: ECDSA produces smaller certs and faster TLS handshakes; many CDNs and load balancers prefer it.

Both groups had to patch the code or maintain a fork.

What it does

• Global default under Settings → General → "Default Certificate Key": default_key_type (rsa / ecdsa), default_key_size (2048 / 3072 / 4096), default_elliptic_curve (secp256r1 / secp384r1). The size and curve selectors are mutually exclusive — picking ECDSA hides the size picker and vice versa.
• Per-cert override under the create-cert form's "Advanced Options". Leaving it on "Use global default" sends no key fields, and the backend inherits the configured default.
• Renewals preserve the original shape automatically, because certbot persists the --key-type / --rsa-key-size / --elliptic-curve flags into its own renewal/<domain>.conf at create time. No new bookkeeping in CertMate.
• Backwards-compatible default: settings without these fields migrate to rsa / 2048 / secp256r1 at load time, so existing installs see no behaviour change after upgrade.

Validation

A validate_key_options(key_type, key_size, elliptic_curve) helper (modules/core/utils.py) rejects contradictory shapes up-front:

• key_type='rsa' with an elliptic_curve override → 400.
• key_type='ecdsa' with a key_size override → 400.
• key_type='rsa' with an unsupported size (e.g. 1024) → 400.
• key_type='ecdsa' with an unsupported curve (e.g. secp521r1) → 400.
• A size or curve specified without a matching key_type → 400.

Runs both on every settings save and on every cert creation, so a bad shape fails fast with a useful error instead of bubbling up as a confusing certbot exit code 5 minutes later.

Bug fix included — Settings dropdown empty on reload

While testing the global selector, the <select> for "Key Type" rendered empty on every page reload, even after explicitly saving ECDSA. Root cause:

modules/web/settings_routes.py masks any field whose name matches the regex (token|secret|password|key|credential) to '********' before returning settings. The key substring matched default_key_type (and would have matched default_key_size if its value were a string), so GET /api/web/settings returned default_key_type: '********'. settings.js then ran selectEl.value = '********', which matches no <option>, so the DOM ended up with selectedIndex = -1. Worse: a save-without-edit then tripped validate_key_options and failed silently.

Fix: an explicit _NON_SECRET_KEYS allowlist of the three key-shape fields inside _mask_dict. The regex stays unchanged — it's the same one used app-wide, and tightening it would require auditing every credential-bearing field. The allowlist documents intent: these specific fields look like secrets to a substring match but aren't.

Four new regression tests in tests/test_settings_masking_allowlist.py pin the contract: allowlisted fields come through literally, genuine secrets stay masked, nested dns_providers.<provider>.accounts.<account>.api_token are still recursed into, and the existing empty-string passthrough is preserved.

Tests

tests/test_key_options.py — 20 cases:

• Accept paths: RSA 2048/3072/4096; ECDSA secp256r1/secp384r1; legacy None (inherit global).
• Reject paths: each contradictory or unsupported shape listed under "Validation" above.
• Command builder: --key-type rsa --rsa-key-size 4096 and --key-type ecdsa --elliptic-curve secp384r1 emitted on the certbot command line. No partial flags when only one half of a shape is provided.
• Settings migration: legacy settings.json (without the three fields) is back-filled with rsa / 2048 / secp256r1 at load time.
• Save-time rejection: inconsistent global defaults fail the settings save before any state is mutated.

Files

• modules/core/utils.py — validate_key_options helper.
• modules/core/settings.py — defaults + migration + save-time validation.
• modules/core/ca_manager.py — --key-type / --rsa-key-size / --elliptic-curve on the certbot command.
• modules/core/certificates.py — plumbs the key shape through create_certificate.
• modules/api/models.py + modules/api/resources.py — exposes the key shape on cert creation and on global settings.
• templates/index.html — Advanced Options section on the create-cert form.
• templates/partials/settings_general.html — Default Certificate Key section on the settings page.
• static/js/dashboard.js — toggleCertKeyOptions + per-cert override on submit.
• static/js/settings.js — toggleDefaultKeyOptions + populate-on-load + form submit.
• modules/web/settings_routes.py — _NON_SECRET_KEYS allowlist.
• tests/test_key_options.py + tests/test_settings_masking_allowlist.py — coverage.

---

2. APScheduler app_context fix

Motivation

Background jobs scheduled by APScheduler in modules/core/factory.py ran outside the Flask request context. Anything that touched current_app (config reads, blueprint URL building, flask.g) blew up with:

RuntimeError: Working outside of application context.

Every renewal cycle, every cron-triggered cleanup, and every deferred deploy hook ran into this — except the ones that happened to be pure-function and never reached for current_app. Symptoms in the wild were intermittent renewal failures that didn't reproduce from the UI.

What it does

Wraps each scheduled job with app.app_context() before dispatch:

def _with_app_context(fn, app):
    @functools.wraps(fn)
    def wrapper(*args, **kwargs):
        with app.app_context():
            return fn(*args, **kwargs)
    return wrapper

Applied to every scheduler.add_job(...) call site in factory.py. The wrapped callable closes over the Flask app, so the scheduler thread can push the context without needing a request.

Tests

Covered indirectly by tests/test_secret_key_env.py (which imports the factory) and the existing per-cert renewal tests. Local smoke test: stop and restart the scheduler with RENEWAL_THRESHOLD_DAYS=999, confirm every scheduled renewal completes without the RuntimeError.

Files

• modules/core/factory.py.

---

3. Build/deploy tweaks

Motivation

Two operational papercuts:

1. Optional storage backends (Azure KV / AWS SM / Vault / Infisical) live in separate requirements files (requirements-azure-storage.txt, …) so the default image stays slim. But there was no Dockerfile path to install one of them — operators had to fork the image, edit RUN pip install, and maintain that diff.

2. Azure DNS is one of the four "Major cloud providers" CertMate already ships in the UI (Cloudflare / Route53 / DigitalOcean / Google). Picking it in a default build produced:

   Certificate creation failed: The certbot plugin 'dns-azure' is not installed.
   Install it with: pip install certbot-dns-azure
   

   …because certbot-dns-azure lived only in the optional requirements-azure.txt.

What it does

• EXTRA_REQUIREMENTS build-arg on the Dockerfile. Takes a space-separated list of requirements file paths and runs pip install -r <file> for each, after the main REQUIREMENTS_FILE install. Example:

  docker build \
    --build-arg REQUIREMENTS_FILE=requirements.txt \
    --build-arg EXTRA_REQUIREMENTS="requirements-azure-storage.txt requirements-aws-storage.txt" \
    -t certmate:azure+aws .

  Defaults to empty, so existing builds are unchanged.

• docker-compose.yml gets the build-arg wired through so docker compose build works the same way.

• certbot-dns-azure==2.5.0 is baked into the main requirements.txt. Pinned to 2.5.0 (not 2.6.x) — 2.6.0 jumped to certbot>=3.0, which would break the repo-wide certbot==2.10.0 pin.

Tests

CI was the verification path here: image build (linux/amd64 + linux/arm64) green on the default install and on a layered Azure-KV + AWS-SM install. No new unit tests because none of the change is in Python code paths.

Files

• Dockerfile, docker-compose.yml, requirements.txt.

---

Notes for reviewers

• Split if you'd rather: these three are independent and could be three smaller PRs. They're bundled here because they were already a single rebase of the work that contaminated feat: Azure Key Vault native Certificate-object storage mode #118, and splitting them would mean another round of cherry-picks. Happy to break them up if you prefer.
• Default behaviour is unchanged in all three cases: existing installs see no behaviour change after upgrade — key shape defaults to RSA-2048, EXTRA_REQUIREMENTS defaults to empty, and the app_context wrapper is invisible to anything that didn't already need a request context.
• The masking allowlist is the smallest possible fix for the dropdown bug. If you'd rather tighten the regex globally (so any future default_* field with "key" in the name doesn't trip the same trap), say the word — but I'd want to do that as its own PR since it has app-wide implications.

Test plan

• `pytest tests/test_key_options.py -v` — 20/20 pass.
• `pytest tests/test_settings_masking_allowlist.py -v` — 4/4 pass.
• `pytest tests/ -m unit -q --ignore=tests/test_secret_key_env.py` — 119 passed, 1 skipped (pre-existing skip unrelated to this PR).
• Image build + smoke: `docker build --build-arg EXTRA_REQUIREMENTS=requirements-azure-storage.txt -t certmate:test .` succeeds, container boots, Settings page → Default Certificate Key shows persisted ECDSA / secp384r1 after a save+reload.
• Issue a cert via the dashboard with an explicit ECDSA / P-384 override; confirm `openssl x509 -in fullchain.pem -text` shows `Public Key Algorithm: id-ecPublicKey` / `NIST CURVE: P-384`.
• Issue a cert from the same UI with "Use global default" + global = ECDSA; confirm the cert inherits the global shape.
• Renew the same cert; confirm certbot reuses the curve (no `--key-type` regression to RSA).
• Restart the scheduler with `RENEWAL_THRESHOLD_DAYS=999` so every cert is "due"; confirm no `RuntimeError: Working outside of application context` in the logs.

🤖 Generated with Claude Code

[fabriziosalmi]

Sorry about the timing — v2.4.12 (PR #147) landed a few minutes before this one could be reviewed, and it touches enough of the same surface (modules/api/resources.py, modules/core/factory.py, modules/core/settings.py, static/js/dashboard.js, static/js/settings.js) that GitHub now reports this PR as CONFLICTING.

Two requests:

1. Rebase onto current main. v2.4.12 added: a strict whitelist on POST /api/settings (see PUBLIC_SETTINGS_WRITABLE_KEYS in modules/core/settings.py), allowed_domains on scoped API keys (modules/core/auth.py), enforcement helpers in resources.py + cert_routes.py, and an extended audit logger. The conflicts should be mechanical but a couple of merge decisions are worth flagging when you hit them:

• Your new top-level settings keys (whatever drives the key-type default) need to be added to PUBLIC_SETTINGS_WRITABLE_KEYS in modules/core/settings.py so POST /api/settings accepts them. If they're nested under an already-allowed dict (e.g. inside certificate_storage or under a new sibling), the whitelist already passes them through.
• If you wire the per-cert override into create/renew, please run it through _check_domain_scope(domain, 'create') in modules/api/resources.py so a scoped API key with allowed_domains can't side-step its scope via the new field.

2. Split into 3 PRs, as you offered. The three items don't share code paths and individually they're each small enough to review in one sitting:

• feat: configurable cert key type/size (the biggest of the three; carries 241 lines of new tests)
• fix: APScheduler app_context for renewal jobs (bug fix; should ship on its own so it gets a release note line)
• build: EXTRA_REQUIREMENTS build-arg + certbot-dns-azure baked in (Dockerfile only)

Reviewing them independently means each one moves at its own pace — the APScheduler fix in particular probably wants to ship faster than the others.

Up to you. If three separate PRs is too much overhead, I'll review this as one after the rebase. The work itself looks good from a first read.

[rocogamer]

Thanks for the thorough review and the actionable feedback, really appreciated.

The PR is now split into three independent PRs as suggested, all rebased on current main (v2.4.15):

PR	Description
#154	fix: APScheduler app_context for background renewal jobs — the bug fix, should ship fast
#155	build: EXTRA_REQUIREMENTS build-arg + certbot-dns-azure baked in — Docker/build only, zero code
#156	feat: configurable cert key type/size (RSA + ECDSA) — the big one

The two integration items from your review are addressed in #156:

1. default_key_type, default_key_size, default_elliptic_curve are added to PUBLIC_SETTINGS_WRITABLE_KEYS so POST /api/settings from the UI saves them.
2. Domain scope: the existing _check_domain_scope(domain, 'create') on the create-cert endpoint already gates the entire request — the per-cert key override is a how, not a which-domain, so no new scope gate is needed. Verified that a scoped API key can't side-step via the new fields.

Sorry for the bundled PR and the extra round-trip. Splitting them was the right call — each one is now small enough to review in one sitting, and they can move at their own pace.

Let me know if anything else needs adjusting.