Skip to content

fix(hedgedoc): Seeded admin account + R2 snapshot/restore persistence#619

Merged
stefanko-ch merged 3 commits into
mainfrom
fix/hedgedoc-admin-and-persistence
May 27, 2026
Merged

fix(hedgedoc): Seeded admin account + R2 snapshot/restore persistence#619
stefanko-ch merged 3 commits into
mainfrom
fix/hedgedoc-admin-and-persistence

Conversation

@stefanko-ch

Copy link
Copy Markdown
Owner

Summary

Two coupled fixes for HedgeDoc, both required for the stack to be useful:

  1. Seeded admin account — HedgeDoc shipped with email signup disabled AND anonymous editing as the in-app workflow. The auth UI surprised users with a sign-in modal that couldn't accept any credentials, and "anonymous editing behind CF Access" misses HedgeDoc's core value (note ownership, "My Notes", private notes).
  2. R2 snapshot/restore persistence — HedgeDoc was NOT in s3_restore.standard_targets, so every teardown discarded ALL notes + uploads. Independently broken regardless of auth mode.

Login without persistence would be nonsense, so both fixes land together.

Closes #618

Admin authentication

Layer Mechanism
Edge Cloudflare Access (email OTP) — gates who reaches the HedgeDoc login page
In-app Single seeded admin account, credentials in Infisical
  • random_password.hedgedoc_admin in Tofu → outputs.tf secrets → Infisical /hedgedoc as HEDGEDOC_USERNAME (admin email) + HEDGEDOC_PASSWORD.
  • service_env._render_hedgedoc extends the fail-fast guard to require all four (session secret, DB pw, admin pw, admin email).
  • render_hedgedoc_hook (services.py) seeds the admin via node /hedgedoc/bin/manage_users --add EMAIL --pass PASS during services-configure. Idempotent: try --add first, fall back to --reset if user exists (mirrors the Superset hook's create→reset pattern, handles snapshot-restore + Tofu password rotation).
  • Password piped via stdin to docker exec -i so the cleartext never appears in host-side ps argv (same as the Superset hook).
  • Compose flips CMD_ALLOW_ANONYMOUS / ANONYMOUS_EDITS / FREEURL to false and CMD_DEFAULT_PERMISSION to private — newly-created notes are admin-only until explicitly shared via the per-note ⚙ permission selector.

Persistence

  • PostgresDumpTarget(container="hedgedoc-db", database="hedgedoc", user="nexus-hedgedoc") added to standard_targets.
  • hedgedoc-uploads converted from a Docker named volume to a bind-mount on /mnt/nexus-data/hedgedoc/uploads so rsync can capture it (named volumes can't be rsynced without --volumes-from gymnastics).
  • New compose_runner.hedgedoc_uploads_prep flag: idempotent mkdir + owner-guarded chown to 10000:10000 (the in-container hedgedoc UID, verified via docker exec hedgedoc id). Mirrors the existing dify_storage_prep / metabase_storage_prep pattern.
  • Matching RsyncTarget(name="hedgedoc-uploads", s3_subpath="hedgedoc/uploads") so the existing teardown→R2 / spin-up→R2 cycle preserves attachments.

Why this shape

  • Single-admin default per user choice. Convention matches Metabase, Grafana, Portainer — Tofu generates the credential, stack ships locked-down. If multi-user is needed (workshop / pair-collaboration), flip CMD_ALLOW_EMAIL_REGISTER=true and re-deploy; CF Access at the edge still gates entry.
  • Bind-mount over named volume. Named volumes need docker run --volumes-from to be rsynced; bind-mounts on /mnt/nexus-data/ align with the Dify / Metabase / Gitea pattern that s3_restore already handles.
  • manage_users CLI over direct SQL. HedgeDoc bcrypt-hashes the password via Sequelize hooks; replicating that in psql would track upstream's algorithm choices. The CLI is the contract.
  • --add → --reset fallback for idempotency. --add fails (exit 2 + "already exists") on second run; --reset succeeds when the user exists. Together: same logical "ensure admin exists with this password" semantics, robust against R2 snapshot restore from a stack with a different password.

Test plan

  • uv run pytest tests/unit/ — 1552 passed
  • All four secret-guards fire individually + together with a single error message naming all missing keys
  • render_hedgedoc_hook: stdin pipe, password NOT in docker exec argv on any line (HD-CANARY check), email via -e VAR=, both --add and --reset paths rendered
  • compose_runner.hedgedoc_uploads_prep: block only when flagged, owner-guarded chown structure
  • s3_restore.standard_targets: pg-dump pin (container/db/user), rsync subpath pin
  • pre-commit run clean (ruff format, ruff check, mypy strict, shellcheck)

Manual deploy test:

gh workflow run spin-up.yml --ref fix/hedgedoc-admin-and-persistence

Then:

  1. Open https://hedgedoc.<domain> → CF Access OTP → HedgeDoc login form should appear (NO sign-up link).
  2. Username = your ADMIN_EMAIL, password = infisical secrets get HEDGEDOC_PASSWORD --path=/hedgedoc --plain.
  3. Create a note, upload an image.
  4. teardown.ymlspin-up.yml.
  5. Note + image should still be there.

Out of scope

  • Gitea OAuth (alternative auth mechanism). Would be a separate enhancement if SSO across the stack is wanted.
  • LDAP / SAML / external IdP integration.
  • HedgeDoc Pro features.

Files

18 files modified:

  • tofu/stack/main.tf, tofu/stack/outputs.tf — random_password + secret output
  • src/nexus_deploy/config.py, infisical.py — schema + Infisical folder
  • src/nexus_deploy/service_env.py — extended guard + new env vars
  • src/nexus_deploy/services.pyrender_hedgedoc_hook + registry entry
  • src/nexus_deploy/compose_runner.pyhedgedoc_uploads_prep flag
  • src/nexus_deploy/s3_restore.py — pg-dump + rsync targets
  • stacks/hedgedoc/docker-compose.yml — auth flags, bind-mount, comment update
  • docs/stacks/hedgedoc.md — auth model + persistence sections rewritten
  • tests/unit/{test_config,test_compose_runner,test_s3_restore,test_service_env,test_services}.py — new tests + updated fixture
  • tests/fixtures/secrets_full.jsonhedgedoc_admin_password
  • tests/unit/__snapshots__/{test_config,test_infisical}.ambr — regenerated

Copilot AI review requested due to automatic review settings May 24, 2026 08:08
@github-actions

github-actions Bot commented May 24, 2026

Copy link
Copy Markdown
Contributor

coverage

Coverage report — nexus_deploy
FileStmtsMissCoverMissing
__init__.py50100% 
_remote.py150100% 
cli.py40100% 
compose_restart.py400100% 
compose_runner.py840100% 
config.py1370100% 
firewall.py2060100% 
gitea.py5875590%691–692, 697, 720–721, 733–734, 770–771, 783–784, 802–803, 828–829, 851–852, 863–864, 919–920, 928–929, 934, 940–941, 965–966, 999–1000, 1003, 1034–1035, 1076–1077, 1082–1083, 1123–1124, 1155–1156, 1179–1180, 1185–1186, 1285–1286, 1291–1292, 1766, 1770, 1791, 1819–1820, 1907
hetzner_capacity.py1260100% 
infisical.py2030100% 
kestra.py176398%223, 427, 768
orchestrator.py6207388%456, 616, 628, 798–799, 804–805, 837–839, 848, 853–855, 866, 903–904, 909–910, 930, 965–966, 971–972, 980, 1005–1006, 1014, 1036–1037, 1042–1043, 1095–1096, 1101–1102, 1335, 1338, 1408, 1414–1415, 1420–1421, 1455, 1562–1563, 1568–1569, 1618–1619, 1624–1625, 1684, 1699, 1756, 1761–1762, 1767–1768, 1775, 1781, 1948, 1955, 1967–1968, 1973–1974, 1980, 1986, 2059–2060, 2081–2082
pipeline.py2071393%165–166, 350, 388, 468, 485, 580–581, 626–627, 717–718, 765
r2_tokens.py113298%87, 150
s3_persistence.py199199%315
s3_restore.py1030100% 
secret_sync.py990100% 
seeder.py980100% 
service_env.py3833391%1083, 1085–1087, 1095–1096, 1424–1427, 1432–1438, 1467–1471, 1487–1491, 1513, 1515, 1537–1538, 1545, 1654
services.py319199%2018
setup.py1651392%238, 308–311, 319, 323–328, 344
ssh.py560100% 
stack_sync.py960100% 
tfvars.py440100% 
tofu.py860100% 
workspace_coords.py1010100% 
TOTAL427219495% 

@codecov

codecov Bot commented May 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes the HedgeDoc stack usable by (1) introducing a seeded in-app admin account (with self-signup + anonymous edits disabled) and (2) adding full R2 snapshot/restore coverage for HedgeDoc’s Postgres DB and uploads directory, so notes and attachments survive teardown/spin-up cycles.

Changes:

  • Add a new OpenTofu-generated HedgeDoc admin password, plumb it through config/Infisical, and add a services-configure hook to idempotently create/reset the admin via manage_users.
  • Add HedgeDoc to s3_restore.standard_targets (Postgres dump + uploads rsync) and switch uploads from a named volume to a /mnt/nexus-data/... bind mount with a guarded chown prep step.
  • Update HedgeDoc compose + docs + unit tests/snapshots to reflect the new auth and persistence model.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tofu/stack/main.tf Adds random_password.hedgedoc_admin for the seeded HedgeDoc admin credential.
tofu/stack/outputs.tf Exposes hedgedoc_admin_password via the secrets output for downstream sync.
src/nexus_deploy/config.py Extends the secrets schema to include hedgedoc_admin_password.
src/nexus_deploy/infisical.py Pushes HedgeDoc admin username/password into the /hedgedoc Infisical folder.
src/nexus_deploy/service_env.py Adds HedgeDoc admin email/password to rendered env and extends fail-fast guards.
src/nexus_deploy/services.py Implements render_hedgedoc_hook and registers it in the hook registry.
src/nexus_deploy/compose_runner.py Adds hedgedoc_uploads_prep to prep/chown the uploads bind mount directory.
src/nexus_deploy/s3_restore.py Adds HedgeDoc Postgres dump + uploads rsync targets to standard persistence.
stacks/hedgedoc/docker-compose.yml Disables anonymous/signup, sets private-by-default notes, converts uploads to bind mount.
docs/stacks/hedgedoc.md Updates usage/auth/persistence docs for the seeded-admin + R2 persistence behavior.
tests/unit/test_config.py Updates schema field-count assertion.
tests/unit/test_service_env.py Adds coverage for new HedgeDoc admin guardrails and env rendering.
tests/unit/test_services.py Adds coverage for the rendered HedgeDoc admin-seed hook script behavior.
tests/unit/test_compose_runner.py Adds coverage for the HedgeDoc uploads prep block rendering/defaulting.
tests/unit/test_s3_restore.py Adds assertions for the new HedgeDoc persistence targets.
tests/fixtures/secrets_full.json Adds fixture value for hedgedoc_admin_password.
tests/unit/snapshots/test_config.ambr Updates snapshot to include the new config field.
tests/unit/snapshots/test_infisical.ambr Updates snapshot to include the new HedgeDoc Infisical secrets.

Comment thread src/nexus_deploy/service_env.py
stefanko-ch added a commit that referenced this pull request May 24, 2026
Copilot flagged that _render_hedgedoc rendered
HEDGEDOC_ADMIN_EMAIL / HEDGEDOC_ADMIN_PASSWORD env vars into the
.env file, but the Infisical-stored keys are HEDGEDOC_USERNAME /
HEDGEDOC_PASSWORD — an operator grepping for the latter in
deploy-error output would come up empty and stay stuck.

Root cause: those env vars were dead code. The
docker-compose.yml doesn't reference HEDGEDOC_ADMIN_* and the
services-configure hook reads the values straight from NexusConfig
/ BootstrapEnv (Python-side), not from the rendered .env. They
were a speculative "future container-side automation" that never
materialised, and added confusion as a naming-mismatch surface.

- Drop HEDGEDOC_ADMIN_EMAIL / HEDGEDOC_ADMIN_PASSWORD from the
  rendered .env. Keep the four-way fail-fast guard (it checks
  the underlying config / env values, no longer the rendered .env
  entries).
- Rewrite the guard error message to use the Infisical key names
  (HEDGEDOC_PASSWORD / HEDGEDOC_USERNAME) so operators grepping
  the failure find the matching secret in Infisical directly.
- Add explicit "(Infisical /hedgedoc)" annotation per missing key
  to remove the translation step entirely.
- Tests updated: assert the admin keys are NOT in rendered env
  vars, error-message regex switched to the Infisical names.

Snapshot regen for test_config.py covers the pre-existing
trailing-newline issue that was masked by the previous full-suite
runs.
@stefanko-ch stefanko-ch requested a review from Copilot May 27, 2026 05:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Comment thread src/nexus_deploy/s3_restore.py
stefanko-ch added a commit that referenced this pull request May 27, 2026
HedgeDoc was added to `standard_targets()` with both a PostgresDumpTarget
(safe) AND an `hedgedoc-uploads` RsyncTarget against
`/mnt/nexus-data/hedgedoc/uploads`, but the corresponding compose-file
entry was missed in `_STANDARD_STOP_COMPOSE_FILES`. Snapshot rsync
therefore ran against the live container, producing torn writes /
half-uploaded attachments — same class of bug the Metabase stop entry
was added to fix in issue #528.

Added the missing stop entry. Also added
`test_standard_stop_compose_files_matches_rsync_targets` as a regression
guard: derives the expected stop list from `standard_targets()` rsync
targets and asserts symmetry in both directions (no RsyncTarget without
a stop entry, no dead stop entry without a matching consumer). Future
stack additions get a hard fail at test time instead of silent
production corruption.

Expanded the inline rationale block on `_STANDARD_STOP_COMPOSE_FILES`
to make the "rsync target ⇒ stop entry" invariant explicit and to
reference the new test by name.

Addresses Copilot review comment 3309024957 on PR #619.
@stefanko-ch stefanko-ch requested a review from Copilot May 27, 2026 07:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated no new comments.

Two coupled fixes, both required for HedgeDoc to be a useful stack:

1. Admin authentication
   Previously HedgeDoc shipped with CMD_ALLOW_EMAIL_REGISTER=false
   and CMD_ALLOW_ANONYMOUS=true — relying on CF Access at the edge
   as the only auth gate and anonymous editing as the in-app
   workflow. That misses HedgeDoc's core value (note ownership,
   "My Notes" list, private notes) and surprised the operator with
   a sign-in modal that couldn't accept any credentials.

   - random_password.hedgedoc_admin in Tofu, surfaced via outputs.tf
     and pushed to Infisical /hedgedoc as HEDGEDOC_USERNAME (admin
     email) + HEDGEDOC_PASSWORD.
   - service_env._render_hedgedoc renders HEDGEDOC_ADMIN_EMAIL +
     HEDGEDOC_ADMIN_PASSWORD env vars and fail-fast guards all four
     required secrets together.
   - render_hedgedoc_hook (services.py) seeds the admin via
     `node /hedgedoc/bin/manage_users --add EMAIL --pass PASS`
     during services-configure, with --reset fallback for
     idempotency on re-runs (matches the Superset hook's
     create→reset pattern). Password piped via stdin so the
     cleartext never appears in host-side `ps` argv.
   - compose flips CMD_ALLOW_ANONYMOUS / ANONYMOUS_EDITS / FREEURL
     to false and CMD_DEFAULT_PERMISSION to private — newly-created
     notes are admin-only until explicitly shared via the per-note
     permission selector.

2. Persistence
   HedgeDoc was NOT in s3_restore.standard_targets, so every
   teardown discarded all notes + uploads. Independently broken
   regardless of auth mode.

   - PostgresDumpTarget(container="hedgedoc-db", database="hedgedoc",
     user="nexus-hedgedoc") added to standard_targets.
   - Convert hedgedoc-uploads from a Docker named volume to a
     bind-mount on /mnt/nexus-data/hedgedoc/uploads so rsync can
     capture it.
   - New compose_runner flag hedgedoc_uploads_prep does the
     idempotent mkdir + owner-guarded chown to 10000:10000 (the
     in-container hedgedoc UID, verified via `docker exec`) before
     compose-up. Mirrors the existing dify_storage_prep /
     metabase_storage_prep pattern.
   - Matching RsyncTarget added to standard_targets so the existing
     teardown→R2 / spin-up→R2 cycle preserves attachments.

Tests cover all four guards in service_env, both compose_runner flag
behaviours, hook stdin-pipe + idempotency, and the standard_targets
additions (Postgres user/db pin + rsync subpath layout). Snapshots
regenerated for test_config.py and test_infisical.py. secrets_full
fixture gains hedgedoc_admin_password.

Closes #618
Copilot flagged that _render_hedgedoc rendered
HEDGEDOC_ADMIN_EMAIL / HEDGEDOC_ADMIN_PASSWORD env vars into the
.env file, but the Infisical-stored keys are HEDGEDOC_USERNAME /
HEDGEDOC_PASSWORD — an operator grepping for the latter in
deploy-error output would come up empty and stay stuck.

Root cause: those env vars were dead code. The
docker-compose.yml doesn't reference HEDGEDOC_ADMIN_* and the
services-configure hook reads the values straight from NexusConfig
/ BootstrapEnv (Python-side), not from the rendered .env. They
were a speculative "future container-side automation" that never
materialised, and added confusion as a naming-mismatch surface.

- Drop HEDGEDOC_ADMIN_EMAIL / HEDGEDOC_ADMIN_PASSWORD from the
  rendered .env. Keep the four-way fail-fast guard (it checks
  the underlying config / env values, no longer the rendered .env
  entries).
- Rewrite the guard error message to use the Infisical key names
  (HEDGEDOC_PASSWORD / HEDGEDOC_USERNAME) so operators grepping
  the failure find the matching secret in Infisical directly.
- Add explicit "(Infisical /hedgedoc)" annotation per missing key
  to remove the translation step entirely.
- Tests updated: assert the admin keys are NOT in rendered env
  vars, error-message regex switched to the Infisical names.

Snapshot regen for test_config.py covers the pre-existing
trailing-newline issue that was masked by the previous full-suite
runs.
HedgeDoc was added to `standard_targets()` with both a PostgresDumpTarget
(safe) AND an `hedgedoc-uploads` RsyncTarget against
`/mnt/nexus-data/hedgedoc/uploads`, but the corresponding compose-file
entry was missed in `_STANDARD_STOP_COMPOSE_FILES`. Snapshot rsync
therefore ran against the live container, producing torn writes /
half-uploaded attachments — same class of bug the Metabase stop entry
was added to fix in issue #528.

Added the missing stop entry. Also added
`test_standard_stop_compose_files_matches_rsync_targets` as a regression
guard: derives the expected stop list from `standard_targets()` rsync
targets and asserts symmetry in both directions (no RsyncTarget without
a stop entry, no dead stop entry without a matching consumer). Future
stack additions get a hard fail at test time instead of silent
production corruption.

Expanded the inline rationale block on `_STANDARD_STOP_COMPOSE_FILES`
to make the "rsync target ⇒ stop entry" invariant explicit and to
reference the new test by name.

Addresses Copilot review comment 3309024957 on PR #619.
@stefanko-ch stefanko-ch force-pushed the fix/hedgedoc-admin-and-persistence branch from 59aefcc to 91e2e59 Compare May 27, 2026 07:33
@stefanko-ch stefanko-ch merged commit a04d529 into main May 27, 2026
8 checks passed
@stefanko-ch stefanko-ch deleted the fix/hedgedoc-admin-and-persistence branch May 27, 2026 07:42
stefanko-ch pushed a commit that referenced this pull request May 28, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.68.0](v0.67.0...v0.68.0)
(2026-05-28)


### 🚀 Features

* **stacks:** Add Evidence — SQL+markdown BI for analytics engineers
([#616](#616))
([ac9ef64](ac9ef64))


### 🐛 Bug Fixes

* **hedgedoc:** Seeded admin account + R2 snapshot/restore persistence
([#619](#619))
([a04d529](a04d529))
* **pipeline:** Pre-create ollama-internal network when LiteLLM enabled
([#617](#617))
([3ea36be](3ea36be))
* **stacks:** Raise Metabase JVM heap to prevent OOM
([#620](#620))
([70bb786](70bb786))
* **stacks:** Remove hardcoded credential fallbacks (audit C1-C3)
([#623](#623))
([3002629](3002629))
* **stacks:** Resolve host-port 8888 collision (adminer ⇔ seaweedfs)
([#624](#624))
([6831d2c](6831d2c))
* **tofu:** Validate cloudflare_account_id, cloudflare_zone_id, domain
shapes ([#622](#622))
([47f25c6](47f25c6))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(stacks): HedgeDoc admin auth + R2 snapshot/restore persistence

2 participants