Add production deployment configuration and CI/CD#194
Draft
cooper667 wants to merge 43 commits intockan211-python310-migration-staging-1from
Draft
Add production deployment configuration and CI/CD#194cooper667 wants to merge 43 commits intockan211-python310-migration-staging-1from
cooper667 wants to merge 43 commits intockan211-python310-migration-staging-1from
Conversation
cooper667
commented
Jan 22, 2026
cooper667
commented
- Add deploy/ folder with Dockerfile.prod, nginx, uwsgi configs
- Add production.ini (secrets externalized to secrets.ini)
- Add entrypoint that merges production.ini + secrets.ini at startup
- Add build-deploy.yml GitHub Actions workflow
- Add dependabot.yml
- Update supervisor config with nginx and uwsgi programs
- Add deploy/ folder with Dockerfile.prod, nginx, uwsgi configs - Add production.ini (secrets externalized to secrets.ini) - Add entrypoint that merges production.ini + secrets.ini at startup - Add build-deploy.yml GitHub Actions workflow - Add dependabot.yml - Update supervisor config with nginx and uwsgi programs
Previous commits were force-pushed away from upstream repos.
Change GitHub environment URL for staging deployments to reflect the new domain.
…ploads Updates ckanext-unaids to 5e557c3 which adds CSRF token to file upload authorization requests, fixing 400 errors when uploading files in CKAN 2.11.
Support all package types (dataset, dataset-2, etc.) in download routes. DataPusher was failing with 404 for resources using custom package types.
- Change staging domain from dev-adr to dev.adr.fjelltopp.org - Enable saml2auth plugin and configure Auth0 IDP - Re-enable login/register redirect to SAML2 login - Update ckanext-unaids submodule URL to fork
Bake production.ini into image so config changes flow through CI/CD.
Secrets are still merged at runtime via entrypoint from secrets.ini.
After this deploys, run:
kubectl patch deployment ckan -n adr-s --type='json' -p='[
{"op": "replace", "path": "/spec/template/spec/volumes/3/projected/sources", "value": [
{"secret": {"name": "jwt-keys"}},
{"secret": {"name": "ckan-ini-secrets"}}
]}
]'
- Dockerfile bakes config as /etc/ckan/base.ini - Entrypoint merges base.ini + secrets.ini → /etc/ckan/production.ini - Allows subPath mounts for secrets without overwriting base config After deploy, apply subPath mount patch (see commit message).
Config merge order at startup: base.ini < env.ini < secrets.ini - deploy/base.ini: common config (baked into image) - deploy/staging.ini: staging-specific (CI creates ConfigMap) - deploy/production.ini: prod-specific (CI creates ConfigMap) - Entrypoint merges all three into /tmp/production.ini - CI workflow creates ckan-env-config ConfigMap per environment
cooper667
force-pushed
the
ckan211-prod-deploy-pr
branch
from
10a5add to
21d7e3b
Compare
Point submodule back to fjelltopp/ckanext-unaids instead of fork, using the same commit as the base branch.
The harvester and background job workers were FATAL in AKS because supervisor had hardcoded /usr/local/bin/ckan and /etc/ckan/ckan.ini. In the prod image the binary lives in the virtualenv and config is merged to /tmp/production.ini. Use supervisor %(ENV_X)s substitution so the same config works for both dev and prod entrypoints.
/dev/fd/N in child processes points to supervisor's captured pipes, not the container stdout, causing all logs to be swallowed. Using /dev/stdout and /dev/stderr ensures output reaches kubectl logs.
The ckan-worker supervisor process was configured to run as user=ckan, but Azure PostgreSQL requires SSL and psycopg2 looks for a client cert at /root/.postgresql/postgresql.crt which the ckan user cannot access. This matches all other supervisor processes which already run as root.
Configures email sending through ACS using the verified adr-s.fjelltopp.org domain. Credentials are in K8s secret.
…g fixes - Bump ckanext-blob-storage: always create fresh LfsClient to avoid token mismatch - Bump ckanext-dhis2harvester: sync to remote fix/ckan-2.11-bugfixes (Python 3 compat) - Add rebuild_solr_index.sh: convenience script to rebuild Solr index after stack restart - fix(util): use docker compose (v2) instead of deprecated docker-compose (v1) - chore(.gitignore): ignore local config and utility scripts
Adds a workflow_dispatch-only job that runs `ckan search-index rebuild -i -q` via kubectl exec against the chosen environment (staging or production). Trigger from the Actions tab when a reindex is actually needed — most deploys don't require one, so wiring it into build-deploy.yml would just add latency and mask real indexing errors.
- ckanext-blob-storage: use current_user context; improve uploader error handling - ckanext-dhis2harvester: sanitize harvester error output; fix harvest_search route - ckanext-fork: address review comments; document resource_autocomplete wildcard limits - ckanext-harvest: fix has_request_context auth check; tighten sysadmin bypass - ckanext-restricted: gate REMOTE_USER fallback to testing; narrow ValueError catch - ckanext-saml2auth: harden SAML session handling - ckanext-scheming: escape alert_warning XSS; add missing asset snippets - ckanext-unaids: Bootstrap 5 pull-right/left fixes; CKAN 2.11 migration follow-ups - ckanext-validation: bound site-user JWT lifetime; restrict auth header scope - ckanext-versions: untrack internal notes - ckanext-ytp-request: secure _get_user() with DB validation of REMOTE_USER - datapusher: migrate to Python 3.10; pin ckanserviceprovider 1.2.0
- build-deploy.yml now only targets staging (adr-s) on push to master or ckan211-prod-deploy-pr; workflow_dispatch takes optional image_tag to redeploy an existing image without rebuilding. - release-deploy.yml (new) triggers on push of v* tags (covers both 'git push --tags' and GitHub Release publish). Does not build; promotes the existing sha-<commit> staging image to a v* tag in ACR via 'docker buildx imagetools create', then rolls adr-p over to it. Fails loud if the sha image is missing.
Override base.ini (which points at the dev Auth0 tenant) so prod authenticates against the production Auth0 app on auth-hivtools.unaids.org.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.