Chart: Helm Refurbish #64037
Description
Body
As part of our ongoing Helm chart refurbish work in Apache Airflow, we will follow a structured approach to ensure stability for existing users while preparing for a clean, well-defined next major release.
https://cwiki.apache.org/confluence/display/AIRFLOW/Helm+Refurbish
Approach
We will first focus on cleanup and refurbishment within the current chart before introducing a new major version. Instead of making breaking changes directly on the main branch while releasing from the main branch, we will follow a staged approach similar to that used for the v3-1-test branch and backport changes specifically.
The chart/v1-2x-test branch has been created to continue development and stabilisation of the current minor version line.
This branch will be used to release any remaining fixes and improvements for older minor versions.
The main branch will be reserved for cleanup, deprecations, and preparation work toward the next major release.
This allows us to separate stability and innovation clearly:
- Stable releases continue from the test branch
- Major version preparation happens on main without constraints
- Plan for Changes
All cleanup, deprecations, and restructuring work will be implemented on the main branch. Once validated, we will cherry-pick relevant changes back to the test branch where appropriate.
- Each configuration change and cleanup task will be tracked via individual tickets
- These tickets will be linked to this umbrella issue for visibility and coordination
- The goal is to progressively reduce technical debt and simplify the chart before cutting a major release
- Important Guidelines
No feature will be further added to the chart/v1-2x-test branch. This branch is strictly for maintenance, stability, and compatibility purposes for the next minor releases.
Parameters from Docs
Common
Move to Kustomize
- schedulerName
- workers.schedulerName
Depercate and Clean
- webserver.webserverConfig
- webserver.webserverConfigConfigMapName
- webserverSecretAnnotations
- webserverSecretKey
Airflow
Move to Kustomize
Elasticsearch
- elasticsearch.connection
- elasticsearch.connection.host
- elasticsearch.connection.pass
- elasticsearch.connection.port
- elasticsearch.connection.scheme
- elasticsearch.connection.user
- elasticsearch.enabled
- elasticsearch.secretAnnotations
- elasticsearch.secretName
Opensearch
- opensearch.connection
- opensearch.connection.host
- opensearch.connection.pass
- opensearch.connection.port
- opensearch.connection.scheme
- opensearch.connection.user
- opensearch.enabled
- opensearch.secretName
Depercate and Clean
- webserverSecretKeySecretName
Will be handled under gitSync bundle in the next heading.
- dags.gitSync.branch
- dags.gitSync.containerName
- dags.gitSync.credentialsSecret
- dags.gitSync.depth
- dags.gitSync.emptyDirConfig
- dags.gitSync.enabled
- dags.gitSync.env
- dags.gitSync.envFrom
- dags.gitSync.extraVolumeMounts
- dags.gitSync.knownHosts
- dags.gitSync.maxFailures
- dags.gitSync.period
- dags.gitSync.ref
- dags.gitSync.repo
- dags.gitSync.resources
- dags.gitSync.rev
- dags.gitSync.securityContext
- dags.gitSync.sshKey
- dags.gitSync.sshKeySecret
- dags.gitSync.subPath
- dags.gitSync.uid
- dags.gitSync.wait
Images
Current Status: In the Images section, defining values in multiple places can become cumbersome. We should evaluate whether duplicate definitions can be removed.
Additionally, consider reorganising these into separate components to improve structure and maintainability.
- TBD
Will be handled under gitSync bundle in the next heading.
- images.gitSync.tag
- images.gitSync.pullPolicy
- images.gitSync.repository
Ports
- TBD
Database
- TBD
PgBouncer
- TBD
API Server
- TBD
Scheduler
- TBD
Webserver
- Entire section
Workers
- TBD
Triggerer
- TBD
DagProcessor
- TBD
Flower
- TBD
Redis
- TBD
StatsD
- TBD
Jobs
- TBD
Kubernetes
- TBD
Ingress
- TBD
Kerberos
- TBD
Parameters considered as bundle
gitSync
The first focus will be enabling GitDagBundle to work smoothly with Helm, along with clear and complete documentation.
The cleanup will not start immediately with branching. Instead, after GitDagBundle includes proper documentation for setting up and using the git CLI or callable configuration, the gitSync as a bundle will be deprecated and cleaned.
Refs
TBD: To Be Discussed
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.