feat: migrate default Apps to HelmReleases#1758
Open
fiunchinho wants to merge 38 commits into
Open
Conversation
fiunchinho
force-pushed
the
migrate-apps-helmreleases
branch
13 times, most recently
from
February 16, 2026 23:06
92d68a2 to
387acd3
Compare
weatherhog
added a commit
that referenced
this pull request
Mar 13, 2026
Replace the App CR template with OCIRepository + HelmRelease pattern, aligning with the ongoing migration in PR #1758. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
5 tasks
fiunchinho
force-pushed
the
migrate-apps-helmreleases
branch
from
March 16, 2026 12:22
95795cc to
197371b
Compare
mcharriere
added a commit
that referenced
this pull request
Apr 13, 2026
* Add external-dns-crossplane-resources App CR
Deploy external-dns-crossplane-resources chart to create the AWS IAM
role for external-dns Route53 access via Crossplane. This replaces
the IAM role previously managed by capa-iam-operator.
The App CR follows the same pattern as cilium-crossplane-resources and
cert-manager-crossplane-resources, using the shared crossplane-config
ConfigMap for cluster-specific values (clusterName, accountID, baseDomain).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Update external-dns role-arn to match Crossplane resource name
The Crossplane IAM Role is named {clusterName}-external-dns, replacing
the {clusterName}-Route53Manager-Role previously created by
capa-iam-operator.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Migrate external-dns-crossplane-resources from App CR to HelmRelease
Replace the App CR template with OCIRepository + HelmRelease pattern,
aligning with the ongoing migration in PR #1758.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Inline values in HelmRelease instead of depending on ConfigMap
Replace valuesFrom referencing the crossplane-config ConfigMap with
inline values. This removes the runtime dependency on
aws-crossplane-cluster-config-operator since clusterName, accountID,
and baseDomain are all available at template time.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Extract aws-oidc-domain helper and inject oidcDomain/oidcDomains into external-dns-crossplane-resources
- Add _oidc.tpl with aws-oidc-domain helper (domain without https://, China-aware)
- Replace inline conditional in cert-manager-crossplane-resources-app.yaml with the helper
- Inject oidcDomain and oidcDomains values into external-dns-crossplane-resources HelmRelease
* Add changelog entries for external-dns-crossplane-resources branch
* Inject awsPartition and awsRegion into external-dns-crossplane-resources HelmRelease
* change role name only if the app is installed
Signed-off-by: Matias Charriere <matias@giantswarm.io>
* fix conditional
Signed-off-by: Matias Charriere <matias@giantswarm.io>
* use a new dict for app validation
Signed-off-by: Matias Charriere <matias@giantswarm.io>
---------
Signed-off-by: Matias Charriere <matias@giantswarm.io>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Matias Charriere <matias@giantswarm.io>
fiunchinho
force-pushed
the
migrate-apps-helmreleases
branch
from
April 21, 2026 07:42
698f51b to
27e3b92
Compare
7 tasks
fiunchinho
marked this pull request as ready for review
May 20, 2026 10:18
fiunchinho
changed the title
Gacko
force-pushed
the
migrate-apps-helmreleases
branch
6 times, most recently
from
July 7, 2026 09:43
7ff2922 to
3f9f6f9
Compare
Switch all HelmReleases in cluster-aws back to per-app OCIRepository (v1beta2). Bump cluster subchart with OCIRepository support and removed HelmRepositories.
Allows Flux to detect and recreate any aws-nth-bundle sub-HelmReleases that get deleted during the App CR to HelmRelease migration.
- irsa-servicemonitors: remove valuesFrom cluster-values (unused) - aws-ebs-csi-driver-servicemonitors: remove valuesFrom cluster-values (unused) - aws-pod-identity-webhook: remove valuesFrom cluster-values, pass provider: capa explicitly in default values template
Pull in the cluster subchart change that runs the bundle App CR cascade in the background after Phase 0b has scaled chart-operator-WC to 0. This prevents the migration hook from exceeding helm's default 5-minute hook timeout under concurrent multi-cluster migration load.
The pre-upgrade migration hook now lifts the cluster App CR's spec.upgrade.timeout to 15m before doing its Phase A→C work, so chart-operator-MC's next helm-upgrade attempt has a generous timeout window. The default 5-min timeout was hitting under fresh-state 5-cluster concurrent migration load and triggering a retry storm.
Round 44 forensic logs (continuous chart-operator-WC log capture) disproved the graceful-drain hypothesis and revealed the actual root cause: a race on the MC between chart-operator-MC's bundle uninstall (triggered by old Phase D's bundle App CR deletion) and Flux helm-controller's bundle upgrade (which had already adopted the bundle helm release and installed sub-app HelmReleases as manifest items). helm uninstall DELETEs the manifest items, helm-controller's HR-deletion handler runs helm uninstall on the WC release, app lost. The new hook pauses bundle App CRs + bundle Chart CRs on MC in Phase A, strips their finalizers in Phase B, and deletes them instantly in Phase D. Bundle helm releases on MC stay alive for Flux to adopt.
The aws-ebs-csi-driver-bundle's sub-chart (aws-ebs-csi-driver) ships a PolicyException that triggers Kyverno's validating webhook on apply. Without this dep the bundle upgrade can race kyverno's own admission-controller rolling restart and fail with a webhook-unreachable error, then wedge Flux v1.3.0 on the missing-rollback-target pattern. Gating the bundle's upgrade on kyverno being Ready closes the migration window where these chart upgrades happen concurrently.
This reverts commit 39c9b9d.
Apply the same change as in aws-nth-helmrelease.yaml (which already had it) to every other HR template in this chart: aws-ebs-csi-driver-*, aws-pod-identity-webhook, cert-manager-crossplane-resources, cilium-*, cloud-provider-aws, irsa-servicemonitors, karpenter. Both install and upgrade remediation blocks. Bypasses Flux helm-controller's rollback step on upgrade failure. Combined with retries: -1, helm-controller skips remediation and re-attempts the upgrade on its next reconcile. Avoids the wedge that occurs when adopting chart-operator-installed v1 releases (rollback fails because v1 isn't owned by helm-controller).
- aws-ebs-csi-driver-servicemonitors HR: revert spec.releaseName from aws-ebs-csi-driver-servicemonitors back to aws-ebs-csi-driver-smons. This makes the HR adopt the existing chart-operator-installed release on the WC instead of leaving it orphaned and installing a parallel release with the new name. The chart pulled by chartRef is still aws-ebs-csi-driver-servicemonitors — only the release name reverts. - Bump cluster subchart from 77c9a80 to 884d0fc to pull in the migration hook fix that adds a sixth neutralization category for bundle-rendered MC HelmReleases. Round 68 confirmed aws-nth-bundle's inner HRs were being deleted by Phase D's helm uninstall, which fired Flux helm-controller's finalizer and ran helm uninstall on the managed WC release.
…Pod Identity webhook and fix cert-manager schema.
Gacko
force-pushed
the
migrate-apps-helmreleases
branch
2 times, most recently
from
July 22, 2026 04:53
ca9975e to
2918aa8
Compare
Contributor
|
There were differences in the rendered Helm template, please check! Output |
Gacko
force-pushed
the
migrate-apps-helmreleases
branch
from
July 22, 2026 05:20
2918aa8 to
5273e99
Compare
Contributor
|
There were differences in the rendered Helm template, please check! Output |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aws-nth-bundle,aws-pod-identity-webhook,aws-ebs-csi-driver-servicemonitors,irsa-servicemonitors,cert-manager-crossplane-resources,cilium-crossplane-resourcesclustersubchart has its own migration hook for the 20 generic apps (see companion PR)How the migration works
helm upgrade:clustersubchart hook (weight -10) handles 20 generic appscluster-awshook (weight -5) handles 6 AWS-specific appshelm uninstallbecause the Chart CRs are already gonehelm upgradeRelated PR
Test plan
helm templaterenders HelmReleases instead of App CRs for all migrated apps (tested all 27 CI values files)Ready: Trueafter upgradegiantswarmnamespace after upgrade