fix: Add controller finalizer when policy is active state only#1775
Conversation
There was a problem hiding this comment.
Pull request overview
This PR changes policy finalizer handling so controller-managed finalizers are no longer added by admission webhooks at creation time, and are instead added during reconciliation once policy activation can proceed. It also adds Helm pre-delete cleanup logic and tests around the finalizer lifecycle.
Changes:
- Removed finalizer injection from policy and policy group defaulters, with unit test expectations updated.
- Added finalizer add/remove logic to the policy reconciler around active/unavailable policy server states.
- Added a Helm pre-delete hook fallback and new controller integration tests for finalizer lifecycle behavior.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
internal/controller/policy_subreconciler.go |
Moves finalizer management into policy reconciliation. |
internal/controller/policy_finalizer_test.go |
Adds integration-style tests for finalizer lifecycle scenarios. |
charts/kubewarden-controller/templates/pre-delete-hook.yaml |
Expands uninstall hook to delete PolicyServers and patch remaining policy finalizers. |
api/policies/v1/clusteradmissionpolicygroup_webhook.go |
Stops adding finalizer during cluster policy group defaulting. |
api/policies/v1/clusteradmissionpolicygroup_webhook_test.go |
Updates defaulter test to expect no finalizer. |
api/policies/v1/clusteradmissionpolicy_webhook.go |
Stops adding finalizer during cluster policy defaulting. |
api/policies/v1/clusteradmissionpolicy_webhook_test.go |
Updates defaulter test to expect no finalizer. |
api/policies/v1/admissionpolicygroup_webhook.go |
Stops adding finalizer during namespaced policy group defaulting. |
api/policies/v1/admissionpolicygroup_webhook_test.go |
Updates defaulter test to expect no finalizer. |
api/policies/v1/admissionpolicy_webhook.go |
Stops adding finalizer during namespaced policy defaulting. |
api/policies/v1/admissionpolicy_webhook_test.go |
Updates defaulter test to expect no finalizer. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8089dfd to
a8a2d87
Compare
Mutating webhooks added finalizers to all policies at creation, even if the policy never became active, leaving policies with finalizers but no webhooks. The reconciler now handles finalizers based on actual webhook state: add when the policy becomes active, remove when the webhook is deleted. This updates all four policy webhook types and their tests. Assisted-by: Claude Code Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
The reconciler adds finalizers before creating webhooks and removes them when the PolicyServer is deleted or unavailable. This prevents webhook leaks if the controller crashes between the two steps. Removing the finalizer when the PolicyServer is gone lets policies be deleted without blocking CRD removal. Assisted-by: Claude Code Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
Updates the pre-delete hook to wait for the policy server deletion. This is added to give time to the controller to reconciate the policies of the policy server. Assisted-by: Claude Code Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
…rver deleted Tests now check that finalizers appear when policies reach active state and disappear when the PolicyServer is deleted. Also covers the upgrade case where policies might have the old pre-1.14 finalizer that needs cleaning up. Assisted-by: Claude Code Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
| // Verify policy has finalizer before deletion | ||
| require.True(t, containsFinalizer(policy.GetFinalizers(), constants.KubewardenFinalizer), | ||
| "Policy should have finalizer before deletion") | ||
|
|
||
| // Delete the policy | ||
| err := cfg.Client().Resources().Delete(ctx, policy) |
| require.True(t, containsFinalizer(policy.GetFinalizers(), constants.KubewardenFinalizer), | ||
| "Policy should have finalizer before deletion") |
There was a problem hiding this comment.
Done in a6aebc2, which tackles this and the previous copilot review comment.
| // Verify finalizer is removed from policy | ||
| err = wait.For(conditions.New(cfg.Client().Resources()).ResourceMatch( | ||
| &policiesv1.ClusterAdmissionPolicy{ObjectMeta: metav1.ObjectMeta{Name: policyName}}, | ||
| func(object k8s.Object) bool { | ||
| p := object.(*policiesv1.ClusterAdmissionPolicy) | ||
| return !containsFinalizer(p.GetFinalizers(), constants.KubewardenFinalizer) | ||
| }, | ||
| ), wait.WithTimeout(testTimeout), wait.WithInterval(testPollInterval)) | ||
| require.NoError(t, err, "Finalizer should be removed when PolicyServer is deleted") | ||
|
|
||
| // Verify policy status transitioned to scheduled | ||
| var policy policiesv1.ClusterAdmissionPolicy | ||
| err = cfg.Client().Resources().Get(ctx, policyName, "", &policy) | ||
| require.NoError(t, err) | ||
| require.Equal(t, policiesv1.PolicyStatusScheduled, policy.Status.PolicyStatus, | ||
| "Policy should be scheduled after PolicyServer deletion") | ||
|
|
| controllerutil.RemoveFinalizer(policy, constants.KubewardenFinalizer) | ||
| controllerutil.RemoveFinalizer(policy, constants.KubewardenFinalizerPre114) | ||
| if err := r.Update(ctx, policy); err != nil { | ||
| return ctrl.Result{}, fmt.Errorf("cannot update policy: %w", err) | ||
| } |
There was a problem hiding this comment.
This is valid.
In addition, this new block in reconcilePolicyServerUnavailable() is the same as we already have in reconcilePolicyDeletion(), we can extract it to a function.
There was a problem hiding this comment.
This commit on my fork addresses this and also DRYes 2 functions into one: viccuad@817f51b
Please review, I'm happy pushing it to this PR's branch.
There was a problem hiding this comment.
I like this refactor, do you want to push a new PR?
Merge `deleteWebhookConfiguration()` and `reconcilePolicyDeletion()` into `removePolicyWebhooksAndFinalizers()`. This removes repetion and gives us a single place where the webhooks are removed, and we change the finalizers. Signed-off-by: Víctor Cuadrado Juan <vcuadradojuan@suse.de>
| require.NoError(t, err, "ValidatingWebhookConfiguration should be deleted") | ||
|
|
||
| // Wait for policy to be deleted (finalizer removed) | ||
| err = wait.For(conditions.New(cfg.Client().Resources()).ResourceDeleted(policy), |
There was a problem hiding this comment.
This can never succeed, as we add a special integrationTestFinalizer in the policy factories: https://github.com/kubewarden/adm-controller/blob/finalizer-removal/api/policies/v1/factories.go#L225-L234
This is the reason why the e2e tests fail on CI.
flavio
left a comment
There was a problem hiding this comment.
It looks good. We need to address the failures and the some of the comments from copilot, as highlighted by Victor.
The behavior has now changed: Finalizers are only added to policies not on creation, but when they get their associated WebhookConfigurations. Don't check for actual policy deletion, as that will never happen. Leave around the `integrationTestsFinalizer`. Signed-off-by: Víctor Cuadrado Juan <vcuadradojuan@suse.de>
When reconciling a policy whose PolicyServer is gone, 2 separate API writes happen: 1. Metadata write: removePolicyWebhooksAndFinalizers calls Update() and persists the finalizer removal. 2. Status write: control returns to reconcile, which calls r.Status().Update(). This persists PolicyStatusScheduled. These are two distinct round-trips to the API server. Between them, an observer (the e2e test polling via Get) can land in any of three states: - Old finalizer + old status (Active): before write #1 lands - No finalizer + old status (Active): between writes #1 and #2 - No finalizer + new status (Scheduled): after write #2 lands The fix Waits until both conditions hold simultaneously, so it cannot observe the intermediate state. Signed-off-by: Víctor Cuadrado Juan <vcuadradojuan@suse.de>
The factories don't add the finalizer, so we need to ensure we fetch the last version of the policy, to check that it indeed has the finalizers now. In the past it was working, because policies had finalizers since creation. Signed-off-by: Víctor Cuadrado Juan <vcuadradojuan@suse.de>
|
Tackled the opened comments, ready for another review. The e2e tests pass locally for me. edit: the e2e tests passed on my fork (same commits) https://github.com/viccuad/adm-controller/actions/runs/26567167995/job/78264737960 |
Description
This changes the controller reconciliation loop to ensure that the finalizer used by the controller will be added in the policy resource only when it is in active state. Therefore, after the
kubewarden-controlleruninstall, the policies can be removed with no issues.In other words, when the policy is in:
Scheduledstatus: no Finalizer (new), no {Validating,Mutating}WebhookConfiguration. Can be safely deleted. Will be garbage-collected if the CRD is removed.Activestatus: has an associated {Validating,Mutating}WebhookConfiguration, must not be garbage-collected away or we break the cluster. Therefore, it also has a Finalizer (as it already had prior to this change).Tests
Beyond the e2e tests,I have a bash script that simulate helm charts uninstall to see if this is working as expected:
Click to see the testing script