feat(policy-server): break out dashboard rejection panels by policy_mode#1742
Open
officialasishkumar wants to merge 2 commits into
Open
Conversation
github-project-automation
Bot
moved this to Pending review
in Kubewarden Admission Controller
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Updates the Kubewarden Grafana dashboard to break down rejection/mutation metrics by policy_mode and adds descriptive tooltips so users can distinguish between actually blocked requests (protect mode) and observed-only requests (monitor mode).
Changes:
- Updated PromQL expressions to
sum by (policy_mode) (...)and legend formats to{{policy_mode}}for rejection-related panels. - Renamed several panel titles to include "(by policy mode)" and added
descriptionfields explaining the protect vs. monitor semantics. - Updated the "Accepted requests rate" and "Mutated requests rate" panels with explanatory descriptions.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "editorMode": "code", | ||
| "exemplar": true, | ||
| "expr": "sum(kubewarden_policy_evaluations_total{accepted=\"false\",request_origin=\"validate\"})*100/sum(kubewarden_policy_evaluations_total{})", | ||
| "expr": "sum by (policy_mode) (kubewarden_policy_evaluations_total{accepted=\"false\",request_origin=\"validate\"})*100/sum(kubewarden_policy_evaluations_total{})", |
Author
There was a problem hiding this comment.
Aligned the percentage denominator with validate-origin evaluations grouped by policy_mode in 3a821e5.
| "type": "stat" | ||
| "title": "Request rejection percentage (by policy mode)", | ||
| "type": "stat", | ||
| "description": "Share of all policy evaluations that rejected the admission request, broken down by the policy's mode. policy_mode=protect is the share that was actually blocked; policy_mode=monitor is the share that would have been blocked in protect mode." |
Author
There was a problem hiding this comment.
Updated the description wording to match validate admission evaluations in 3a821e5.
Comment on lines
2693
to
2694
| "title": "Rejected requests by $policy_name policy (by policy mode)", | ||
| "type": "stat" |
Author
There was a problem hiding this comment.
Verified the rejected-requests description is on the intended panel and moved it next to the title/type in 3a821e5.
Member
|
Can you share with us a screenshot of before and after? |
The Kubewarden Grafana dashboard treated every `accepted=false` evaluation as a single concept and could not distinguish between admission requests that were actually blocked (policy in `protect` mode) and admission requests that the cluster still let through but that would have been blocked if the policy was in `protect` mode (policy in `monitor` mode). The `policy_mode` label is already attached to `kubewarden_policy_evaluations_total` in `crates/policy-server/src/metrics.rs`, so the breakdown only needs to be surfaced in the dashboard queries. This commit updates the validate-origin rejection panels so they group by `policy_mode`, and adds explanatory `description` fields that spell out the semantics: - "Rejected admission requests rate (by policy mode)" (kubewarden#66) - "Rejected admission requests (by policy mode)" (kubewarden#40) - "Request rejection percentage (by policy mode)" (kubewarden#48) - "Pod creation rejections (by policy mode)" (kubewarden#8) - "Rejected requests by $policy_name policy (by policy mode)" (kubewarden#32) - "$policy_name policy request rejection percentage (by policy mode)" (kubewarden#56) - "Rate of validate requests to $policy_name policy" (kubewarden#34) also receives the `policy_mode` legend so the per-policy rate panel is aligned with the rejection panels above. The "Accepted requests rate" (kubewarden#67) and "Mutated requests rate" (kubewarden#68) panels keep their existing queries (the `policy_mode` dimension does not change which requests were accepted), but receive descriptions that clarify how the cluster interprets each policy mode. In particular, mutations from monitor-mode policies are reported but not applied. The audit-origin panels are intentionally left untouched: audit scans are background reports against existing resources and do not block anything regardless of policy mode, so adding a `policy_mode` breakdown there would add noise without changing the operational meaning. Closes kubewarden#1284 Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
officialasishkumar
force-pushed
the
issue-1284-grafana-dashboard-policy-mode
branch
from
65ad793 to
3a821e5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1284.
The Kubewarden Grafana dashboard currently aggregates every
accepted=falseevaluationinto a single number, so it cannot distinguish:
policy_mode=protect+accepted=false: the admission request was actually blockedby the cluster.
policy_mode=monitor+accepted=false: the admission request was allowed throughby the cluster, but would have been blocked if the policy was in protect mode.
The
policy_modelabel is already attached tokubewarden_policy_evaluations_totalincrates/policy-server/src/metrics.rs,so the only thing missing was surfacing that dimension in the dashboard.
What changes
The validate-origin rejection panels in
crates/policy-server/kubewarden-dashboard.jsonnowgroup by
policy_modeand carry explanatory descriptions:66Rejected admission requests rate (by policy mode)sum by (policy_mode) (rate(...)), legend{{policy_mode}}, new description40Rejected admission requests (by policy mode)sum by (policy_mode) (...), new description48Request rejection percentage (by policy mode)policy_mode, new description8Pod creation rejections (by policy mode)sum by (policy_mode) (...), new description32Rejected requests by $policy_name policy (by policy mode)policy_mode56$policy_name policy request rejection percentage (by policy mode)policy_mode34Rate of validate requests to $policy_name policypolicy_modeso this rate panel stays aligned with the rejection panels above67Accepted requests rate68Mutated requests rateThe audit-origin panels (
#96,#85,#89,#92,#75,#78,#70) are intentionallyleft untouched. Audit scans are background reports against existing resources and do not
block anything regardless of policy mode, so adding a
policy_modebreakdown would addnoise without changing the operational meaning.
No template variables, panel ids, or
gridPosvalues are touched, so the dashboardre-imports cleanly without re-laying-out anything.
Test plan
jq -e . crates/policy-server/kubewarden-dashboard.jsonparses successfullyschemaVersionandversionfields are unchangedgridPosvalues touchedpolicy_modeseries on thetime-series rejection panel, the two values side-by-side on the stat panels, and the
description tooltip on each updated panel