verify-helm-chart.sh CRD synchronization check diffs the two checked-in CRD copies against each other, never against a fresh generate

[jordigilh]

Summary

The "CRD synchronization" check in scripts/verify-helm-chart.sh (lines 75-84) only
diffs deploy/crds/*.yaml against charts/grid-operator/crds/*.yaml — the two
checked-in copies compared against each other. It never regenerates CRDs from
the current #[derive(CustomResource)] structs (via generate_crds /
scripts/generate-deployment-crds.sh) and diffs that fresh output against either
checked-in copy.

This is structurally incapable of catching a contributor who edits a CRD struct
(e.g. adds/renames/retypes a field) and forgets to run the regeneration step: both
checked-in files would remain byte-identical to each other (both stale in the same
way) and the check would still pass. All 3 CRDs happen to be fresh today only
because a human remembered to regenerate correctly last time — not because CI
enforces it.

Evidence

75  # ── CRD synchronization ─────────────────────────────────────────────
76  echo ""
77  echo "=== CRD synchronization ==="
78  for crd in gridnetwork gridsite inferenceprovider; do
79    if diff -q "$DEPLOY_CRDS/${crd}.yaml" "$CHART_DIR/crds/${crd}.yaml" >/dev/null 2>&1; then
80      pass "crd sync: ${crd}.yaml"
81    else
82      fail "crd sync: ${crd}.yaml differs from $DEPLOY_CRDS/${crd}.yaml"
83    fi
84  done

(scripts/verify-helm-chart.sh:75-84, verified against fresh upstream/main.)

DEPLOY_CRDS="deploy/crds" and CHART_DIR="charts/grid-operator" (lines 59-60) —
both are static, checked-in files. The actual CRD-generation entry point,
scripts/generate-deployment-crds.sh, is never invoked by this check (confirmed via
repo-wide search — it's only referenced from xtask/src/env/operator.rs and
xtask/src/env/mod.rs, unrelated to verify-helm-chart.sh).

Risk

This is the same defect class ("checked-in copies validated only against each
other, not against source-of-truth regeneration") that produced earlier CRD
staleness bugs found in prior reviews this session. Because both checked-in copies
can drift identically whenever a contributor forgets to regenerate after touching a
#[derive(CustomResource)] struct, this CI gate provides false confidence: it will
report PASS on a genuinely stale CRD pair, silently shipping a schema mismatch
between what the Kubernetes API server accepts and what the operator's Rust types
expect.

Suggested fix

Add a third comparison point: run scripts/generate-deployment-crds.sh (or the
equivalent generate_crds codegen path) into a temp directory and diff its output
against deploy/crds/*.yaml (and/or charts/grid-operator/crds/*.yaml) in addition
to the existing checked-in-vs-checked-in diff, failing the check if the freshly
generated CRD differs from either checked-in copy.

Severity: High