feat: node scan#944
Conversation
537d68d to
978f529
Compare
2a11260 to
ebd018a
Compare
baa6561 to
bd5decc
Compare
04ed271 to
a3562ae
Compare
a3562ae to
9aff7c5
Compare
9aff7c5 to
ea1a045
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #944 +/- ##
==========================================
- Coverage 53.52% 49.85% -3.67%
==========================================
Files 61 75 +14
Lines 5323 6420 +1097
==========================================
+ Hits 2849 3201 +352
- Misses 2078 2761 +683
- Partials 396 458 +62
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
560219e to
7968ddf
Compare
There was a problem hiding this comment.
Pull request overview
Adds the “node scan” feature to sbomscanner: a singleton NodeScanConfiguration drives node selection/platform filtering and dispatches per-node work via NATS to a DaemonSet-based worker running in “node” mode. The scan results are stored as new cluster-scoped storage resources (NodeSBOM, NodeVulnerabilityReport) backed by the storage API server.
Changes:
- Introduces NodeScan CRDs (
NodeScanConfiguration,NodeScanJob), controllers (runner + reconcilers), and a validating webhook. - Adds storage-layer support for node SBOMs and vulnerability reports (CRDs, OpenAPI, client/informers/listers, apiserver stores, DB migrations).
- Extends the worker to support
-mode=nodewith a Helm DaemonSet and adds e2e coverage for the node scan flow.
Reviewed changes
Copilot reviewed 69 out of 91 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
| test/e2e/nodescan_test.go | New e2e scenario covering node scan creation, completion, and cleanup. |
| test/e2e/main_test.go | Adjusts pod security labels for e2e namespace to allow node worker requirements. |
| test/crd/storage.sbomscanner.kubewarden.io_nodevulnerabilityreports.yaml | Test CRD manifest for NodeVulnerabilityReport. |
| test/crd/storage.sbomscanner.kubewarden.io_nodesboms.yaml | Test CRD manifest for NodeSBOM. |
| pkg/generated/openapi/zz_generated.openapi.go | Generated OpenAPI definitions for new node storage types. |
| pkg/generated/listers/storage/v1alpha1/nodevulnerabilityreport.go | Generated lister for NodeVulnerabilityReport. |
| pkg/generated/listers/storage/v1alpha1/nodesbom.go | Generated lister for NodeSBOM. |
| pkg/generated/listers/storage/v1alpha1/expansion_generated.go | Adds lister expansion hooks for new listers. |
| pkg/generated/informers/externalversions/storage/v1alpha1/nodevulnerabilityreport.go | Generated informer for NodeVulnerabilityReport. |
| pkg/generated/informers/externalversions/storage/v1alpha1/nodesbom.go | Generated informer for NodeSBOM. |
| pkg/generated/informers/externalversions/storage/v1alpha1/interface.go | Wires node informers into the storage informer interface. |
| pkg/generated/informers/externalversions/generic.go | Adds generic informer mapping for node storage resources. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/storage_client.go | Adds typed getters for node storage resources. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/nodevulnerabilityreport.go | Generated typed client for NodeVulnerabilityReport. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/nodesbom.go | Generated typed client for NodeSBOM. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/generated_expansion.go | Adds client expansion interfaces for node resources. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/fake/fake_storage_client.go | Adds fake typed client getters for node resources. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/fake/fake_nodevulnerabilityreport.go | Fake typed client for NodeVulnerabilityReport. |
| pkg/generated/clientset/versioned/typed/storage/v1alpha1/fake/fake_nodesbom.go | Fake typed client for NodeSBOM. |
| pkg/generated/applyconfiguration/utils.go | Adds applyconfiguration mappings for node storage kinds. |
| pkg/generated/applyconfiguration/storage/v1alpha1/nodevulnerabilityreport.go | Generated applyconfiguration for NodeVulnerabilityReport. |
| pkg/generated/applyconfiguration/storage/v1alpha1/nodesbom.go | Generated applyconfiguration for NodeSBOM. |
| pkg/generated/applyconfiguration/storage/v1alpha1/nodemetadata.go | Generated applyconfiguration for NodeMetadata. |
| Makefile | Includes new internal package in worker build inputs. |
| internal/webhook/v1alpha1/nodescanconfiguration_webhook.go | Validating webhook for NodeScanConfiguration (interval/platform/selector validation). |
| internal/webhook/v1alpha1/nodescanconfiguration_webhook_test.go | Unit tests for NodeScanConfiguration webhook validation. |
| internal/storage/watcher.go | Adjusts watcher rehydration keying for cluster-scoped resources. |
| internal/storage/transform.go | Adds cache transforms for NodeSBOM / NodeVulnerabilityReport size reduction. |
| internal/storage/store.go | Adds cluster-scoped key parsing support in generic store. |
| internal/storage/repository/cluster_scoped_object_repository.go | New repository implementation for cluster-scoped objects (no namespace column). |
| internal/storage/nodevulnerabilityreport_strategy.go | Storage strategy for NodeVulnerabilityReport. |
| internal/storage/nodevulnerabilityreport_store.go | Storage registry/store for NodeVulnerabilityReport with NATS watch integration. |
| internal/storage/node_sbom_strategy.go | Storage strategy for NodeSBOM. |
| internal/storage/node_sbom_store.go | Storage registry/store for NodeSBOM with NATS watch integration. |
| internal/storage/migrations.go | Adds DB migrations for node SBOM/report tables. |
| internal/storage/matcher.go | Minor selector/matcher comment tweak. |
| internal/skippatterns/skippatterns.go | Parses skipPatterns into trivy skip-dirs vs skip-files. |
| internal/skippatterns/skippatterns_test.go | Unit tests for skipPatterns parsing. |
| internal/skippatterns/doc.go | Package docs for skipPatterns. |
| internal/messaging/subscriber.go | Renames handler registry map type and wires into subscriber. |
| internal/messaging/subscriber_test.go | Updates tests for handler map type rename. |
| internal/handlers/scan_sbom_helpers.go | Shared trivy scan helpers used by image and node scan handlers. |
| internal/handlers/nodescanjob_failure.go | Failure handler to mark NodeScanJobs failed on message failure. |
| internal/handlers/node_scan_sbom.go | Node SBOM vulnerability scanning handler creating NodeVulnerabilityReport. |
| internal/handlers/messages.go | Adds node scan NATS subjects and message types. |
| internal/handlers/image_scan_sbom.go | Refactors image SBOM scan handler into a dedicated file. |
| internal/handlers/generate_sbom.go | Minor trivy args lint annotations. |
| internal/handlers/create_catalog.go | Uses GetUID() when forming message IDs. |
| internal/handlers/create_catalog_test.go | Updates tests to match GetUID() usage. |
| internal/controller/nodescanjob_controller.go | Controller that publishes per-node SBOM generation messages for NodeScanJobs. |
| internal/controller/nodescanjob_controller_test.go | Tests for NodeScanJob controller publish + deletion behavior. |
| internal/controller/nodescan_runner.go | Periodic runner that creates NodeScanJobs per scan interval and filters. |
| internal/controller/nodescan_runner_test.go | Tests for NodeScanRunner scan interval/platform behaviors. |
| internal/controller/nodescan_controller.go | Reconciles singleton NodeScanConfiguration to create/GC NodeScanJobs and NodeSBOMs. |
| internal/controller/indexer.go | Adds field indexer for NodeScanJob.spec.nodeName. |
| internal/apiserver/storage.go | Registers new node storage resources/stores in storage API server. |
| examples/nodescanjob.yaml | Example NodeScanJob manifest. |
| examples/nodescanconfiguration.yaml | Example NodeScanConfiguration manifest. |
| examples/nodesbom.yaml | Example NodeSBOM manifest. |
| docs/crds/CRD-docs-for-docs-repo.md | Updates CRD docs to include node scan types and node storage types. |
| cmd/worker/main.go | Adds worker -mode switch and node mode subscriber subjects + daemonset support. |
| cmd/controller/main.go | Adds flag and wiring for node scan controllers and webhook setup. |
| charts/sbomscanner/templates/worker/registry-serviceaccount.yaml | Adds dedicated SA for registry worker. |
| charts/sbomscanner/templates/worker/registry-rolebinding.yaml | Adds dedicated rolebinding for registry worker. |
| charts/sbomscanner/templates/worker/registry-role.yaml | Renames/splits registry worker ClusterRole. |
| charts/sbomscanner/templates/worker/node-serviceaccount.yaml | Renames/splits node worker ServiceAccount. |
| charts/sbomscanner/templates/worker/node-rolebinding.yaml | Renames/splits node worker ClusterRoleBinding. |
| charts/sbomscanner/templates/worker/node-role.yaml | Adds node worker ClusterRole permissions (node scan resources + storage). |
| charts/sbomscanner/templates/worker/deployment.yaml | Registry worker deployment now uses registry SA and explicit -mode=registry. |
| charts/sbomscanner/templates/worker/daemonset.yaml | Adds node worker DaemonSet mounting host filesystem and running -mode=node. |
| charts/sbomscanner/templates/crd/sbomscanner.kubewarden.io_nodescanjobs.yaml | Helm-installed NodeScanJob CRD. |
| charts/sbomscanner/templates/crd/sbomscanner.kubewarden.io_nodescanconfigurations.yaml | Helm-installed NodeScanConfiguration CRD. |
| charts/sbomscanner/templates/controller/webhooks.yaml | Registers validating webhook for NodeScanConfiguration. |
| charts/sbomscanner/templates/controller/role.yaml | Extends controller permissions to support node scan resources. |
| api/v1alpha1/zz_generated.deepcopy.go | Generated deep-copies for new v1alpha1 node scan types. |
| api/v1alpha1/nodescanconfiguration_types.go | Defines NodeScanConfiguration API type (singleton). |
| api/v1alpha1/node_scanjob_types.go | Defines NodeScanJob API type and status helpers. |
| api/storage/v1alpha1/zz_generated.model_name.go | Generated OpenAPI model names for new node storage types. |
| api/storage/v1alpha1/zz_generated.deepcopy.go | Generated deep-copies for new node storage types. |
| api/storage/v1alpha1/register.go | Registers node storage kinds + field selector conversion for node metadata. |
| api/storage/v1alpha1/node_vulnerabilityreport_types.go | Defines NodeVulnerabilityReport API type. |
| api/storage/v1alpha1/node_sbom_types.go | Defines NodeSBOM API type. |
| api/storage/v1alpha1/node_metadata.go | Defines NodeMetadata and related accessor/index constants. |
| api/storage/register.go | Wires node storage types into the top-level storage scheme registration. |
| api/labels.go | Adds labels for node scan managed resources. |
Files not reviewed (3)
- api/storage/v1alpha1/zz_generated.deepcopy.go: Language not supported
- api/storage/v1alpha1/zz_generated.model_name.go: Language not supported
- api/v1alpha1/zz_generated.deepcopy.go: Language not supported
Comments suppressed due to low confidence (1)
examples/nodescanjob.yaml:6
- This example manifest is invalid as-is:
specis empty butNodeScanJobSpec.nodeNameis required by the CRD. Add anodeNamefield (or remove the example until it’s complete) so users can apply it successfully.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| scanJobList := &v1alpha1.NodeScanJobList{} | ||
|
|
||
| if err := r.List(ctx, scanJobList); err != nil { | ||
| return fmt.Errorf("failed to list NodeScanJobs: %w", err) | ||
| } |
There was a problem hiding this comment.
Copilot is right. The retention should be per Node, to mimic the ScanJob retention.
| Assess("Delete NodeScanConfiguration and verify cleanup", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context { | ||
| nodeScanConfig := &v1alpha1.NodeScanConfiguration{ | ||
| ObjectMeta: metav1.ObjectMeta{ | ||
| Name: v1alpha1.NodeScanConfigurationName, | ||
| }, | ||
| } | ||
| err := cfg.Client().Resources().Delete(ctx, nodeScanConfig) | ||
| require.NoError(t, err, "failed to delete NodeScanConfiguration") | ||
|
|
||
| err = wait.For(func(ctx context.Context) (bool, error) { | ||
| jobs := &v1alpha1.NodeScanJobList{} | ||
| if err := cfg.Client().Resources().List(ctx, jobs, | ||
| resources.WithLabelSelector(nodeScanLabelSelector), | ||
| ); err != nil { | ||
| return false, err | ||
| } | ||
| return len(jobs.Items) == 0, nil | ||
| }, wait.WithTimeout(scanTimeout)) | ||
| require.NoError(t, err, "NodeScanJobs should be cleaned up after deleting NodeScanConfiguration") | ||
|
|
||
| err = wait.For(func(ctx context.Context) (bool, error) { | ||
| sboms := &storagev1alpha1.NodeSBOMList{} | ||
| if err := cfg.Client().Resources().List(ctx, sboms, | ||
| resources.WithLabelSelector(labels.FormatLabels(map[string]string{ | ||
| api.LabelManagedByKey: api.LabelManagedByValue, | ||
| }))); err != nil { | ||
| return false, err | ||
| } | ||
| return len(sboms.Items) == 0, nil | ||
| }, wait.WithTimeout(scanTimeout)) | ||
| require.NoError(t, err, "NodeSBOMs should be cleaned up after deleting NodeScanConfiguration") | ||
|
|
| listOpts = append(listOpts, client.MatchingLabelsSelector{Selector: selector}) | ||
| } | ||
|
|
||
| if err := r.List(ctx, &nodeList, listOpts...); err != nil { |
There was a problem hiding this comment.
Using a platform indexer, we could add the filtering here.
| func (r *NodeScanRunner) checkNodeForScan(ctx context.Context, config *v1alpha1.NodeScanConfiguration, node *corev1.Node) error { | ||
| log := log.FromContext(ctx) | ||
|
|
||
| if !filters.IsPlatformAllowed( |
There was a problem hiding this comment.
see comment about platform indexer above
Signed-off-by: Alessio Greggi <alessio.greggi@suse.com>
Signed-off-by: Alessio Greggi <alessio.greggi@suse.com>
Signed-off-by: Alessio Greggi <alessio.greggi@suse.com>
Signed-off-by: Alessio Greggi <alessio.greggi@suse.com>
Signed-off-by: Alessio Greggi <alessio.greggi@suse.com>
f0f03e5 to
4e10530
Compare
Description
This PR adds the ability to scan cluster node filesystems for SBOMs and vulnerability reports.
A new
NodeScanConfigurationCRD lets users configure which nodes to scan (via label selectors and platformfilters), skip patterns for trivy, and scan intervals.
The controller creates
NodeScanJobresources per matching node and dispatches work via NATS to a new DaemonSet-based worker running in node mode.Closes #1154
Test
To test this pull request, you can run the following commands:
Manual testing
Expected: one
NodeScanJobper cluster node that matches the platform filter and node selector. Each job's spec.nodeName should correspond to an actual node.Watch for the status.completionTime field to be set.
Expected: each job should reach a Complete condition with status True.
Expected: one
NodeSBOMper scanned node, containing SPDX data. Verify it has content:Should show the node name and platform (e.g.
linux/amd64).Expected: one report per scanned node, with a vulnerability summary:
Then verify all managed resources are cleaned up:
Expected: both should return No resources found.
Recreate the config with a selector that matches no nodes:
Expected: no
NodeScanJobsshould be created.Additional Information
Tradeoff
Potential improvement
TODO
NodeScanConfiguration,NodeScanJob)NodeSBOM,NodeVulnerabilityReport) and implement storage logic for themNodeScanJobstatusesscanIntervallogic forNodeScanConfigurationnodeSelectorlogic forNodeScanConfigurationskipPatternslogic forNodeScanConfigurationplatformsfilter logic forNodeScanConfigurationNodeScanlogicChecklist