Add upgradePolicy to NVIDIADriver CRD

[cdesiniotis]

The upgradePolicy influences how the driver-upgrade controller upgrades GPU driver daemonsets. Adding this field to the NVIDIADriver CRD allows users to define different upgrade policies for different NVIDIADriver CRs. If nil or empty, the driver-upgrade controller will fallback to using a default upgradePolicy defined in the code which aligns with the defaults in our helm values.

Code changes in this PR were drafted with the assistance of Claude Code.

Testing

I tested the following scenarios on a k8s cluster with two GPUs:

1. Install gpu operator with default options. Verify entire stack comes up.
2. Upgrade ClusterPolicy-managed driver daemonset. Verify upgrade controller uprades driver pods in rolling fashion based on configured upgrade policy.
3. Migrate to NVIDIADriver-managed driver daemonset by a) setting driver.useNvidiaDriverCRD=true in ClusterPolicy; b) creating a default NVIDIADriver CR. Verify ClusterPolicy-managed pods gets orphaned then deleted by upgrade controller in a rolling fashion.
4. Upgrade the default NVIDIADriver-managed driver daemonset by changing spec.version. Verify upgrade controller upgrades pods in a rolling fashion.
5. Repeat step 4 with a different upgradePolicy configured in the NVIDIADriver CR. Verify upgrade controller upgrades pods according to the new upgrade policy.

[rahulait]

LGTM. I am wondering about backward compatibility. Existing user-defined nvidiadriver CRs won't have upgrade configuration, and previously they were defaulting to the config in clusterpolicy. If custom upgrade config was used in clusterpolicy previously and user specified nvidiadriver was using it and now user upgrades to newer version, nvidiadriver will start using default config we specify. Would we be documenting this as a breaking change or something for users to be aware of when they are using nvidiadrivers and they jump to v26.7.0?

[rajathagasthya]

Just a couple of minor comments. Overall LGTM.

[cdesiniotis]

LGTM. I am wondering about backward compatibility. Existing user-defined nvidiadriver CRs won't have upgrade configuration, and previously they were defaulting to the config in clusterpolicy. If custom upgrade config was used in clusterpolicy previously and user specified nvidiadriver was using it and now user upgrades to newer version, nvidiadriver will start using default config we specify. Would we be documenting this as a breaking change or something for users to be aware of when they are using nvidiadrivers and they jump to v26.7.0?

Good observation. If users are depending on a custom upgradePolicy from ClusterPolicy, then yes, this would be a breaking change for user-created NVIDIADriver CRs. We will need to call this out in our release notes.