Add and validate GB300 recipe overlays

[yuanchen8911]

Goal

Add concrete service-bound recipe overlays for GB300 so aicr recipe --accelerator gb300 --service <y> --intent <z> resolves a usable recipe end-to-end, with no GB300 coverage today.

GB300 is currently undeclared in pkg/recipe/criteria.go (declared accelerators: a100, b200, gb200, h100, h200, l40, rtx-pro-6000) and has zero overlays under recipes/overlays/. ROADMAP.md frames GB300 as a near-horizon target.

Scope

1. Declare CriteriaAcceleratorGB300 in pkg/recipe/criteria.go and wire ParseCriteriaAcceleratorType + GetCriteriaAcceleratorTypes.
2. Add a gb300-any.yaml wildcard stub plus concrete service-bound overlays for the primary deployment shapes (training and inference on the major hyperscalers), modeled on the existing GB200 overlays (gb200-eks-*, gb200-oke-*).
3. Reference the new overlays in recipes/overlays_validation_floor_test.go (or record an exemption).

Success criteria (per parent epic #1042)

1. recipes/overlays/ contains at least one concrete service-bound overlay (non-wildcard) for GB300's primary deployment intent.
2. aicr recipe --accelerator gb300 --service <y> --intent <z> resolves a recipe without manual overlay authoring.
3. The overlay is covered by recipes/overlays_validation_floor_test.go.

Sub-issue of #1042 (Recipe overlay coverage). Tuning-profile reuse from GB200 should be considered where applicable.

[yuanchen8911]

PR #1319 implements this — declares CriteriaAcceleratorGB300 in pkg/recipe/criteria.go and adds the GB300 EKS overlay set (gb300-any + training/inference, ubuntu variants, kubeflow, dynamo), modeled on gb200-eks-*.

GB300 shares GB200's NVL72 topology and gpu-operator v25.10 floor, so the recipe mirrors GB200 EKS; nodewright reuses the GB200 tuning profile (per the nodewright maintainer, with a DOCA/AWS-AMI caveat noted in-file). Currently draft/WIP.

Other GB300 services (OKE/AKS/GKE) remain follow-ups under the parent epic #1042.

[yuanchen8911]

Follow-up: GB300 needs a CUDA 13 runtime image (sm_103)

While validating the merged GB300 EKS inference-dynamo overlay on a live GB300
cluster, the inference workload crash-looped on a GPU-architecture / runtime-image
issue (independent of the recipe structure itself):

• GB300 is Blackwell Ultra, compute capability sm_103 (gpu.compute.major/minor = 10.3).
• The Dynamo vllm-runtime:1.0.2 image (and 1.2.0) are CUDA 12.9 builds — their
  flashinfer kernels do not include sm_103. The vLLM worker reaches the
  sampling stage and then crashes:

  flashinfer .../renorm.cu  top_k_mask_logits
  RuntimeError: TopKMaskLogits failed ... no kernel image is available for execution on the device
  

  → Engine core initialization failed → worker CrashLoopBackOff.
• Fix: use a -cuda13 runtime build (CUDA 13.0, which supports Blackwell Ultra
  sm_103), e.g. nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.0.2-cuda13 — same
  Dynamo version as the operator (v1alpha1 DGD, no control-plane skew), multi-arch
  (amd64 + arm64 for Grace). Per the Dynamo release-artifacts doc, the 1.x line
  has both CUDA 12.9 (default) and CUDA 13.0 (-cuda13) variants; CUDA 12 images are
  discontinued starting v1.3.0.

Implication for the GB300 recipe: the inference-dynamo overlay's worker runtime
image must be a CUDA 13 build on GB300; the cuda12 default won't serve. This is a
runtime-image-pin consideration for the recipe, not an overlay-structure bug.

Scope / related:

• A CUDA-13 build is backward-compatible at the toolkit level with H100 (sm_90),
  GB200 (sm_100), and GB300 (sm_103); RTX Pro 6000 (sm_120) coverage in a given
  image should be verified. So "default the runtime image to cuda13" is a broader,
  fleet-wide decision worth tracking separately.
• Distinct from the GPU-allocation findings (Allocate GPUs via device plugin, not DRA: align with driver default (DRA = ComputeDomain only) #1327, device-plugin vs DRA) — this is
  purely about the runtime image's compiled kernel set.

(Verified on a live GB300 EKS cluster: with vllm-runtime:1.0.2-cuda13 the worker
gets past the flashinfer sampler; run in progress.)