snapshotctl is a lower-level utility for checkpointing and restoring a pod
directly from a pod manifest. It is not the primary path — most users drive
Snapshot through the Kubernetes resources — but it is handy
for validation and debugging, and it is a quick way to try checkpoint/restore by
hand.
- The Snapshot Helm chart is installed in the target namespace, with the
snapshot-agentDaemonSet running and the checkpoint PVC mounted. checkpointrequires the operator (it resolves thePodSnapshotinto a checkpoint).restoreis handled by the agent directly from pod annotations.
snapshotctl checkpoint creates a PodSnapshot from a pod manifest and waits for
the agent to checkpoint it:
snapshotctl checkpoint \
--manifest ./vllm-replica-pod.yaml \
--snapshot vllm-snapshot \
--container main \
--namespace my-inferenceThe manifest must be a Pod (not a Deployment or Job) using a
snapshot-ready workload — snapshotctl rejects any
other kind. The framework guides ship deployment.yaml, not a standalone
Pod; wrap its spec.template.spec in a Pod manifest to use it here:
apiVersion: v1
kind: Pod
metadata:
name: vllm-replica-pod
spec:
# deployment.yaml's spec.template.spec, unchanged
...snapshotctl restore creates a new pod from a manifest and restores it from a
named PodSnapshot:
snapshotctl restore \
--manifest ./vllm-replica-pod.yaml \
--snapshot vllm-snapshot \
--namespace my-inferenceThe restore manifest must contain a container with the same name checkpointed by that
PodSnapshot. snapshotctl returns once the restore is submitted — watch the
pod's nvidia.com/Restored status condition, readiness, and events for progress.
The source README for the tool lives at
operator/cmd/snapshotctl/README.md.