Skip to content

[BUG] transfer-pvc: pod-start timeout error from mover pod omits all diagnostic state (pod phase, PVC binding, events) #997

Description

@nachandr

What version of crane are you running, and what are your clutsters+platform
latest on main branch

Summary
When a transfer-pvc mover pod fails to start within the 5-minute pod-start
timeout, crane returns a bare timed out waiting for pod ... to start: context deadline exceeded. The error names the pod but nothing else — not the pod phase,
not whether it was even scheduled, not the binding status or storage class of the
PVC it mounts, and none of the Warning events that state the actual cause. crane
already has read access to all of that; it just never reads it.

Steps to Reproduce

  1. Trigger any condition that leaves a mover pod Pending past the 5-minute
    start timeout. The simplest is a cross-provider indirect transfer without
    --dest-storage-class, which leaves the destination PVC unbindable
  2. Observe the error crane prints when the download pod never reaches Running.

Actual Result

[3/6] Uploading data to cloud storage ... ok
[4/6] Downloading data from cloud storage ...
[6/6] Cleaning up transfer pods ... ok
Error: download pod failed: timed out waiting for pod
indirect-rclone-secret-mysql/rclone-download-mysql-data to start:
context deadline exceeded

Everything needed to diagnose the failure exists in the cluster at that moment and is
readable with the same credentials crane is already using — but none of it is
surfaced.

Expected Result

Before returning the timeout error, crane should gather and include:

  • the pod's phase and the PodScheduled condition,
  • container waiting reasons (e.g. ImagePullBackOff, ContainerCreating),
  • the binding status and storage class of each PVC the pod references,
  • recent Warning events on the pod and on its PVCs,

and, when the state points to a recognizable cause, a hint. Target output:

Error: download pod indirect-rclone-secret-mysql/rclone-download-mysql-data did not
start within 5m0s (phase: Pending).
  PVC indirect-rclone-secret-mysql/mysql-data is Pending (storageClass "gp3-csi").
  Warning events:
    - PVC  ProvisioningFailed: storageclass.storage.k8s.io "gp3-csi" not found
    - Pod  FailedScheduling: 0/3 nodes available: pod has unbound immediate PVCs
Hint: destination has no storage class "gp3-csi". Re-run with
--dest-storage-class <name> (available: standard-csi, ssd-csi).

Please include any relevant logs or errors

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions