kubernetes: Pet Set stuck in ContainerCreating

A previously healthy pod was running. It got unhealthy for some reason, and I did a silly thing. Deleted the pod. Now it is stuck:

Volumes:
  cassandra-analytics:
    Type:   PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  cassandra-analytics-cassandra-analytics-112
    ReadOnly:   false
  default-token-mu6y5:
    Type:   Secret (a volume populated by a Secret)
    SecretName: default-token-mu6y5
QoS Tier:   Guaranteed
Events:
  FirstSeen LastSeen    Count   From                    SubobjectPath   Type        Reason      Message
  --------- --------    -----   ----                    -------------   --------    ------      -------
  14d       22s     9100    {kubelet kubernetes-minion-group-0jg9}          Warning     FailedMount Unable to mount volumes for pod "cassandra-analytics-112_default(13171b3b-3a50-11e6-b7d0-42010a800002)": timeout expired waiting for volumes to attach/mount for pod "cassandra-analytics-112"/"default". list of unattached/unmounted volumes=[cassandra-analytics]
  14d       22s     9100    {kubelet kubernetes-minion-group-0jg9}          Warning     FailedSync  Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "cassandra-analytics-112"/"default". list of unattached/unmounted volumes=[cassandra-analytics]

Can we document the process of getting this unstuck? I am guessing this is an error, because the PV has data in it. Here is the PV.

$ kubectl describe pv pvc-a5f45f53-38f0-11e6-b7d0-42010a800002
Name:       pvc-a5f45f53-38f0-11e6-b7d0-42010a800002
Labels:     failure-domain.beta.kubernetes.io/region=us-central1
        failure-domain.beta.kubernetes.io/zone=us-central1-c
Status:     Bound
Claim:      default/cassandra-analytics-cassandra-analytics-112
Reclaim Policy: Delete
Access Modes:   RWO
Capacity:   380Gi
Message:
Source:
    Type:   GCEPersistentDisk (a Persistent Disk resource in Google Compute Engine)
    PDName: kubernetes-dynamic-pvc-a5f45f53-38f0-11e6-b7d0-42010a800002
    FSType:
    Partition:  0
    ReadOnly:   false
No events.

FYI I am on 1.3 beta without the last minute volume fix stuff. I can try to recreate with the new volume code, but I don’t think it will make a difference.

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 1
  • Comments: 26 (16 by maintainers)

Most upvoted comments