kubernetes: Mounting/unmounting volume error on GKE

Greetings all. I’m running my cluster on the GKE and after upgrading to 1.3.3 I’ve been having errors of the following format:

Events: FirstSeen LastSeen Count From SubobjectPath Type Reason Message

8m 4m 20 {default-scheduler } Warning FailedScheduling no nodes available to schedule pods 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned mongodb-321322295-mnmo7 to gke-test-cluster-default-pool-f56cf9ee-f2kb 1m 1m 1 {kubelet gke-test-cluster-default-pool-f56cf9ee-f2kb} Warning FailedMount Unable to mount volumes for pod “mongodb-321322295-mnmo7_dev(20c985df-5882-11e6-ac8b-42010af00014)”: timeout expired waiting for volumes to attach/mount for pod “mongodb-321322295-mnmo7”/“dev”. list of unattached/unmounted volumes=[mongodb-data] 1m 1m 1 {kubelet gke-test-cluster-default-pool-f56cf9ee-f2kb} Warning FailedSync Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod “mongodb-321322295-mnmo7”/“dev”. list of unattached/unmounted volumes=[mongodb-data]

Evidently, that’s not something specific to mongodb-data volume. I get these same errors for other volumes too (majority of them are secrets). I’m anticipating the possibility of upgrading to 1.3.4 release on GKE as I’ve read that it possibly resolves the problem but it’s not available yet.

But for now I’d like to know if there’re some workarounds/hacks for the problem that could allow me to continue deploying my services?

Also, huge thanks to the whole Kubernetes community. You’re awesome 🖖🏻

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 1
  • Comments: 57 (24 by maintainers)

Most upvoted comments

Thanks for confirming, @jingxu97 We’re on Google Container Engine so we have to wait until they approve v1.3.4 there.

#34859 is already merged to the master. It should be in 1.4.6 branch. Please kindly let me know if you continue experience any issues related to mounts. Thank you!

@mesmerizingr: false alarm: it was still happening today … only the downgrade to 1.2.6 really fixes it … thx for the hint 👍

@mwuertinger @mesmerizingr, thank you for reporting your issue. Yes, we recently fixed a few issues for mounting/umounting volumes in v1.3.3. The fix is already in v1.3.4. Please upgrade to v1.3.4 and kindly let us know if you still have problems.

Jing

On Thu, Aug 4, 2016 at 7:46 AM, Roman notifications@github.com wrote:

@mwuertinger https://github.com/mwuertinger take a look on this thread: #28750 https://github.com/kubernetes/kubernetes/issues/28750

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kubernetes/kubernetes/issues/29903#issuecomment-237575351, or mute the thread https://github.com/notifications/unsubscribe-auth/ASSNxSOTF3WlyjFGuOv-zrLj-wNuMwEeks5qcfs1gaJpZM4JaVMN .

  • Jing