kubernetes: kubectl wait works for Deployment, but does not for StatefulSet

What happened:

 kubectl wait -f schema-registry.yaml --for condition=available

works for Deployment, but it does not work for StatefulSet

What you expected to happen: Expected that kubectl wait works for StatefulSet

kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.2", GitCommit:"cff46ab41ff0bb44d8584413b598ad8360ec1def", GitTreeState:"clean", BuildDate:"2019-01-13T23:15:13Z", GoVersion:"go1.11.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:"2019-06-06T01:36:19Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 40
  • Comments: 51 (11 by maintainers)

Most upvoted comments

For everybody coming here, I suggest you try to use the rollout status command like this:

kubectl rollout status statefulset/name-of-statefulset

It is not exactly the same thing, but it might solve your use-case for this feature.

For everybody coming here, I suggest you try to use the rollout status command like this:

If you want to wait until a statefulset rollout is finished, you can do the following:

kubectl rollout status --watch --timeout=600s statefulset/name-of-statefulset

Hey, I just wanted to share something that may be of use - instead of checking a condition on the StatefulSet, it might be easier to look for the same condition on the pods.

With a StatefulSet:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: my-custom-sts
spec:
  selector:
    matchLabels:
      app: custom-sts
  template:
    metadata:
      labels:
        app: custom-sts

I can do a

kubectl wait pods --selector app=custom-sts --for condition=Ready
pod/my-custom-sts-0 condition met

and achieve the same result.

In https://github.com/kubernetes/kubernetes/issues/51594#issuecomment-334646068 @kow3ns wrote:

Add Conditions to StatefulSet and DeamonSet. The StatefulSet and DeamonSet controller will not use Conditions at the time of GA release, but they may do so in the future.

It looks like statefulset API supports conditions, but the statefulset controller doesn’t set them…at least it didn’t at time of writing: https://github.com/kubernetes/kubernetes/blob/v1.19.0-alpha.0/staging/src/k8s.io/api/apps/v1/types.go#L219

Is rollout status replacing wait generally, or is there a timeline for StatefulsetConditions to be updated properly?

Can confirm what @wallrj wrote: According to the docs, conditions are supported:

$ kubectl explain statefulset.status.conditions                                                                                                                                                                                
KIND:     StatefulSet
VERSION:  apps/v1

RESOURCE: conditions <[]Object>

DESCRIPTION:
     Represents the latest available observations of a statefulset's current
     state.

     StatefulSetCondition describes the state of a statefulset at a certain
     point.

FIELDS:
   lastTransitionTime	<string>
     Last time the condition transitioned from one status to another.

   message	<string>
     A human readable message indicating details about the transition.

   reason	<string>
     The reason for the condition's last transition.

   status	<string> -required-
     Status of the condition, one of True, False, Unknown.

   type	<string> -required-

But the condition field is never actually set.

Version:

$ kubectl version                                                                                                                                                                                                              
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.5", GitCommit:"20c265fef0741dd71a66480e35bd69f18351daea", GitTreeState:"clean", BuildDate:"2019-10-15T19:16:51Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.0", GitCommit:"70132b0f130acc0bed193d9ba59dd186f0e634cf", GitTreeState:"clean", BuildDate:"2020-01-27T21:11:27Z", GoVersion:"go1.13.4", Compiler:"gc", Platform:"linux/amd64"}

That is really confusing.

just ran into this myself; here’s a workaround; it’s not sexy but it totally works.

Output from a run just iterates from the replica count:

Wait for all replicas in the StatefulSet...
+ (( replica = 0 ))
+ (( replica < 3 ))
+ pMsg '  Waiting for zookeepercluster-0...'
+ theMessage='  Waiting for zookeepercluster-0...'
+ printf '\n%s\n' '  Waiting for zookeepercluster-0...'

  Waiting for zookeepercluster-0...
+ kubectl wait --namespace=zookeeper --for=condition=ready pod --timeout=60s -l statefulset.kubernetes.io/pod-name=zookeepercluster-0
pod/zookeepercluster-0 condition met
+ (( replica++  ))
+ (( replica < 3 ))
+ pMsg '  Waiting for zookeepercluster-1...'
+ theMessage='  Waiting for zookeepercluster-1...'
+ printf '\n%s\n' '  Waiting for zookeepercluster-1...'

  Waiting for zookeepercluster-1...
+ kubectl wait --namespace=zookeeper --for=condition=ready pod --timeout=60s -l statefulset.kubernetes.io/pod-name=zookeepercluster-1
pod/zookeepercluster-1 condition met
+ (( replica++  ))
+ (( replica < 3 ))
+ pMsg '  Waiting for zookeepercluster-2...'
+ theMessage='  Waiting for zookeepercluster-2...'
+ printf '\n%s\n' '  Waiting for zookeepercluster-2...'

  Waiting for zookeepercluster-2...
+ kubectl wait --namespace=zookeeper --for=condition=ready pod --timeout=60s -l statefulset.kubernetes.io/pod-name=zookeepercluster-2
pod/zookeepercluster-2 condition met
+ (( replica++  ))
+ (( replica < 3 ))
+ exit 0

And, there are only 5 variables to keep track of; easy.

@javajon You mentioned me by mistake, instead of @seeker25