kubernetes: Liveness Probe with an invalid command doesn't trigger container restarts and ContainersReady remains True

What happened?

Create a deployment with an invalid liveness probe and you’ll see events like

Liveness probe errored: rpc error: code = Unknown desc = failed to exec in container: failed to start exec “184850d48d723924f62e815706ee39ce8ff8b1d36abbb78552cfe634ccbc1276”: OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: “invalid”: executable file not found in $PATH: unknown

But the container did not restart and ContainersReady status is True

What did you expect to happen?

ContainersReady is False Container is restarted

How can we reproduce it (as minimally and precisely as possible)?

Create a deployment with an invalid liveness probe

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  namespace: default
spec:
  progressDeadlineSeconds: 10
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
        containers:
        - name: liveness
          image: k8s.gcr.io/busybox
          args:
          - /bin/sh
          - -c
          - touch /tmp/healthy; sleep 10; rm -rf /tmp/healthy; sleep 600
          livenessProbe:
            exec:
              command:
                - "invalid"
            initialDelaySeconds: 5
            periodSeconds: 5

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:45:37Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-21T23:01:33Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and and version (if applicable)

Related plugins (CNI, CSI, …) and versions (if applicable)