kubernetes: EventRateLimit - Admission Control - Fails to start kube-apiserver

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened: When EventRateLimit has included in Admission Control, kubelet is unable to start kube-apiserver pod. What you expected to happen: kube-apiserver to start. How to reproduce it (as minimally and precisely as possible): setup kubernetes master using kubeadm. Apply weave-net cni Once the master is ready, stop the kubelet. Make changes to kube-apiserver.yaml as below to include EventRateLimit in admission control, - --runtime-config=apiserver.k8s.io/v1alpha1,eventratelimit.admission.k8s.io/v1alpha1=true

apiVersion: v1
kind: Pod
metadata:
  annotations:
    scheduler.alpha.kubernetes.io/critical-pod: ""
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    - --runtime-config=apiserver.k8s.io/v1alpha1,eventratelimit.admission.k8s.io/v1alpha1=true
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --requestheader-username-headers=X-Remote-User
    - --tls-cert-file=/etc/kubernetes/pki/apiserver.crt
    - --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
    - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt
    - --requestheader-group-headers=X-Remote-Group
    - --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
    - --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
    - --secure-port=6443
    - --insecure-port=0
    - --admission-control=EventRateLimit,Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestrict
ion,ResourceQuota
    - --admission-control-config-file=/etc/kubernetes/admission-control/control-config.yaml
    - --enable-bootstrap-token-auth=true
    - --requestheader-allowed-names=front-proxy-client
    - --service-cluster-ip-range=10.96.0.0/12
    - --service-account-key-file=/etc/kubernetes/pki/sa.pub
    - --allow-privileged=true
    - --requestheader-extra-headers-prefix=X-Remote-Extra-
    - --advertise-address=10.142.0.2
    - --client-ca-file=/etc/kubernetes/pki/ca.crt
    - --tls-private-key-file=/etc/kubernetes/pki/apiserver.key
    - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key
    - --authorization-mode=Node,RBAC
    - --etcd-servers=http://127.0.0.1:2379
    image: gcr.io/google_containers/kube-apiserver-amd64:v1.9.5
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 10.142.0.2
        path: /healthz
        port: 6443
        scheme: HTTPS
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: kube-apiserver
    resources:
      requests:
        cpu: 250m
    volumeMounts:
    - mountPath: /etc/kubernetes/pki
      name: k8s-certs
      readOnly: true
    - mountPath: /etc/ssl/certs
      name: ca-certs
      readOnly: true
    - mountPath: /etc/pki
      name: ca-certs-etc-pki
      readOnly: true
    - mountPath: /etc/kubernetes/admission-control
      name: admission-control
      readOnly: true
  hostNetwork: true
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki
      type: DirectoryOrCreate
    name: k8s-certs
  - hostPath:
      path: /etc/ssl/certs
      type: DirectoryOrCreate
    name: ca-certs
  - hostPath:
      path: /etc/pki
      type: DirectoryOrCreate
    name: ca-certs-etc-pki
  - hostPath:
      path: /etc/kubernetes/admission-control
      type: DirectoryOrCreate
    name: admission-control
status: {}

Create eventRateLimit .yaml file under /etc/kubernetes/admission-control/

kind: Configuration
apiVersion: eventratelimit.admission.k8s.io/v1alpha1
limits:
- type: Namespace
  qps: 50
  burst: 100
  cacheSize: 2000
- type: User
  qps: 10
  burst: 50

create admission-control-config file as below:

kind: AdmissionConfiguration
apiVersion: apiserver.k8s.io/v1alpha1
plugins:
- name: EventRateLimit
  path: /etc/kubernetes/admission-control/eventRateLimit.yaml
- name: Initializers
- name: NamespaceLifecycle
- name: LimitRanger
- name: ServiceAccount
- name: DefaultStorageClass
- name: DefaultTolerationSeconds
- name: NodeRestriction
- name: ResourceQuota

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
  • Cloud provider or hardware configuration:
  • OS (e.g. from /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Comments: 44 (18 by maintainers)

Most upvoted comments

/reopen

I hit the same issue trying to use EventRateLimit with the --runtime-config flag; leaving admissionregistration.k8s.io/v1alpha1 off of that seems to get the admission controller working fine.

I think this is a documentation bug, since the admission controller doc is what’s telling us to use --runtime-config for this. https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/