kops: kops create cluster centos not working

Hi,

I am using kops 1.7.1 version and trying to create k8s cluster in aws env as below with RBAC enabled.

kops create cluster --name=rbactwo-test.com \
  --state=s3://{myS3} --zones=us-west-2a \
  --node-count=1 \
  --node-size "m4.4xlarge" \
  --master-size  "m3.large" \
  --networking "weave" \
  --topology "private" \
  --master-count=1 \
  --image "{}" \
  --api-loadbalancer-type "public" \
  --authorization "RBAC" \
  --channel "stable" \
  --cloud "aws" \
  --dns-zone "{}}" \
  --admin-access "{}" \
  --kubernetes-version "1.8.2" \
  --vpc "{}}" \
  --network-cidr "{}" \
  --ssh-access "{my IP list}" \
  --dns "public" \
  --ssh-public-key "../ssh/public/admin/id_rsa.pub"

Once cluster is created, seeing below errors.

-> to get the nodes

kubectl get nodes
No resources found.

-> to get the pods

kubectl get pods --all-namespaces
NAMESPACE     NAME                                  READY     STATUS    RESTARTS   AGE
kube-system   dns-controller-76866bcfdf-vx7k9       0/1       Pending   0          8m
kube-system   kube-dns-7f56f9f8c7-vh2b6             0/3       Pending   0          8m
kube-system   kube-dns-autoscaler-f4c47db64-8zqfs   0/1       Pending   0          8m

–> Error in master node

I1101 07:53:15.400044       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "services" cluster-wide
I1101 07:53:15.400220       5 wrap.go:42] GET /api/v1/services?resourceVersion=0: (557.262µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:15.401450       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "pods" cluster-wide
I1101 07:53:15.401573       5 wrap.go:42] GET /api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-2-107-69.us-west-2.compute.internal&resourceVersion=0: (367.424µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:15.470661       5 wrap.go:42] GET /api/v1/namespaces/kube-system/endpoints/kube-scheduler: (1.265591ms) 200 [[kube-scheduler/v1.8.2 (linux/amd64) kubernetes/bdaeafa/leader-election] 127.0.0.1:41764]
I1101 07:53:15.473207       5 wrap.go:42] PUT /api/v1/namespaces/kube-system/endpoints/kube-scheduler: (1.867976ms) 200 [[kube-scheduler/v1.8.2 (linux/amd64) kubernetes/bdaeafa/leader-election] 127.0.0.1:41764]
I1101 07:53:15.632282       5 wrap.go:42] GET /apis/batch/v1/jobs: (2.344537ms) 200 [[kube-controller-manager/v1.8.2 (linux/amd64) kubernetes/bdaeafa/system:serviceaccount:kube-system:cronjob-controller] 127.0.0.1:41696]
I1101 07:53:15.633527       5 wrap.go:42] GET /apis/batch/v1beta1/cronjobs: (630.696µs) 200 [[kube-controller-manager/v1.8.2 (linux/amd64) kubernetes/bdaeafa/system:serviceaccount:kube-system:cronjob-controller] 127.0.0.1:41696]
I1101 07:53:15.704804       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "nodes" cluster-wide
I1101 07:53:15.704944       5 wrap.go:42] GET /api/v1/nodes?fieldSelector=metadata.name%3Dip-10-2-107-69.us-west-2.compute.internal&resourceVersion=0: (433.442µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:15.964380       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "create" resource "pods" in namespace "kube-system"
I1101 07:53:15.964529       5 wrap.go:42] POST /api/v1/namespaces/kube-system/pods: (545.067µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:16.401595       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "services" cluster-wide
I1101 07:53:16.401770       5 wrap.go:42] GET /api/v1/services?resourceVersion=0: (487.769µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:16.402765       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "pods" cluster-wide
I1101 07:53:16.402872       5 wrap.go:42] GET /api/v1/pods?fieldSelector=spec.nodeName%3Dip-10-2-107-69.us-west-2.compute.internal&resourceVersion=0: (374.864µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]
I1101 07:53:16.706232       5 rbac.go:116] RBAC DENY: user "kubelet" groups ["system:nodes" "system:authenticated"] cannot "list" resource "nodes" cluster-wide
I1101 07:53:16.706386       5 wrap.go:42] GET /api/v1/nodes?fieldSelector=metadata.name%3Dip-10-2-107-69.us-west-2.compute.internal&resourceVersion=0: (465.193µs) 403 [[kubelet/v1.8.2 (linux/amd64) kubernetes/bdaeafa] 127.0.0.1:41564]

Can anyone help with what is missing here? and how to fix this issue?

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 19 (9 by maintainers)

Most upvoted comments

@chrislovecnm I have done the following.

-> Downloaded latest source code version from kops repo master branch -> Build kops and other libs as S3_BUCKET=s3://mybucket make kops upload -> Created cluster as below with our centos image and RBAC enabled

 export KOPS_BASE_URL=http://mys3bucket/kops/1.8.0-alpha.1/
  export NODEUP_URL=${KOPS_BASE_URL}linux/amd64/nodeup
  export KOPS=kops
  $KOPS create cluster {vaules deleted -- same command from above logs}

-> Cluster came up without any issues… -> RBAC also works fine

Thank you very much for your help.

ssubramanian123 on Nov 3, 2017

Somehow AMI Image which we are using does not install docker if we enable RBAC in cluster creation. So I used default Image and it works fine. Trying to find out what is actually causing the issue.

As of now, RBAC is working fine with latest manually build kops and default ami image.

Thanks @KashifSaadat & @chrislovecnm for your help.

ssubramanian123 on Nov 1, 2017