verrazzano: Verrazzano Installation failing on on-premise K8s cluster

Hello! I’m trying to install Verrazzano on a local K8s cluster.

I install the operator and create this CRD:

apiVersion: install.verrazzano.io/v1alpha1
kind: Verrazzano
metadata:
  name: my-verrazzano
spec:
  environmentName: env
  profile: dev
  components:
    ingress:
      type: NodePort
      nginxInstallArgs:
      - name: controller.kind
        value: DaemonSet
      - name: controller.hostPort.enabled
        value: "true"
      - name: controller.nodeSelector.ingress-ready
        value: "true"
        setString: true
      - name: controller.tolerations[0].key
        value: node-role.kubernetes.io/master
      - name: controller.tolerations[0].operator
        value: Equal
      - name: controller.tolerations[0].effect
        value: NoSchedule

It fails during the installation of rancher:

ancher admin user: admin
Reset Rancher admin password and create secrets
Installing Rancher                                                           [  OK  ]
Setting Rancher Server URL                                                   [ .... ]
Retrieving the rancher admin token from Rancher at rancher.env.127.0.0.1.nip.io
Retrying get rancher_admin_token
Retrying get rancher_admin_token
Retrying get rancher_admin_token
Retrying get rancher_admin_token

From rancher:

ime="2021-06-04T07:26:35Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=ClusterGroup controller"
time="2021-06-04T07:26:35Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=GitRepo controller"
time="2021-06-04T07:26:35Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=ClusterRegistrationToken controller"
time="2021-06-04T07:26:35Z" level=info msg="All controllers are started"
time="2021-06-04T07:26:35Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=Cluster controller"
time="2021-06-04T07:26:35Z" level=error msg="error syncing 'fleet-local/local': handler workspace-backport: fleetworkspaces.management.cattle.io \"fleet-local\" not found, requeuing"

From fleet-system:

(more or less all pod have this problem)

time="2021-06-04T07:18:59Z" level=info msg="Starting fleet.cattle.io/v1alpha1, Kind=BundleDeployment controller"
E0604 07:19:00.701370       1 memcache.go:196] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
E0604 07:19:00.718770       1 memcache.go:101] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
time="2021-06-04T07:20:58Z" level=info msg="Deployed new agent for cluster fleet-local/local"
time="2021-06-04T07:21:23Z" level=info msg="Cluster registration fleet-local/request-xmf72, cluster fleet-local/local granted [false]"
time="2021-06-04T07:21:23Z" level=info msg="Cluster registration fleet-local/request-xmf72, cluster fleet-local/local granted [true]"
time="2021-06-04T07:21:24Z" level=info msg="Cluster registration fleet-local/request-xmf72, cluster fleet-local/local granted [true]"

I guess it should come with Verrazzano ? Anyway I tried to install the metric-server manually:

kubectl api-versions  | grep metric
metrics.k8s.io/v1beta1

but situation didn’t change. Do you know what I could do ?

Cheers, Antonio

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (8 by maintainers)

Most upvoted comments

I think we can close it. Thanks for the help Deva. As we discussed on Slack, I found the problem in this line

- name: controller.nodeSelector.ingress-ready
        value: "true"
        setString: true

The easy fix was to add the label to my nodes. As you said the other options is to remove the nginxInstallerArgs, that are not needed anymore.