kubevela: Install of kubevela in OpenShift fails

Describe the bug

The installation of kubevela into an OpenShift cluster fails, even though a successful Crossplane self-hosted install with OAM is in place in the same cluster.

To Reproduce

Install Crossplane into an OpenShift cluster as per Crossplane docs and using this install command to ensure security context with OAM enabled

helm install crossplane --namespace crossplane-system crossplane-stable/crossplane --version 1.2.1 --set securityContextCrossplane.runAsUser=null --set securityContextCrossplane.runAsGroup=null --set securityContextRBACManager.runAsUser=null --set securityContextRBACManager.runAsGroup=null --set alpha.oam.enabled=true

This creates the crossplane-system namespace and installs Crossplane, but also creates the vela-system namespace.

Then go to install Kubevela as per docs

Tried it two ways:

# option 1
helm install --create-namespace -n vela-system kubevela kubevela/vela-core

# error received
Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "ScopeDefinition" in version "core.oam.dev/v1beta1", unable to recognize "": no matches for kind "TraitDefinition" in version "core.oam.dev/v1beta1", unable to recognize "": no matches for kind "WorkloadDefinition" in version "core.oam.dev/v1beta1"]

and

# option 2 
## delete oam crds first
kubectl get crd |grep oam |awk '{print $1}'|xargs kubectl delete crd

## try kubevela install( thinking it would need same security context as Crossplane install)
helm install kubevela --create-namespace --namespace vela-system kubevela/vela-core --set securityContextCrossplane.runAsUser=null --set securityContextCrossplane.runAsGroup=null --set securityContextRBACManager.runAsUser=null --set securityContextRBACManager.runAsGroup=null

# error received
Error: failed pre-install: timed out waiting for the condition

Checking the OpenShift CRDs I can see them created e.g. customresourcedefinition.apiextensions.k8s.io "appdeployments.core.oam.dev" etc

But in the events log for vela-system you see

kubevela-vela-core-admission-create
Namespace vela-system

Error creating: pods "kubevela-vela-core-admission-create-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.runAsUser: Invalid value: 2000: must be in the ranges: [1000670000, 1000679999]]

Based on experiences of understanding how Crossplane needs to install in OpenShift this looks like a similar RBAC problem around security context, despite using the securityContext settings (which worked for Crossplane install).

Expected behavior

Kubevela installed with no error message

Cluster information

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2-0-g52c56ce", GitCommit:"297a4ac4250237ed5aed54e916ec7b88fda61bbe", GitTreeState:"clean", BuildDate:"2021-03-06T07:17:32Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.0+263ee0d", GitCommit:"263ee0d276ef68a1958d000be103d7c69f3e5494", GitTreeState:"clean", BuildDate:"2021-03-21T08:03:52Z", GoVersion:"go1.15.7", Compiler:"gc", Platform:"linux/amd64"}

About this issue

Commits related to this issue

Most upvoted comments

Click on the “Install OpenShift on your laptop” > then you need to register for a free Red Hat account and login to do the install -

The link takes to different options - go to Local install https://cloud.redhat.com/openshift/create/local and download for your environment.

I’ll reach out to my Red Hat friends.

So at the moment when runAsUser=null is that overriding runAsUser: 60000 in kubevela’s values.yaml - but the Helm implementation doesn’t know what to do about it … i.e. nothing being passed through from upstream (i.e, no controller config)?

Still in error.

I followed these steps, cleaning up my vela-system namespace and tried helm install a number of ways.

clone PR 1713

> git clone https://github.com/oam-dev/kubevela
...
> gh pr checkout 1713
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 5 (delta 3), reused 5 (delta 3), pack-reused 0
Unpacking objects: 100% (5/5), 513 bytes | 85.00 KiB/s, done.
From https://github.com/oam-dev/kubevela
 * [new ref]           refs/pull/1713/head -> security
Switched to branch 'security'

A new release of gh is available: 1.9.2 → v1.10.3
To upgrade, run: brew update && brew upgrade gh
https://github.com/cli/cli/releases/tag/v1.10.3

clean up

> kubectl get crd |grep oam |awk '{print $1}'|xargs kubectl delete crd
> oc delete project vela-system

clean up and then try each of these install

> helm install --create-namespace -n vela-system kubevela charts/vela-core
Error: failed pre-install: timed out waiting for the condition

> helm install kubevela --create-namespace --namespace vela-system charts/vela-core --set securityContext.runAsUser=null 
Error: failed pre-install: timed out waiting for the condition

> helm install kubevela --create-namespace --namespace vela-system charts/vela-core --set securityContext.runAsUser=null --set podSecurityContext.fsGroup=null 
Error: failed pre-install: timed out waiting for the condition

Check event log

> oc project vela-system

> oc get events
LAST SEEN   TYPE      REASON         OBJECT                                    MESSAGE
34s         Warning   FailedCreate   job/kubevela-vela-core-admission-create   Error creating: pods "kubevela-vela-core-admission-create-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.runAsUser: Invalid value: 2000: must be in the ranges: [1000690000, 1000699999]]

@cg2p Yes, indeed, it does the root case, we will fix it ASAP, thanks again!