kubebuilder: make deploy is broken for kustomize v2.0.0 release

kustomize v2.0.0 requires that all paths other than bases have to terminate in same or subdirectory of the kustomization directory. So kustomize build config/default in make deploy fails because kustomization.yaml contains resource paths like ../rbac/rbac_role.yaml. According to kustomize v2.0.0, rbac and manager folder should be under config/default directory.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 4
  • Comments: 15 (8 by maintainers)

Commits related to this issue

Most upvoted comments

For existing projects, please follow this instruction to migrate the config directory structure to the new structure, which works with kustomize 2.0.0

  1. In config/rbac/ directory, add a kustomization.yaml with following content:
resources:
- rbac_role.yaml
- rbac_role_binding.yaml
  # Comment the following 3 lines if you want to disable
  # the auth proxy (https://github.com/brancz/kube-rbac-proxy)
  # which protects your /metrics endpoint.
- auth_proxy_service.yaml
- auth_proxy_role.yaml
- auth_proxy_role_binding.yaml
  1. In config/manager/ directory, add a kustomization.yaml with following content:
resources:
- manager.yaml
  1. In config/default/ directory, change following block in kustomization.yaml
# Each entry in this list must resolve to an existing	bases:
# resource definition in YAML.  These are the resource	- ../rbac
# files that kustomize reads, modifies and emits as a	- ../manager
# YAML string, with resources separated by document	
# markers ("---").	
resources:	
- ../rbac/rbac_role.yaml	
- ../rbac/rbac_role_binding.yaml	
- ../manager/manager.yaml	
  # Comment the following 3 lines if you want to disable	
  # the auth proxy (https://github.com/brancz/kube-rbac-proxy)	
  # which protects your /metrics endpoint.	
- ../rbac/auth_proxy_service.yaml	
- ../rbac/auth_proxy_role.yaml	
- ../rbac/auth_proxy_role_binding.yaml

to

bases:
- ../rbac
- ../manager

Agree with the change suggested by @figo We can change it to following directory structure

config/
   crds/
      kustomization.yaml
   rbac/
      kustomization.yaml
  manager/
     kustomization.yaml
  default/
     kustomization.yaml   # This one contains the bases pointing to rbac/ and manager/

I’ve had a play with config directory structures over the past few days and arrived at the following, feeling like it’s the cleanest I’ve found, keeping all the kustomize builds from config/default, as from what I see manager or rbac manifests aren’t used outside of this.

config
├── crds
│   └── your_v1beta1_sample.yaml
├── default
│   ├── kustomization.yaml
│   ├── manager
│   │   └── manager.yaml
│   ├── manager_auth_proxy_patch.yaml
│   ├── manager_image_patch.yaml
│   ├── manager_prometheus_metrics_patch.yaml
│   └── rbac
│       ├── auth_proxy_role.yaml
│       ├── auth_proxy_role_binding.yaml
│       ├── auth_proxy_service.yaml
│       ├── rbac_role.yaml
│       └── rbac_role_binding.yaml
└── samples
    └── your_v1beta1_sample.yaml

I believe the reason for the “default” directory is so that a project can maintain multiple different kustomize overlays. “default”, “production”, “debug”, etc. that each allow for customizations based on a different environments while still maintaining the same bases and resources. At least that is how we are using that directory level within our projects. I would like to see that continue to exist moving forward.

at cluster-api, we are using bases instead of resources to work with kustomize 2.0, folder structure does not need be changed, but it does need to add a kustomize.yaml in each folder.

bases:
- ../crds/
- ../rbac/
- ../manager/

please refer to https://github.com/kubernetes-sigs/cluster-api/blob/master/config/default/kustomization.yaml

Also seeing this. A quick hack is to bring the config/default manifests into the top-level config/ directory, and updating your Makefile deploy stage to use kustomize build config | kubectl apply -f -.

Whether having the kustomize manifests in the top-level config directory is a longterm solution, I’m unsure.