kapp: kapp should work with kubernetes APIs that are affected by faulty APIService resources
Hi,
I’ve recently played with certmanager and mess up my environment a little bit (my namespaces refused to be deleted so I had to force the deletion). But, since then, I couldn’t launch kapp anymore:
kapp deploy -n hello -a app-hello -f ./sandbox/config.app.yml
Error: unable to retrieve the complete list of server APIs: admission.certmanager.k8s.io/v1beta1: the server is currently unable to handle the request
Delete the orphaned apiservices did the trick: kubectl delete apiservices v1beta1.admission.certmanager.k8s.io
Could kapp be more resilient here and start even in case of errors in server APIs?
Thanks –nick
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 2
- Comments: 19 (11 by maintainers)
Commits related to this issue
- Add hints for kapp where needed Derived from https://github.com/k14s/kapp/issues/12#issuecomment-528585156 — committed to scothis/release by scothis 5 years ago
- Add hints for kapp where needed (#88) Derived from https://github.com/k14s/kapp/issues/12#issuecomment-528585156 — committed to vmware-archive/release by scothis 5 years ago
- Retry installing cert-manager with kapp It frequently fails because of a known issue in kapp. Perhaps retrying is enough to get it to work. Refs k14s/kapp#12 — committed to scothis/system by scothis 5 years ago
- Retry installing cert-manager with kapp (#251) It frequently fails because of a known issue in kapp. Perhaps retrying is enough to get it to work. Refs k14s/kapp#12 — committed to projectriff/system by scothis 5 years ago
similar issue in #35 related to installing certmanager. replied there as well:
if new APIService is installed and it does not have backing components available, it breaks certain k8s APIs that kapp happens to use (also used by
kubectl api-resourcescommand).one way to fix this issue is by making sure that kapp install apiservice backing components before installing APIService itself. to do so one can use kapp ordering directives. ive actually previously installed certmgr this way as shown here: https://github.com/k14s/k8s-resource-protection/tree/develop/config-certmanager. this directory can be installed via
ytt -f . --ignore-unknown-comments | kapp deploy -a certmgr -f- -y.https://github.com/k14s/k8s-resource-protection/tree/develop/config-certmanager/patches.yml makes sure that APIService is installed after deployment+service, and deleted first in case of deletion.
reopening since im thinking more about this. someone else just recently reported this in slack: https://kubernetes.slack.com/archives/CH8KCCKA5/p1565900345299800. i’ve also run into this myself a few days ago with certmgr as well.
thinking about putting some apiservice ordering constraints that by default would make ux better… (may be something like this but more generic https://github.com/k14s/k8s-resource-protection/blob/bb6e2411f0ed35c4b759bddfe7d3a4defe527f67/config-certmanager/patches.yml)
It was more of a one off. Feel free to close the issue. Thanks