k9s: Fails to start after a while

Describe the bug
running the version 0.19.5
I am having some issues appearing first after a while and then blocking completly the start of the tool (see at the botton of the issue for logs).
So from a fresh cluster (docker-for-mac or K3d) eveything is running fine until I have some error messages appearing at the bottom like [list watch] access denied on resource "default":"v1/pods"
Then if I quit K9s and start to relaunch it it fails with the logs below.
I am doing some experiments with a webhook admission controller so I wonder if this could be related.
If I delete my cluster and start a fresh one the issue disapear and come later somehow.
To Reproduce it is hard to describe some steps, I am playing with Kyvero ClusterPolicies but and this issu happen’s after a while
Expected behavior not to crash
Screenshots
Versions (please complete the following information): On MacOs 10.15.3 It fails on both Docker-for-Desktop : 2.3.0.2 or on k3d version v1.7.0 k9s version: Version: 0.19.5 Commit: 9f1b099e290f6e73d7dead475b34a180a18eb9a5 Date: 2020-05-15T22:35:38Z Additional context start logs
3:40PM INF 🐶 K9s starting up...
3:40PM DBG Active Context "k3s-default"
3:40PM DBG Connecting to API Server https://localhost:6443
3:40PM DBG RESETING CON!!
3:40PM INF ✅ Kubernetes connectivity
3:40PM DBG [Config] Saving configuration...
3:40PM INF No context specific skin file found -- /Users/myname/.k9s/k3s-default_skin.yml
3:40PM DBG CURRENT-NS "" -- No active namespace specified
3:40PM INF No namespace specified using cluster default namespace
3:40PM DBG Factory START with ns `""
3:40PM DBG Connecting to API Server https://localhost:6443
3:40PM WRN Dial Failed! error="Post \"https://localhost:6443/apis/authorization.k8s.io/v1/selfsubjectaccessreviews\": context deadline exceeded"
3:40PM WRN Fail CRDs load error="Post \"https://localhost:6443/apis/authorization.k8s.io/v1/selfsubjectaccessreviews\": context deadline exceeded"
3:40PM DBG SkinWatcher watching `/Users/myname/.k9s/skin.yml
3:40PM DBG CustomView watching `/Users/myname/.k9s/views.yml
3:40PM WRN Dial Failed! error="Post \"https://localhost:6443/apis/authorization.k8s.io/v1/selfsubjectaccessreviews\": context deadline exceeded"
3:40PM ERR Saved command load failed. Loading default view error="Post \"https://localhost:6443/apis/authorization.k8s.io/v1/selfsubjectaccessreviews\": context deadline exceeded"
3:40PM ERR Boom! [list watch] access denied on resource "default":"v1/pods"
3:40PM ERR goroutine 1 [running]:
runtime/debug.Stack(0x4195040, 0x2c1c703, 0x0)
runtime/debug/stack.go:24 +0x9d
github.com/derailed/k9s/cmd.run.func1()
github.com/derailed/k9s/cmd/root.go:73 +0x11d
panic(0x2936e00, 0xc00043b450)
runtime/panic.go:969 +0x166
github.com/derailed/k9s/cmd.run(0x41753c0, 0xc00000da80, 0x0, 0x2)
github.com/derailed/k9s/cmd/root.go:89 +0x1ef
github.com/spf13/cobra.(*Command).execute(0x41753c0, 0xc00004c0d0, 0x2, 0x2, 0x41753c0, 0xc00004c0d0)
github.com/spf13/cobra@v1.0.0/command.go:846 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0x41753c0, 0x0, 0x0, 0x0)
github.com/spf13/cobra@v1.0.0/command.go:950 +0x349
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.0.0/command.go:887
github.com/derailed/k9s/cmd.Execute()
github.com/derailed/k9s/cmd/root.go:64 +0x2d
main.main()
github.com/derailed/k9s/main.go:27 +0x1a6
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 7
- Comments: 39 (23 by maintainers)
Commits related to this issue
- add pvc usage check + bugz #642 #754 #753 #743 #728 #718 — committed to derailed/k9s by derailed 4 years ago
For anyone else finding this through keyword searches, my issues was similar: a misbehaving ValidatingWebhook.
I was using Gatekeeper, and had installed an over-restrictive NetworkPolicy which was blocking the webhook.
Check for any Validating Webhooks with
kubectl get Validatingwebhookconfigurations
, then check for network policies in those namespaces withkubectl get netpol -A
.A note to others that may view this with a similar issue: the
MEOW! Post "https://<MASTER-IP>/apis/authorization.k8s.io/v1/selfsubjectaccessreviews": context deadline exceeded
issue, at least on GCP/GKE was resolved by by refreshing credentials (gcloud container clusters get-credentials ...
).