prometheus-operator: Upgrading to version 0.26.0 breaks PV access
Suspect that this is possibly the result of https://github.com/coreos/prometheus-operator/pull/2109
What did you do? Upgraded from prometheus-operator v0.25 to v0.26
What did you expect to see? Prometheus pods should come up with Prometheus v0.25
What did you see instead? Under which circumstances? Pods crashlooping with
level=error ts=2018-11-30T13:20:02.660393999Z caller=main.go:625 err="opening storage failed: open /prometheus/wal/00000949: permission denied"
This may be only apparent when using persistent volumes
Environment Azure
- Prometheus Operator version:
v0.26
- Kubernetes version information:
Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:46:06Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:43:26Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
- Kubernetes cluster kind:
kubeadm
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 7
- Comments: 21 (10 by maintainers)
^^ Having the same problem
It worked for me by adding the following under prometheusSpec
prometheusSpec: retention: 90d # Data retention value securityContext: runAsNonRoot: false runAsUser: 0 fsGroup: 0
Also having this problem when using Trident as the volume provisioner. Reverting to 0.25.0 resolved the issue.