rook: ceph-mgr errors with KeyError mimic in rook 1.1.0 and Nautilus 14.2.3
Is this a bug report or feature request?
- Bug Report
Deviation from expected behavior: After starting the mgr pod issues the following logs over and over again
[16/Sep/2019:14:44:33] HTTP
Request Headers:
HOST: 10.6.70.14:9283
X-PROMETHEUS-SCRAPE-TIMEOUT-SECONDS: 5.000000
Remote-Addr: ::ffff:10.6.69.4
ACCEPT: application/openmetrics-text; version=0.0.1,text/plain;version=0.0.4;q=0.5,*/*;q=0.1
USER-AGENT: Prometheus/2.12.0
ACCEPT-ENCODING: gzip
::ffff:10.6.69.4 - - [16/Sep/2019:14:44:33] "GET /metrics HTTP/1.1" 500 1733 "" "Prometheus/2.12.0"
debug 2019-09-16 14:44:34.098 7f79b18b6700 0 log_channel(cluster) log [DBG] : pgmap v411: 1212 pgs: 1212 active+clean; 2.6 TiB data, 5.5 TiB used, 5.7 TiB / 11 TiB avail; 767 B/s rd, 11 MiB/s wr, 311 op/s
::ffff:10.6.70.1 - - [16/Sep/2019:14:44:34] "GET / HTTP/1.1" 200 155 "" "kube-probe/1.15"
[16/Sep/2019:14:44:34] HTTP Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "/usr/share/ceph/mgr/prometheus/module.py", line 1054, in metrics
return self._metrics(instance)
File "/usr/share/ceph/mgr/prometheus/module.py", line 1068, in _metrics
instance.collect_cache = instance.collect()
File "/usr/share/ceph/mgr/prometheus/module.py", line 918, in collect
self.get_mgr_status()
File "/usr/share/ceph/mgr/prometheus/module.py", line 490, in get_mgr_status
always_on_modules = mgr_map['always_on_modules'][ceph_release]
KeyError: 'mimic'
I did not yet run the feature update to nautilus since the upgrade is not yet completed.
Expected behavior: ceph mgr works as normal without this error.
Environment:
- OS (e.g. from /etc/os-release): containerlinux
- Kernel (e.g.
uname -a): - Cloud provider or hardware configuration: bare-metal
- Rook version (use
rook versioninside of a Rook Pod): v1.1.0 - Storage backend version (e.g. for ceph do
ceph -v): 14.2.3 - Kubernetes version (use
kubectl version): 1.15.2 - Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): bare-metal
- Storage backend status (e.g. for Ceph use
ceph healthin the Rook Ceph toolbox): HEALTH_OK
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 18 (15 by maintainers)
I expect the fix to end up in Nautilus 14.2.4
Isn’t this the same as https://github.com/rook/rook/issues/3424 (https://tracker.ceph.com/issues/40754) ?
stay_signed_inis a login property from Mimic that was removed in Nautilus. It remains in the stale cached HTML/JS assets. Refreshing any intermediate proxy/caching layer should resolve this.