rook: ceph-mgr errors with KeyError mimic in rook 1.1.0 and Nautilus 14.2.3

Is this a bug report or feature request?

  • Bug Report

Deviation from expected behavior: After starting the mgr pod issues the following logs over and over again

[16/Sep/2019:14:44:33] HTTP
Request Headers:
  HOST: 10.6.70.14:9283
  X-PROMETHEUS-SCRAPE-TIMEOUT-SECONDS: 5.000000
  Remote-Addr: ::ffff:10.6.69.4
  ACCEPT: application/openmetrics-text; version=0.0.1,text/plain;version=0.0.4;q=0.5,*/*;q=0.1
  USER-AGENT: Prometheus/2.12.0
  ACCEPT-ENCODING: gzip
::ffff:10.6.69.4 - - [16/Sep/2019:14:44:33] "GET /metrics HTTP/1.1" 500 1733 "" "Prometheus/2.12.0"
debug 2019-09-16 14:44:34.098 7f79b18b6700  0 log_channel(cluster) log [DBG] : pgmap v411: 1212 pgs: 1212 active+clean; 2.6 TiB data, 5.5 TiB used, 5.7 TiB / 11 TiB avail; 767 B/s rd, 11 MiB/s wr, 311 op/s
::ffff:10.6.70.1 - - [16/Sep/2019:14:44:34] "GET / HTTP/1.1" 200 155 "" "kube-probe/1.15"
[16/Sep/2019:14:44:34] HTTP Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/prometheus/module.py", line 1054, in metrics
    return self._metrics(instance)
  File "/usr/share/ceph/mgr/prometheus/module.py", line 1068, in _metrics
    instance.collect_cache = instance.collect()
  File "/usr/share/ceph/mgr/prometheus/module.py", line 918, in collect
    self.get_mgr_status()
  File "/usr/share/ceph/mgr/prometheus/module.py", line 490, in get_mgr_status
    always_on_modules = mgr_map['always_on_modules'][ceph_release]
KeyError: 'mimic'

I did not yet run the feature update to nautilus since the upgrade is not yet completed.

Expected behavior: ceph mgr works as normal without this error.

Environment:

  • OS (e.g. from /etc/os-release): containerlinux
  • Kernel (e.g. uname -a):
  • Cloud provider or hardware configuration: bare-metal
  • Rook version (use rook version inside of a Rook Pod): v1.1.0
  • Storage backend version (e.g. for ceph do ceph -v): 14.2.3
  • Kubernetes version (use kubectl version): 1.15.2
  • Kubernetes cluster type (e.g. Tectonic, GKE, OpenShift): bare-metal
  • Storage backend status (e.g. for Ceph use ceph health in the Rook Ceph toolbox): HEALTH_OK

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 18 (15 by maintainers)

Most upvoted comments

I expect the fix to end up in Nautilus 14.2.4

Isn’t this the same as https://github.com/rook/rook/issues/3424 (https://tracker.ceph.com/issues/40754) ? stay_signed_in is a login property from Mimic that was removed in Nautilus. It remains in the stale cached HTML/JS assets. Refreshing any intermediate proxy/caching layer should resolve this.