Bug #49808
openCeph manager becomes unresponsive and is replaced by standby daemon
0%
Description
Running (for dashboard) not supported OS CentOS7.9.2009, dependencies not in EPEL installed via pip. Ceph version 15.2.9. MGR process was running > 2 weeks this time before becoming unresponsive has been shorter previously. I do not expect to get support running an unsupported OS / component, just providing information if this is affecting users running supported combinations.
Manager logs:
2021-03-15T13:32:46.585+0100 7f908eabe700 0 [prometheus DEBUG root] Starting method get_rbd_stats.
2021-03-15T13:32:46.585+0100 7f908eabe700 0 [prometheus DEBUG root] Method get_rbd_stats ran 0.000 seconds.
2021-03-15T13:32:46.597+0100 7f90775d0700 0 [dashboard DEBUG request] [********:50471] [GET] [dash] /api/summary
2021-03-15T13:32:46.597+0100 7f90775d0700 0 [dashboard DEBUG auth] token: *******************
2021-03-15T13:32:46.597+0100 7f90775d0700 4 mgr get_store get_store key: mgr/dashboard/jwt_token_black_list
2021-03-15T13:32:46.597+0100 7f90775d0700 0 [dashboard DEBUG auth] checking authorization...
2021-03-15T13:32:46.629+0100 7f9061ee2700 0 [dashboard DEBUG viewcache] starting execution of <function get_daemons_and_pools at 0x7f90a005c2f0>
2021-03-15T13:32:46.757+0100 7f9099713700 0 log_channel(audit) log [DBG] : from='client.151670041 -' entity='client.admin' cmd=[{"prefix": "osd pool stats", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2021-03-15T13:32:46.812+0100 7f908eabe700 0 [prometheus DEBUG root] Method collect ran 0.733 seconds.
2021-03-15T13:32:46.812+0100 7f908eabe700 0 [prometheus DEBUG root] collecting cache in thread done
2021-03-15T13:32:46.840+0100 7f9061ee2700 0 [dashboard DEBUG controllers.rbd_mirror] Constructing IOCtx rbd
2021-03-15T13:32:46.841+0100 7f9061ee2700 0 [dashboard DEBUG controllers.rbd_mirror] Constructing IOCtx ssc-cinder-volumes-md
2021-03-15T13:32:46.841+0100 7f9061ee2700 0 [dashboard DEBUG controllers.rbd_mirror] Constructing IOCtx ssc-glance-images-md
2021-03-15T13:32:46.842+0100 7f9061ee2700 0 [dashboard DEBUG controllers.rbd_mirror] Constructing IOCtx ssc-nova-vms-md
2021-03-15T13:32:46.843+0100 7f9061ee2700 0 [dashboard DEBUG controllers.rbd_mirror] Constructing IOCtx mare4
2021-03-15T13:32:46.844+0100 7f9061ee2700 0 [dashboard DEBUG viewcache] execution of <function get_daemons_and_pools at 0x7f90a005c2f0> finished in: 0.21473383903503418
2021-03-15T13:32:46.845+0100 7f90775d0700 0 [dashboard INFO request] [********:50471] [GET] [200] [0.249s] [dash] [241.0B] /api/summary
2021-03-15T13:32:47.373+0100 7f9071dc5700 0 [dashboard DEBUG notification_queue] processing queue: 1
2021-03-15T13:32:47.508+0100 7f9071dc5700 0 [dashboard DEBUG notification_queue] processing queue: 1
2021-03-15T13:32:47.571+0100 7f9077dd1700 0 [dashboard DEBUG request] [********:62014] [GET] [dash] /api/cluster_conf/
2021-03-15T13:32:47.571+0100 7f9077dd1700 0 [dashboard DEBUG auth] token: *******************
2021-03-15T13:32:47.571+0100 7f9077dd1700 4 mgr get_store get_store key: mgr/dashboard/jwt_token_black_list
2021-03-15T13:32:47.572+0100 7f9077dd1700 0 [dashboard DEBUG auth] checking authorization...
2021-03-15T13:32:47.572+0100 7f9077dd1700 0 [dashboard DEBUG auth] checking '['read']' access to 'config-opt' scope
2021-03-15T13:32:47.751+0100 7f9087970700 0 [rbd_support DEBUG root] TaskHandler: tick
2021-03-15T13:32:47.751+0100 7f908d97c700 0 [rbd_support DEBUG root] PerfHandler: tick
Files