Actions
Bug #52462
closedmgr/prometheus: mgr getting restarted when the prometheus deployment starts.
Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
cephadm/monitoring
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
After the cephadm bootstrap, I see the mgr is getting restarted when its trying to deploy the prometheus module. I noticed this in master and not at all reproducible in pacific. Tried this several time in the kcli environment. To verify this I bootstrapped with --skip-monitoring-stack option and I didn't noticed the mgr getting restarted.
Aug 31 08:32:26 ceph-node-00.cephlab.com ceph-mgr[13726]: [progress INFO root] update: starting ev c53bc746-9c8b-47e3-91a3-f51b58618068 (Updating prometheus deployment (+1 -> 1)) Aug 31 08:32:27 ceph-node-00.cephlab.com ceph-mgr[13726]: log_channel(cluster) log [DBG] : pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail Aug 31 08:32:27 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr handle_mgr_map respawning because set of enabled modules changed! Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: ignoring --setuser Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: ceph Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: since I am not root Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: ignoring --setgroup Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: ceph Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: since I am not root Aug 31 08:32:27 ceph-node-00.cephlab.com conmon[13721]: Aug 31 08:32:27 ceph-node-00.cephlab.com ceph-mgr[13726]: ceph version 17.0.0-7372-ga2827edf (a2827edf48e194a21026b4cf146b64663677109f) quincy (dev), process ceph-mgr, pid 2 Aug 31 08:32:27 ceph-node-00.cephlab.com ceph-mgr[13726]: pidfile_write: ignore empty --pid-file Aug 31 08:32:27 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'alerts' Aug 31 08:32:28 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'balancer' Aug 31 08:32:28 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'crash' Aug 31 08:32:28 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'dashboard' Aug 31 08:32:30 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'devicehealth' Aug 31 08:32:30 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'diskprediction_local' Aug 31 08:32:31 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'hello' Aug 31 08:32:31 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'influx' Aug 31 08:32:31 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'insights' Aug 31 08:32:32 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'iostat' Aug 31 08:32:32 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'k8sevents' Aug 31 08:32:33 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'localpool' Aug 31 08:32:33 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'orchestrator' Aug 31 08:32:33 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'osd_perf_query' Aug 31 08:32:33 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'pg_autoscaler' Aug 31 08:32:34 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'progress' Aug 31 08:32:34 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'prometheus' Aug 31 08:32:34 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'rbd_support' Aug 31 08:32:34 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'restful' Aug 31 08:32:35 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Loading python module 'rook' Aug 31 08:32:36 ceph-node-00.cephlab.com ceph-mgr[13726]: mgr[py] Module not found: 'rook'
Actions