Project

General

Profile

Bug #38621 » test-standby-mgr-x.log

Alfonso Martínez, 03/07/2019 10:22 AM

 
#### Only the relevant part as the full log exceeds the tracker maximum size for attachments:


-197> 2019-03-07 09:52:42.034 7f013ad4b700 10 mgr.server handle_report daemon_health_metrics [SLOW_OPS(0|(0,0))]
-196> 2019-03-07 09:52:42.163 7f014989f700 -1 received signal: Terminated from python ../qa/tasks/vstart_runner.py tasks.mgr.test_dashboard tasks.mgr.dashboard.test_auth tasks.mgr.dashboard.test_cephfs tasks.mgr.dashboard.test_cluster_configuration tasks.mgr.dashboard.test_erasure_code_profile tasks.mgr.dashboard.test_ganesha tasks.mgr.dashboard.test_health tasks.mgr.dashboard.test_host tasks.mgr.dashboard.test_logs tasks.mgr.dashboard.test_mgr_module tasks.mgr.dashboard.test_monitor tasks.mgr.dashboard.test_osd tasks.mgr.dashboard.test_perf_counters tasks.mgr.dashboard.test_pool tasks.mgr.dashboard.test_rbd tasks.mgr.dashboard.test_rbd_mirroring tasks.mgr.dashboard.test_requests tasks.mgr.dashboard.test_rgw tasks.mgr.dashboard.test_role tasks.mgr.dashboard.test_settings tasks.mgr.dashboard.test_summary tasks.mgr.dashboard.test_user (PID: 3920) UID: 0
-195> 2019-03-07 09:52:42.163 7f014989f700 -1 mgr handle_signal *** Got signal Terminated ***
-194> 2019-03-07 09:52:42.163 7f014909e700 4 mgr operator() Shutting down
-193> 2019-03-07 09:52:42.163 7f014909e700 1 client.0 shutdown
-192> 2019-03-07 09:52:42.163 7f014909e700 5 asok(0x55c48044e000) unregister_commands dump_cache
-191> 2019-03-07 09:52:42.164 7f014909e700 5 asok(0x55c48044e000) unregister_commands kick_stale_sessions
-190> 2019-03-07 09:52:42.164 7f014909e700 5 asok(0x55c48044e000) unregister_commands mds_requests
-189> 2019-03-07 09:52:42.164 7f014909e700 5 asok(0x55c48044e000) unregister_commands mds_sessions
-188> 2019-03-07 09:52:42.164 7f014909e700 5 asok(0x55c48044e000) unregister_commands status
-187> 2019-03-07 09:52:42.164 7f014909e700 1 -- 172.20.0.2:0/745 >> [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] conn(0x55c48415bb00 msgr2=0x55c48417c380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
-186> 2019-03-07 09:52:42.164 7f014909e700 1 --2- 172.20.0.2:0/745 >> [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] conn(0x55c48415bb00 0x55c48417c380 crc :-1 s=READY pgs=3 cs=0 l=1 rx=0 tx=0).stop
-185> 2019-03-07 09:52:42.164 7f014909e700 10 monclient: shutdown
-184> 2019-03-07 09:52:42.164 7f014909e700 1 -- 172.20.0.2:0/745 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x55c480fa8d80 msgr2=0x55c480fc0700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
-183> 2019-03-07 09:52:42.164 7f014909e700 1 --2- 172.20.0.2:0/745 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x55c480fa8d80 0x55c480fc0700 crc :-1 s=READY pgs=13 cs=0 l=1 rx=0 tx=0).stop
-182> 2019-03-07 09:52:42.164 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/745 conn(0x55c48415c400 msgr2=0x55c48417ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 34
-181> 2019-03-07 09:52:42.164 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/745 conn(0x55c48415c400 msgr2=0x55c48417ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed
-180> 2019-03-07 09:52:42.164 7f014c0a4700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/745 conn(0x55c48415c400 0x55c48417ca80 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted)
-179> 2019-03-07 09:52:42.164 7f014c0a4700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/745 conn(0x55c48415c400 0x55c48417ca80 crc :-1 s=READY pgs=7 cs=0 l=1 rx=0 tx=0).stop
-178> 2019-03-07 09:52:42.164 7f013b54c700 10 mgr.server shutdown begin
-177> 2019-03-07 09:52:42.164 7f013b54c700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] shutdown_connections
-176> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/2768733399 conn(0x55c48484df80 0x55c48474e700 crc :-1 s=READY pgs=4 cs=0 l=1 rx=0 tx=0).stop
-175> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/984 conn(0x55c48435d200 0x55c484387180 crc :-1 s=READY pgs=5 cs=0 l=1 rx=0 tx=0).stop
-174> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/745 conn(0x55c48415c400 0x55c48417ca80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rx=0 tx=0).stop
-173> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/863 conn(0x55c48416c880 0x55c48417dc00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rx=0 tx=0).stop
-172> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/1334171846 conn(0x55c48484c000 0x55c48474f880 crc :-1 s=READY pgs=4 cs=0 l=1 rx=0 tx=0).stop
-171> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> [v2:172.20.0.2:6826/2615,v1:172.20.0.2:6827/2615] conn(0x55c484763200 0x55c48474e380 crc :-1 s=READY pgs=6 cs=0 l=1 rx=0 tx=0).stop
-170> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/514 conn(0x55c48415c880 0x55c48417ce00 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).stop
-169> 2019-03-07 09:52:42.164 7f013ad4b700 10 mgr.server ms_handle_reset unregistering osd.3 session 0x55c48452d680 con 0x55c484763200
-168> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> [v2:172.20.0.2:6802/1418,v1:172.20.0.2:6803/1418] conn(0x55c484579200 0x55c484386a80 crc :-1 s=READY pgs=5 cs=0 l=1 rx=0 tx=0).stop
-167> 2019-03-07 09:52:42.164 7f013ad4b700 10 mgr.server ms_handle_reset unregistering osd.0 session 0x55c48452cfc0 con 0x55c484579200
-166> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> [v2:172.20.0.2:6810/1817,v1:172.20.0.2:6811/1817] conn(0x55c48435c000 0x55c484671c00 crc :-1 s=READY pgs=5 cs=0 l=1 rx=0 tx=0).stop
-165> 2019-03-07 09:52:42.164 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> [v2:172.20.0.2:6818/2216,v1:172.20.0.2:6819/2216] conn(0x55c48416c400 0x55c484670a80 crc :-1 s=READY pgs=5 cs=0 l=1 rx=0 tx=0).stop
-164> 2019-03-07 09:52:42.165 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/555 conn(0x55c48415cd00 0x55c48417c700 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).stop
-163> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-162> 2019-03-07 09:52:42.165 7f013b54c700 1 --2- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] >> 172.20.0.2:0/473 conn(0x55c48415bf80 0x55c480fc0e00 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).stop
-161> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-160> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-159> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-158> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-157> 2019-03-07 09:52:42.165 7f014c0a4700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] reap_dead start
-156> 2019-03-07 09:52:42.165 7f013ad4b700 10 mgr.server ms_handle_reset unregistering osd.1 session 0x55c48452d560 con 0x55c48435c000
-155> 2019-03-07 09:52:42.165 7f013ad4b700 10 mgr.server ms_handle_reset unregistering osd.2 session 0x55c48452cd80 con 0x55c48416c400
-154> 2019-03-07 09:52:42.167 7f013b54c700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] shutdown_connections
-153> 2019-03-07 09:52:42.167 7f013b54c700 1 -- [v2:172.20.0.2:6800/745,v1:172.20.0.2:6801/745] wait complete.
-152> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr.server shutdown done
-151> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown calling module balancer shutdown()
-150> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-149> 2019-03-07 09:52:42.167 7f013b54c700 4 mgr[balancer] Stopping
-148> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-147> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown module balancer shutdown() returned
-146> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown calling module crash shutdown()
-145> 2019-03-07 09:52:42.167 7f0139548700 20 mgr ~Gil Destroying new thread state 0x55c4841c6000
-144> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-143> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-142> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown module crash shutdown() returned
-141> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown calling module dashboard shutdown()
-140> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-139> 2019-03-07 09:52:42.167 7f013b54c700 4 mgr[dashboard] Stopping engine...
-138> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-137> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown module dashboard shutdown() returned
-136> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown calling module devicehealth shutdown()
-135> 2019-03-07 09:52:42.167 7f0138546700 20 mgr[dashboard] waiting for notification queue to finish
-134> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-133> 2019-03-07 09:52:42.167 7f013b54c700 4 mgr[devicehealth] Stopping
-132> 2019-03-07 09:52:42.167 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-131> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown module devicehealth shutdown() returned
-130> 2019-03-07 09:52:42.167 7f013b54c700 10 mgr shutdown calling module iostat shutdown()
-129> 2019-03-07 09:52:42.168 7f012e132700 20 mgr[dashboard] NQ: flush remaining events: 0
-128> 2019-03-07 09:52:42.168 7f012e132700 20 mgr[dashboard] notification queue finished
-127> 2019-03-07 09:52:42.168 7f0137d45700 20 mgr ~Gil Destroying new thread state 0x55c4841c6240
-126> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-125> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-124> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown module iostat shutdown() returned
-123> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown calling module orchestrator_cli shutdown()
-122> 2019-03-07 09:52:42.168 7f0138546700 20 mgr[dashboard] notification queue stopped
-121> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-120> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-119> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown module orchestrator_cli shutdown() returned
-118> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown calling module progress shutdown()
-117> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-116> 2019-03-07 09:52:42.168 7f0136542700 20 mgr ~Gil Destroying new thread state 0x55c4841c6480
-115> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-114> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown module progress shutdown() returned
-113> 2019-03-07 09:52:42.168 7f013b54c700 10 mgr shutdown calling module restful shutdown()
-112> 2019-03-07 09:52:42.168 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-111> 2019-03-07 09:52:42.171 7f0138546700 4 mgr[dashboard] Engine stopped
-110> 2019-03-07 09:52:42.171 7f0138546700 20 mgr ~Gil Destroying new thread state 0x55c4841c6120
-109> 2019-03-07 09:52:42.472 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-108> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown module restful shutdown() returned
-107> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown calling module status shutdown()
-106> 2019-03-07 09:52:42.472 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-105> 2019-03-07 09:52:42.472 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-104> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown module status shutdown() returned
-103> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown calling module volumes shutdown()
-102> 2019-03-07 09:52:42.472 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-101> 2019-03-07 09:52:42.472 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-100> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown module volumes shutdown() returned
-99> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joining module balancer
-98> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joined module balancer
-97> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joining module crash
-96> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joined module crash
-95> 2019-03-07 09:52:42.472 7f0135d41700 20 mgr ~Gil Destroying new thread state 0x55c4841c6360
-94> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joining module dashboard
-93> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joined module dashboard
-92> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joining module devicehealth
-91> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joined module devicehealth
-90> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joining module iostat
-89> 2019-03-07 09:52:42.472 7f013b54c700 10 mgr shutdown joined module iostat
-88> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joining module orchestrator_cli
-87> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joined module orchestrator_cli
-86> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joining module progress
-85> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joined module progress
-84> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joining module restful
-83> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joined module restful
-82> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joining module status
-81> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joined module status
-80> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joining module volumes
-79> 2019-03-07 09:52:42.473 7f013b54c700 10 mgr shutdown joined module volumes
-78> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-77> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-76> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-75> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-74> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-73> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-72> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-71> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-70> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-69> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-68> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-67> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-66> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-65> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-64> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-63> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-62> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-61> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-60> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr Gil Switched to new thread state 0x55c480f64000
-59> 2019-03-07 09:52:42.473 7f013b54c700 20 mgr ~Gil Destroying new thread state 0x55c480f64000
-58> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c483df1d40
-57> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c483df1d40
-56> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c483df0240
-55> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c483df0240
-54> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c483df1d40
-53> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c483df1d40
-52> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273afc0
-51> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273afc0
-50> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273afc0
-49> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273afc0
-48> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273ab40
-47> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273ab40
-46> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273bc20
-45> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273bc20
-44> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273a7e0
-43> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273a7e0
-42> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273aa20
-41> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273aa20
-40> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273b320
-39> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273b320
-38> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273b8c0
-37> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273b8c0
-36> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273bc20
-35> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273bc20
-34> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273a5a0
-33> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273a5a0
-32> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273b320
-31> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273b320
-30> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273a120
-29> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273a120
-28> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273a360
-27> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273a360
-26> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f65c20
-25> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f65c20
-24> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c48273b8c0
-23> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c48273b8c0
-22> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f64d80
-21> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f64d80
-20> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f64a20
-19> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f64a20
-18> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f647e0
-17> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f647e0
-16> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f645a0
-15> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f645a0
-14> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f64360
-13> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f64360
-12> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f65320
-11> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f65320
-10> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f65560
-9> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f65560
-8> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f658c0
-7> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f658c0
-6> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c4802d1560
-5> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c4802d1560
-4> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f650e0
-3> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f650e0
-2> 2019-03-07 09:52:42.473 7f014909e700 20 mgr Gil Switched to new thread state 0x55c480f64120
-1> 2019-03-07 09:52:42.473 7f014909e700 20 mgr ~Gil Destroying new thread state 0x55c480f64120
0> 2019-03-07 09:52:42.753 7f014909e700 -1 *** Caught signal (Aborted) **
in thread 7f014909e700 thread_name:mgrsb-fin

ceph version Development (no_version) nautilus (rc)
1: (()+0x9cc178) [0x55c47e5dd178]
2: (()+0x13030) [0x7f014eff2030]
3: (gsignal()+0x10f) [0x7f014e6a953f]
4: (abort()+0x127) [0x7f014e693895]
5: (()+0x7012e) [0x7f014f7f412e]
6: (PyGrammar_LabelRepr()+0) [0x7f014f7f418e]
7: (()+0x722ea) [0x7f014f7f62ea]
8: (Py_FinalizeEx()+0x13e) [0x7f014f9bf89e]
9: (PyModuleRegistry::shutdown()+0xa3) [0x55c47e43d263]
10: (()+0x7f3b02) [0x55c47e404b02]
11: (()+0x7f6932) [0x55c47e407932]
12: (boost::function1<void, int>::operator()(int) const+0x6c) [0x55c47e312a7c]
13: (FunctionContext::finish(int)+0x24) [0x55c47e30f936]
14: (Context::complete(int)+0x27) [0x55c47e30f7fd]
15: (Finisher::finisher_thread_entry()+0x38b) [0x7f0150e0a36d]
16: (Finisher::FinisherThread::entry()+0x1c) [0x55c47e3f1a30]
17: (Thread::entry_wrapper()+0x78) [0x7f0150e7659e]
18: (Thread::_entry_func(void*)+0x18) [0x7f0150e7651c]
19: (()+0x858e) [0x7f014efe758e]
20: (clone()+0x43) [0x7f014e76e6a3]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
1/ 1 ms
20/20 mon
20/20 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
20/20 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /ceph/build/out/mgr.x.log
--- end dump of recent events ---
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals debug_monc = 20/20
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals debug_ms = 1/1
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/dashboard/x/server_port = 11000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/dashboard/y/server_port = 13000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/dashboard/z/server_port = 15000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/restful/x/server_port = 12000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/restful/y/server_port = 14000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mgr/restful/z/server_port = 16000 (unrecognized option)
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals mon_pg_warn_min_per_osd = 3
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals osd_pool_default_min_size = 1
2019-03-07 09:53:02.549 7f39b57e1700 10 set_mon_vals osd_pool_default_size = 3
2019-03-07 09:53:02.552 7f39b93c3e80 1 -- 172.20.0.2:0/2134037525 shutdown_connections
2019-03-07 09:53:02.552 7f39b93c3e80 1 --2- 172.20.0.2:0/2134037525 >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8000 0x565195746700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:02.552 7f39b93c3e80 1 --2- 172.20.0.2:0/2134037525 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8900 0x565195746e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:02.552 7f39b93c3e80 1 --2- 172.20.0.2:0/2134037525 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8480 0x565195746a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:02.552 7f39b93c3e80 1 -- 172.20.0.2:0/2134037525 shutdown_connections
2019-03-07 09:53:02.552 7f39b93c3e80 1 -- 172.20.0.2:0/2134037525 wait complete.
2019-03-07 09:53:02.552 7f39b93c3e80 1 -- 172.20.0.2:0/2134037525 >> 172.20.0.2:0/2134037525 conn(0x565194acda80 msgr2=0x565194b7ab00 unknown :-1 s=STATE_NONE l=0).mark_down
2019-03-07 09:53:02.553 7f39b93c3e80 -1 WARNING: all dangerous and experimental features are enabled.
2019-03-07 09:53:02.553 7f39b93c3e80 0 ceph version Development (no_version) nautilus (rc), process ceph-mgr, pid 4187
2019-03-07 09:53:02.586 7f39b93c3e80 -1 WARNING: all dangerous and experimental features are enabled.
2019-03-07 09:53:02.591 7f39b93c3e80 1 Processor -- start
2019-03-07 09:53:02.592 7f39b93c3e80 1 -- start start
2019-03-07 09:53:02.592 7f39b93c3e80 10 monclient: build_initial_monmap
2019-03-07 09:53:02.592 7f39b93c3e80 10 monclient: monmap:
epoch 0
fsid 83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0
last_changed 2019-03-07 09:53:02.593266
created 2019-03-07 09:53:02.593266
min_mon_release 0 (unknown)
0: [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] mon.noname-a
1: [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] mon.noname-b
2: [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] mon.noname-c

2019-03-07 09:53:02.592 7f39b93c3e80 4 mgr init Registered monc callback
2019-03-07 09:53:02.592 7f39b93c3e80 10 monclient: init
2019-03-07 09:53:02.594 7f39b93c3e80 10 monclient: _reopen_session rank -1
2019-03-07 09:53:02.594 7f39b93c3e80 1 --2- >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:02.594 7f39b93c3e80 10 monclient(hunting): picked mon.noname-c con 0x5651957f8d80 addr [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0]
2019-03-07 09:53:02.594 7f39b93c3e80 1 --2- >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:02.594 7f39b93c3e80 10 monclient(hunting): picked mon.noname-b con 0x5651957f8000 addr [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0]
2019-03-07 09:53:02.594 7f39b93c3e80 1 --2- >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 0x565195746e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:02.595 7f39b93c3e80 10 monclient(hunting): picked mon.noname-a con 0x5651957f8480 addr [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0]
2019-03-07 09:53:02.595 7f39b93c3e80 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:02.595 7f39b93c3e80 1 -- --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_getmap magic: 0 v1 -- 0x565194b96540 con 0x5651957f8480
2019-03-07 09:53:02.595 7f39b93c3e80 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:02.595 7f39b93c3e80 1 -- --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_getmap magic: 0 v1 -- 0x565194b961c0 con 0x5651957f8000
2019-03-07 09:53:02.595 7f39b93c3e80 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:02.595 7f39b93c3e80 1 -- --> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] -- mon_getmap magic: 0 v1 -- 0x565194b96fc0 con 0x5651957f8d80
2019-03-07 09:53:02.595 7f39b93c3e80 10 monclient(hunting): _renew_subs
2019-03-07 09:53:02.595 7f39b67e3700 1 --2- >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:02.595 7f39b6fe4700 1 --2- >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:02.595 7f39b67e3700 10 monclient(hunting): get_auth_request con 0x5651957f8000 auth_method 0
2019-03-07 09:53:02.595 7f39b67e3700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1]
2019-03-07 09:53:02.595 7f39b67e3700 10 monclient(hunting): _init_auth method 2
2019-03-07 09:53:02.595 7f39b6fe4700 10 monclient(hunting): get_auth_request con 0x5651957f8d80 auth_method 0
2019-03-07 09:53:02.595 7f39b87e7700 1 --2- >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 0x565195746e00 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:02.595 7f39b6fe4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1]
2019-03-07 09:53:02.595 7f39b6fe4700 10 monclient(hunting): _init_auth method 2
2019-03-07 09:53:02.595 7f39b87e7700 10 monclient(hunting): get_auth_request con 0x5651957f8480 auth_method 0
2019-03-07 09:53:02.595 7f39b87e7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1]
2019-03-07 09:53:02.595 7f39b87e7700 10 monclient(hunting): _init_auth method 2
2019-03-07 09:53:02.596 7f39b87e7700 10 monclient(hunting): handle_auth_reply_more payload 9
2019-03-07 09:53:02.596 7f39b87e7700 10 monclient(hunting): handle_auth_reply_more payload_len 9
2019-03-07 09:53:02.596 7f39b87e7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
2019-03-07 09:53:02.596 7f39b67e3700 10 monclient(hunting): handle_auth_reply_more payload 9
2019-03-07 09:53:02.596 7f39b67e3700 10 monclient(hunting): handle_auth_reply_more payload_len 9
2019-03-07 09:53:02.596 7f39b67e3700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
2019-03-07 09:53:02.596 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload 9
2019-03-07 09:53:02.596 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload_len 9
2019-03-07 09:53:02.596 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
2019-03-07 09:53:02.597 7f39b87e7700 10 monclient(hunting): handle_auth_done global_id 4455 payload 1102
2019-03-07 09:53:02.597 7f39b87e7700 1 -- >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 msgr2=0x565195746700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
2019-03-07 09:53:02.597 7f39b87e7700 1 --2- >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:02.597 7f39b87e7700 1 -- >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 msgr2=0x565195746a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
2019-03-07 09:53:02.597 7f39b87e7700 1 --2- >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:02.597 7f39b87e7700 10 monclient: _finish_hunting 0
2019-03-07 09:53:02.597 7f39b87e7700 1 monclient: found mon.noname-a
2019-03-07 09:53:02.597 7f39b87e7700 20 monclient: _un_backoff reopen_interval_multipler now 1
2019-03-07 09:53:02.598 7f39b87e7700 10 monclient: _send_mon_message to mon.noname-a at v2:172.20.0.2:10000/0
2019-03-07 09:53:02.598 7f39b87e7700 1 -- --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_subscribe({config=0+,mgrmap=0+,monmap=0+}) v3 -- 0x565194c8f400 con 0x5651957f8480
2019-03-07 09:53:02.598 7f39b87e7700 1 --2- >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 0x565195746e00 crc :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_client_ident getsockname reveals I am 172.20.0.2:42070 when talking to v2:172.20.0.2:10000/0
2019-03-07 09:53:02.598 7f39b87e7700 1 -- 172.20.0.2:0/4214 learned_addr learned my addr 172.20.0.2:0/4214 (peer_addr_for_me v2:172.20.0.2:0/0)
2019-03-07 09:53:02.599 7f39b87e7700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 0x565195746e00 crc :-1 s=READY pgs=44 cs=0 l=1 rx=0 tx=0).ready entity=mon.0 client_cookie=759cff8ed629cd39 server_cookie=0 in_seq=0 out_seq=0
2019-03-07 09:53:02.601 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (crc 0 0 0) 0x56519570e600 con 0x5651957f8480
2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: handle_monmap mon_map magic: 0 v1
2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: got monmap 1 from mon.noname-a (according to old e1)
2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: dump:
epoch 1
fsid 83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0
last_changed 2019-03-07 09:51:10.636488
created 2019-03-07 09:51:10.636488
min_mon_release 14 (nautilus)
0: [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] mon.a
1: [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] mon.b
2: [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] mon.c

2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: _finish_auth 0
2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-03-07 09:52:32.602665)
2019-03-07 09:53:02.601 7f39b37dd700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:02.601 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- auth(proto 2 2 bytes epoch 0) v1 -- 0x5651957b2240 con 0x5651957f8480
2019-03-07 09:53:02.602 7f39b93c3e80 5 monclient: authenticate success, global_id 4455
2019-03-07 09:53:02.602 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 2 ==== config(13 keys) v1 ==== 420+0+0 (crc 0 0 0) 0x565194ad8780 con 0x5651957f8480
2019-03-07 09:53:02.602 7f39b37dd700 10 monclient: handle_config config(13 keys) v1
2019-03-07 09:53:02.602 7f39b93c3e80 20 log_channel(cluster) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info}
2019-03-07 09:53:02.602 7f39b93c3e80 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
2019-03-07 09:53:02.602 7f39b93c3e80 20 log_channel(audit) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info}
2019-03-07 09:53:02.602 7f39b93c3e80 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
2019-03-07 09:53:02.603 7f39b93c3e80 10 monclient: _renew_subs
2019-03-07 09:53:02.603 7f39b93c3e80 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:02.603 7f39b93c3e80 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565194b7f800 con 0x5651957f8480
2019-03-07 09:53:02.603 7f39b1fda700 10 mgr operator() config_callback: debug_mgr : 20/20
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored debug_mgr
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals debug_mgr = 20/20 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: debug_mon : 20/20
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored debug_mon
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals debug_mon = 20/20 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: debug_monc : 20/20
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored debug_monc
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals debug_monc = 20/20 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: debug_ms : 1/1
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored debug_ms
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals debug_ms = 1/1 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/x/server_port : 11000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/x/server_port:11000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/x/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/y/server_port : 13000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/y/server_port:13000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/y/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/z/server_port : 15000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/z/server_port:15000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/z/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/x/server_port : 12000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/x/server_port:12000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/x/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/y/server_port : 14000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/y/server_port:14000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/y/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/z/server_port : 16000
2019-03-07 09:53:02.604 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/z/server_port:16000
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/z/server_port
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: mon_pg_warn_min_per_osd : 3
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored mon_pg_warn_min_per_osd
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals mon_pg_warn_min_per_osd = 3 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: osd_pool_default_min_size : 1
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored osd_pool_default_min_size
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals osd_pool_default_min_size = 1 (no change)
2019-03-07 09:53:02.604 7f39b1fda700 10 mgr operator() config_callback: osd_pool_default_size : 3
2019-03-07 09:53:02.604 7f39b1fda700 4 set_mon_vals callback ignored osd_pool_default_size
2019-03-07 09:53:02.604 7f39b1fda700 20 set_mon_vals osd_pool_default_size = 3 (no change)
2019-03-07 09:53:02.604 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 3 ==== mgrmap(e 11) v1 ==== 12763+0+0 (crc 0 0 0) 0x565194b02f00 con 0x5651957f8480
2019-03-07 09:53:02.636 7f39b93c3e80 1 mgr[py] Loading python module 'ansible'
2019-03-07 09:53:02.656 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:02.813 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'ansible.Module'
2019-03-07 09:53:02.813 7f39b93c3e80 10 mgr[py] loaded 0 commands
2019-03-07 09:53:02.813 7f39b93c3e80 20 mgr[py] loaded module option server_url
2019-03-07 09:53:02.813 7f39b93c3e80 20 mgr[py] loaded module option username
2019-03-07 09:53:02.814 7f39b93c3e80 20 mgr[py] loaded module option password
2019-03-07 09:53:02.814 7f39b93c3e80 20 mgr[py] loaded module option verify_server
2019-03-07 09:53:02.814 7f39b93c3e80 10 mgr[py] loaded 4 options
2019-03-07 09:53:02.814 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'ansible'
2019-03-07 09:53:02.814 7f39b93c3e80 1 mgr[py] Loading python module 'balancer'
2019-03-07 09:53:02.833 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:02.873 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'balancer.Module'
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer status
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer mode name=mode,type=CephChoices,strings=none|crush-compat|upmap
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer on
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer off
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer pool ls
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer pool add name=pools,type=CephString,n=N
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer pool rm name=pools,type=CephString,n=N
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer eval name=option,type=CephString,req=false
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer eval-verbose name=option,type=CephString,req=false
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer optimize name=plan,type=CephString name=pools,type=CephString,n=N,req=false
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer show name=plan,type=CephString
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer rm name=plan,type=CephString
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer reset
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer dump name=plan,type=CephString
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer ls
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded command balancer execute name=plan,type=CephString
2019-03-07 09:53:02.873 7f39b93c3e80 10 mgr[py] loaded 16 commands
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded module option active
2019-03-07 09:53:02.873 7f39b93c3e80 20 mgr[py] loaded module option begin_time
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option end_time
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option begin_weekday
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option end_weekday
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option crush_compat_max_iterations
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option crush_compat_metrics
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option crush_compat_step
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option min_score
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option mode
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option sleep_interval
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option upmap_max_iterations
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option upmap_max_deviation
2019-03-07 09:53:02.874 7f39b93c3e80 20 mgr[py] loaded module option pool_ids
2019-03-07 09:53:02.874 7f39b93c3e80 10 mgr[py] loaded 14 options
2019-03-07 09:53:02.874 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'balancer'
2019-03-07 09:53:02.874 7f39b93c3e80 1 mgr[py] Loading python module 'crash'
2019-03-07 09:53:02.892 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:02.922 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'crash.Module'
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash info name=id,type=CephString
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash ls
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash post
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash prune name=keep,type=CephString
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash rm name=id,type=CephString
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash stat
2019-03-07 09:53:02.923 7f39b93c3e80 20 mgr[py] loaded command crash json_report name=hours,type=CephString
2019-03-07 09:53:02.923 7f39b93c3e80 10 mgr[py] loaded 7 commands
2019-03-07 09:53:02.923 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:02.923 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'crash'
2019-03-07 09:53:02.923 7f39b93c3e80 1 mgr[py] Loading python module 'dashboard'
2019-03-07 09:53:02.944 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.446 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'dashboard.Module'
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-jwt-token-ttl name=seconds,type=CephInt
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-jwt-token-ttl
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard create-self-signed-cert
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-alertmanager-api-host
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-alertmanager-api-host name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-alertmanager-api-host
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-audit-api-enabled
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-audit-api-enabled name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-audit-api-enabled
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-audit-api-log-payload
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-audit-api-log-payload name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-audit-api-log-payload
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-enable-browsable-api
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-enable-browsable-api name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-enable-browsable-api
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-ganesha-clusters-rados-pool-namespace
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-ganesha-clusters-rados-pool-namespace name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-ganesha-clusters-rados-pool-namespace
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-grafana-api-password
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-grafana-api-password name=value,type=CephString
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-grafana-api-password
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard get-grafana-api-url
2019-03-07 09:53:03.446 7f39b93c3e80 20 mgr[py] loaded command dashboard set-grafana-api-url name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-grafana-api-url
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-grafana-api-username
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-grafana-api-username name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-grafana-api-username
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-prometheus-api-host
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-prometheus-api-host name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-prometheus-api-host
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rest-requests-timeout
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rest-requests-timeout name=value,type=CephInt
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rest-requests-timeout
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-access-key
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-access-key name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-access-key
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-admin-resource
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-admin-resource name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-admin-resource
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-host
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-host name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-host
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-port
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-port name=value,type=CephInt
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-port
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-scheme
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-scheme name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-scheme
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-secret-key
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-secret-key name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-secret-key
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-ssl-verify
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-ssl-verify name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-ssl-verify
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard get-rgw-api-user-id
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-rgw-api-user-id name=value,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard reset-rgw-api-user-id
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard sso enable saml2
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard sso disable
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard sso status
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard sso show saml2
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard sso setup saml2 name=ceph_dashboard_base_url,type=CephString name=idp_metadata,type=CephString name=idp_username_attribute,type=CephString,req=false name=idp_entity_id,type=CephString,req=false name=sp_x_509_cert,type=CephString,req=false name=sp_private_key,type=CephString,req=false
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard set-login-credentials name=username,type=CephString name=password,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-role-show name=rolename,type=CephString,req=false
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-role-create name=rolename,type=CephString name=description,type=CephString,req=false
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-role-delete name=rolename,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-role-add-scope-perms name=rolename,type=CephString name=scopename,type=CephString name=permissions,type=CephString,n=N
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-role-del-scope-perms name=rolename,type=CephString name=scopename,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-show name=username,type=CephString,req=false
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-create name=username,type=CephString name=password,type=CephString,req=false name=rolename,type=CephString,req=false name=name,type=CephString,req=false name=email,type=CephString,req=false
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-delete name=username,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-set-roles name=username,type=CephString name=roles,type=CephString,n=N
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-add-roles name=username,type=CephString name=roles,type=CephString,n=N
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-del-roles name=username,type=CephString name=roles,type=CephString,n=N
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-set-password name=username,type=CephString name=password,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard ac-user-set-info name=username,type=CephString name=name,type=CephString name=email,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard iscsi-gateway-list
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard iscsi-gateway-add name=name,type=CephString name=service_url,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard iscsi-gateway-rm name=name,type=CephString
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded command dashboard feature name=action,type=CephChoices,strings=enable|disable|status name=features,type=CephChoices,strings=rbd|mirroring|iscsi|cephfs|rgw,req=false,n=N
2019-03-07 09:53:03.447 7f39b93c3e80 10 mgr[py] loaded 80 commands
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option server_addr
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option server_port
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option jwt_token_ttl
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option password
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option url_prefix
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option username
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option key_file
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option crt_file
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option ssl
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option ALERTMANAGER_API_HOST
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option AUDIT_API_ENABLED
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option AUDIT_API_LOG_PAYLOAD
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option ENABLE_BROWSABLE_API
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option GRAFANA_API_PASSWORD
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option GRAFANA_API_URL
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option GRAFANA_API_USERNAME
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option PROMETHEUS_API_HOST
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option REST_REQUESTS_TIMEOUT
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_ACCESS_KEY
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_ADMIN_RESOURCE
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_HOST
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_PORT
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_SCHEME
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_SECRET_KEY
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_SSL_VERIFY
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option RGW_API_USER_ID
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option FEATURE_TOGGLE_rbd
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option FEATURE_TOGGLE_mirroring
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option FEATURE_TOGGLE_iscsi
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option FEATURE_TOGGLE_cephfs
2019-03-07 09:53:03.447 7f39b93c3e80 20 mgr[py] loaded module option FEATURE_TOGGLE_rgw
2019-03-07 09:53:03.447 7f39b93c3e80 10 mgr[py] loaded 32 options
2019-03-07 09:53:03.447 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'dashboard.StandbyModule'
2019-03-07 09:53:03.447 7f39b93c3e80 4 mgr[py] Standby mode available in module 'dashboard'
2019-03-07 09:53:03.447 7f39b93c3e80 1 mgr[py] Loading python module 'deepsea'
2019-03-07 09:53:03.465 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.584 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'deepsea.DeepSeaOrchestrator'
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded command deepsea config-set name=key,type=CephString name=value,type=CephString
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded command deepsea config-show
2019-03-07 09:53:03.584 7f39b93c3e80 10 mgr[py] loaded 2 commands
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded module option salt_api_url
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded module option salt_api_eauth
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded module option salt_api_username
2019-03-07 09:53:03.584 7f39b93c3e80 20 mgr[py] loaded module option salt_api_password
2019-03-07 09:53:03.584 7f39b93c3e80 10 mgr[py] loaded 4 options
2019-03-07 09:53:03.584 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'deepsea'
2019-03-07 09:53:03.584 7f39b93c3e80 1 mgr[py] Loading python module 'devicehealth'
2019-03-07 09:53:03.603 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.630 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'devicehealth.Module'
2019-03-07 09:53:03.630 7f39b93c3e80 20 mgr[py] loaded command device query-daemon-health-metrics name=who,type=CephString
2019-03-07 09:53:03.630 7f39b93c3e80 20 mgr[py] loaded command device scrape-daemon-health-metrics name=who,type=CephString
2019-03-07 09:53:03.630 7f39b93c3e80 20 mgr[py] loaded command device scrape-health-metrics name=devid,type=CephString,req=False
2019-03-07 09:53:03.630 7f39b93c3e80 20 mgr[py] loaded command device get-health-metrics name=devid,type=CephString name=sample,type=CephString,req=False
2019-03-07 09:53:03.630 7f39b93c3e80 20 mgr[py] loaded command device check-health
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded command device monitoring on
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded command device monitoring off
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded command device predict-life-expectancy name=devid,type=CephString,req=true
2019-03-07 09:53:03.631 7f39b93c3e80 10 mgr[py] loaded 8 commands
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option enable_monitoring
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option scrape_frequency
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option pool_name
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option retention_period
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option mark_out_threshold
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option warn_threshold
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option self_heal
2019-03-07 09:53:03.631 7f39b93c3e80 20 mgr[py] loaded module option sleep_interval
2019-03-07 09:53:03.631 7f39b93c3e80 10 mgr[py] loaded 8 options
2019-03-07 09:53:03.631 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'devicehealth'
2019-03-07 09:53:03.631 7f39b93c3e80 1 mgr[py] Loading python module 'diskprediction_cloud'
2019-03-07 09:53:03.674 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.700 7f39b93c3e80 -1 mgr[py] Module not found: 'diskprediction_cloud'
2019-03-07 09:53:03.700 7f39b93c3e80 -1 mgr[py] Traceback (most recent call last):
File "/ceph/src/pybind/mgr/diskprediction_cloud/__init__.py", line 1, in <module>
from .module import Module
File "/ceph/src/pybind/mgr/diskprediction_cloud/module.py", line 12, in <module>
from string import maketrans
ImportError: cannot import name 'maketrans' from 'string' (/usr/lib64/python3.7/string.py)

2019-03-07 09:53:03.701 7f39b93c3e80 -1 mgr[py] Class not found in module 'diskprediction_cloud'
2019-03-07 09:53:03.701 7f39b93c3e80 -1 mgr[py] Error loading module 'diskprediction_cloud': (2) No such file or directory
2019-03-07 09:53:03.701 7f39b93c3e80 1 mgr[py] Loading python module 'diskprediction_local'
2019-03-07 09:53:03.719 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.748 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'diskprediction_local.Module'
2019-03-07 09:53:03.748 7f39b93c3e80 10 mgr[py] loaded 0 commands
2019-03-07 09:53:03.748 7f39b93c3e80 20 mgr[py] loaded module option sleep_interval
2019-03-07 09:53:03.748 7f39b93c3e80 20 mgr[py] loaded module option predict_interval
2019-03-07 09:53:03.748 7f39b93c3e80 10 mgr[py] loaded 2 options
2019-03-07 09:53:03.748 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'diskprediction_local'
2019-03-07 09:53:03.748 7f39b93c3e80 1 mgr[py] Loading python module 'hello'
2019-03-07 09:53:03.769 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.792 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'hello.Hello'
2019-03-07 09:53:03.792 7f39b93c3e80 20 mgr[py] loaded command hello name=person_name,type=CephString,req=false
2019-03-07 09:53:03.792 7f39b93c3e80 10 mgr[py] loaded 1 commands
2019-03-07 09:53:03.792 7f39b93c3e80 20 mgr[py] loaded module option place
2019-03-07 09:53:03.792 7f39b93c3e80 20 mgr[py] loaded module option emphatic
2019-03-07 09:53:03.792 7f39b93c3e80 10 mgr[py] loaded 2 options
2019-03-07 09:53:03.792 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'hello'
2019-03-07 09:53:03.792 7f39b93c3e80 1 mgr[py] Loading python module 'influx'
2019-03-07 09:53:03.810 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.837 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'influx.Module'
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded command influx config-set name=key,type=CephString name=value,type=CephString
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded command influx config-show
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded command influx send
2019-03-07 09:53:03.837 7f39b93c3e80 10 mgr[py] loaded 3 commands
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option hostname
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option port
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option database
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option username
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option password
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option interval
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option ssl
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option verify_ssl
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option threads
2019-03-07 09:53:03.837 7f39b93c3e80 20 mgr[py] loaded module option batch_size
2019-03-07 09:53:03.837 7f39b93c3e80 10 mgr[py] loaded 10 options
2019-03-07 09:53:03.837 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'influx'
2019-03-07 09:53:03.837 7f39b93c3e80 4 mgr[py] Module influx reported that it cannot run: influxdb python module not found
2019-03-07 09:53:03.837 7f39b93c3e80 1 mgr[py] Loading python module 'insights'
2019-03-07 09:53:03.856 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.882 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'insights.Module'
2019-03-07 09:53:03.882 7f39b93c3e80 20 mgr[py] loaded command insights
2019-03-07 09:53:03.883 7f39b93c3e80 20 mgr[py] loaded command insights prune-health name=hours,type=CephString
2019-03-07 09:53:03.883 7f39b93c3e80 10 mgr[py] loaded 2 commands
2019-03-07 09:53:03.883 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:03.883 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'insights'
2019-03-07 09:53:03.883 7f39b93c3e80 1 mgr[py] Loading python module 'iostat'
2019-03-07 09:53:03.900 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.925 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'iostat.Module'
2019-03-07 09:53:03.925 7f39b93c3e80 20 mgr[py] loaded command iostat
2019-03-07 09:53:03.925 7f39b93c3e80 10 mgr[py] loaded 1 commands
2019-03-07 09:53:03.925 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:03.925 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'iostat'
2019-03-07 09:53:03.925 7f39b93c3e80 1 mgr[py] Loading python module 'localpool'
2019-03-07 09:53:03.947 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:03.975 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'localpool.Module'
2019-03-07 09:53:03.975 7f39b93c3e80 10 mgr[py] loaded 0 commands
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option subtree
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option failure_domain
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option min_size
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option num_rep
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option pg_num
2019-03-07 09:53:03.975 7f39b93c3e80 20 mgr[py] loaded module option prefix
2019-03-07 09:53:03.975 7f39b93c3e80 10 mgr[py] loaded 6 options
2019-03-07 09:53:03.975 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'localpool'
2019-03-07 09:53:03.975 7f39b93c3e80 1 mgr[py] Loading python module 'orchestrator_cli'
2019-03-07 09:53:03.994 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.060 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'orchestrator_cli.OrchestratorCli'
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator host add name=host,type=CephString,req=true
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator host rm name=host,type=CephString,req=true
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator host ls
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator device ls name=host,type=CephString,n=N,req=false name=format,type=CephChoices,strings=json|plain,req=false name=refresh,type=CephBool,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator service ls name=host,type=CephString,req=false name=svc_type,type=CephChoices,strings=mon|mgr|osd|mds|nfs|rgw|rbd-mirror,req=false name=svc_id,type=CephString,req=false name=format,type=CephChoices,strings=json|plain,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator osd create name=svc_arg,type=CephString,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator osd rm name=svc_id,type=CephString,n=N
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator mds add name=svc_arg,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator rgw add name=svc_arg,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator nfs add name=svc_arg,type=CephString name=pool,type=CephString name=namespace,type=CephString,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator mds rm name=svc_id,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator rgw rm name=svc_id,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator nfs rm name=svc_id,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator nfs update name=svc_id,type=CephString name=num,type=CephInt
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator service name=action,type=CephChoices,strings=start|stop|reload name=svc_type,type=CephString name=svc_name,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator service-instance name=action,type=CephChoices,strings=start|stop|reload name=svc_type,type=CephString name=svc_id,type=CephString
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator mgr update name=num,type=CephInt,req=true name=hosts,type=CephString,n=N,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator mon update name=num,type=CephInt,req=true name=hosts,type=CephString,n=N,req=false
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator set backend name=module_name,type=CephString,req=true
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded command orchestrator status
2019-03-07 09:53:04.060 7f39b93c3e80 10 mgr[py] loaded 20 commands
2019-03-07 09:53:04.060 7f39b93c3e80 20 mgr[py] loaded module option orchestrator
2019-03-07 09:53:04.060 7f39b93c3e80 10 mgr[py] loaded 1 options
2019-03-07 09:53:04.060 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'orchestrator_cli'
2019-03-07 09:53:04.060 7f39b93c3e80 1 mgr[py] Loading python module 'osd_perf_query'
2019-03-07 09:53:04.079 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.115 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'osd_perf_query.OSDPerfQuery'
2019-03-07 09:53:04.115 7f39b93c3e80 20 mgr[py] loaded command osd perf query add name=query,type=CephChoices,strings=client_id|rbd_image_id|all_subkeys
2019-03-07 09:53:04.115 7f39b93c3e80 20 mgr[py] loaded command osd perf query remove name=query_id,type=CephInt,req=true
2019-03-07 09:53:04.115 7f39b93c3e80 20 mgr[py] loaded command osd perf counters get name=query_id,type=CephInt,req=true
2019-03-07 09:53:04.115 7f39b93c3e80 10 mgr[py] loaded 3 commands
2019-03-07 09:53:04.115 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:04.115 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'osd_perf_query'
2019-03-07 09:53:04.115 7f39b93c3e80 1 mgr[py] Loading python module 'pg_autoscaler'
2019-03-07 09:53:04.142 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.189 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'pg_autoscaler.PgAutoscaler'
2019-03-07 09:53:04.189 7f39b93c3e80 20 mgr[py] loaded command osd pool autoscale-status
2019-03-07 09:53:04.189 7f39b93c3e80 10 mgr[py] loaded 1 commands
2019-03-07 09:53:04.189 7f39b93c3e80 20 mgr[py] loaded module option sleep_interval
2019-03-07 09:53:04.189 7f39b93c3e80 10 mgr[py] loaded 1 options
2019-03-07 09:53:04.189 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'pg_autoscaler'
2019-03-07 09:53:04.189 7f39b93c3e80 1 mgr[py] Loading python module 'progress'
2019-03-07 09:53:04.208 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.235 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'progress.Module'
2019-03-07 09:53:04.235 7f39b93c3e80 20 mgr[py] loaded command progress
2019-03-07 09:53:04.235 7f39b93c3e80 20 mgr[py] loaded command progress json
2019-03-07 09:53:04.235 7f39b93c3e80 20 mgr[py] loaded command progress clear
2019-03-07 09:53:04.235 7f39b93c3e80 10 mgr[py] loaded 3 commands
2019-03-07 09:53:04.235 7f39b93c3e80 20 mgr[py] loaded module option max_completed_events
2019-03-07 09:53:04.235 7f39b93c3e80 20 mgr[py] loaded module option persist_interval
2019-03-07 09:53:04.235 7f39b93c3e80 10 mgr[py] loaded 2 options
2019-03-07 09:53:04.235 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'progress'
2019-03-07 09:53:04.235 7f39b93c3e80 1 mgr[py] Loading python module 'prometheus'
2019-03-07 09:53:04.253 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.519 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'prometheus.Module'
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded command prometheus file_sd_config
2019-03-07 09:53:04.519 7f39b93c3e80 10 mgr[py] loaded 1 commands
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded module option server_addr
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded module option server_port
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded module option scrape_interval
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded module option rbd_stats_pools
2019-03-07 09:53:04.519 7f39b93c3e80 20 mgr[py] loaded module option rbd_stats_pools_refresh_interval
2019-03-07 09:53:04.519 7f39b93c3e80 10 mgr[py] loaded 5 options
2019-03-07 09:53:04.519 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'prometheus.StandbyModule'
2019-03-07 09:53:04.519 7f39b93c3e80 4 mgr[py] Standby mode available in module 'prometheus'
2019-03-07 09:53:04.519 7f39b93c3e80 1 mgr[py] Loading python module 'rbd_support'
2019-03-07 09:53:04.537 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:04.619 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'rbd_support.Module'
2019-03-07 09:53:04.619 7f39b93c3e80 20 mgr[py] loaded command rbd perf image stats name=pool_spec,type=CephString,req=false name=sort_by,type=CephChoices,strings=write_ops|write_bytes|write_latency|read_ops|read_bytes|read_latency,req=false
2019-03-07 09:53:04.619 7f39b93c3e80 20 mgr[py] loaded command rbd perf image counters name=pool_spec,type=CephString,req=false name=sort_by,type=CephChoices,strings=write_ops|write_bytes|write_latency|read_ops|read_bytes|read_latency,req=false
2019-03-07 09:53:04.619 7f39b93c3e80 10 mgr[py] loaded 2 commands
2019-03-07 09:53:04.619 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:04.619 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'rbd_support'
2019-03-07 09:53:04.619 7f39b93c3e80 1 mgr[py] Loading python module 'restful'
2019-03-07 09:53:04.638 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.072 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'restful.Module'
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded command restful create-key name=key_name,type=CephString
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded command restful delete-key name=key_name,type=CephString
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded command restful list-keys
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded command restful create-self-signed-cert
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded command restful restart
2019-03-07 09:53:05.072 7f39b93c3e80 10 mgr[py] loaded 5 commands
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded module option server_addr
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded module option server_port
2019-03-07 09:53:05.072 7f39b93c3e80 20 mgr[py] loaded module option key_file
2019-03-07 09:53:05.072 7f39b93c3e80 10 mgr[py] loaded 3 options
2019-03-07 09:53:05.072 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'restful'
2019-03-07 09:53:05.072 7f39b93c3e80 1 mgr[py] Loading python module 'rook'
2019-03-07 09:53:05.093 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.136 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'rook.RookOrchestrator'
2019-03-07 09:53:05.136 7f39b93c3e80 10 mgr[py] loaded 0 commands
2019-03-07 09:53:05.136 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:05.136 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'rook'
2019-03-07 09:53:05.136 7f39b93c3e80 4 mgr[py] Module rook reported that it cannot run: `kubernetes` python module not found
2019-03-07 09:53:05.136 7f39b93c3e80 1 mgr[py] Loading python module 'selftest'
2019-03-07 09:53:05.159 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.189 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'selftest.Module'
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test run
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test background start name=workload,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test background stop
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test config get name=key,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test config get_localized name=key,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test remote
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test module name=module,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test health set name=checks,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test health clear name=checks,type=CephString,n=N,req=False
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test insights_set_now_offset name=hours,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded command mgr self-test cluster-log name=channel,type=CephString name=priority,type=CephString name=message,type=CephString
2019-03-07 09:53:05.189 7f39b93c3e80 10 mgr[py] loaded 11 commands
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option testkey
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option testlkey
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option testnewline
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option roption1
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option roption2
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option rwoption1
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option rwoption2
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option rwoption3
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option rwoption4
2019-03-07 09:53:05.189 7f39b93c3e80 20 mgr[py] loaded module option rwoption5
2019-03-07 09:53:05.189 7f39b93c3e80 10 mgr[py] loaded 10 options
2019-03-07 09:53:05.189 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'selftest'
2019-03-07 09:53:05.189 7f39b93c3e80 1 mgr[py] Loading python module 'ssh'
2019-03-07 09:53:05.207 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.260 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'ssh.SSHOrchestrator'
2019-03-07 09:53:05.260 7f39b93c3e80 20 mgr[py] loaded command ssh set-ssh-config
2019-03-07 09:53:05.260 7f39b93c3e80 20 mgr[py] loaded command ssh clear-ssh-config
2019-03-07 09:53:05.260 7f39b93c3e80 10 mgr[py] loaded 2 commands
2019-03-07 09:53:05.260 7f39b93c3e80 20 mgr[py] loaded module option ssh_config_file
2019-03-07 09:53:05.260 7f39b93c3e80 20 mgr[py] loaded module option inventory_cache_timeout_min
2019-03-07 09:53:05.260 7f39b93c3e80 10 mgr[py] loaded 2 options
2019-03-07 09:53:05.260 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'ssh'
2019-03-07 09:53:05.260 7f39b93c3e80 4 mgr[py] Module ssh reported that it cannot run: loading remoto library:No module named 'remoto'
2019-03-07 09:53:05.260 7f39b93c3e80 1 mgr[py] Loading python module 'status'
2019-03-07 09:53:05.278 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.318 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'status.Module'
2019-03-07 09:53:05.318 7f39b93c3e80 20 mgr[py] loaded command fs status name=fs,type=CephString,req=false
2019-03-07 09:53:05.318 7f39b93c3e80 20 mgr[py] loaded command osd status name=bucket,type=CephString,req=false
2019-03-07 09:53:05.318 7f39b93c3e80 10 mgr[py] loaded 2 commands
2019-03-07 09:53:05.318 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:05.318 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'status'
2019-03-07 09:53:05.318 7f39b93c3e80 1 mgr[py] Loading python module 'telegraf'
2019-03-07 09:53:05.337 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.368 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'telegraf.Module'
2019-03-07 09:53:05.368 7f39b93c3e80 20 mgr[py] loaded command telegraf config-set name=key,type=CephString name=value,type=CephString
2019-03-07 09:53:05.368 7f39b93c3e80 20 mgr[py] loaded command telegraf config-show
2019-03-07 09:53:05.368 7f39b93c3e80 20 mgr[py] loaded command telegraf send
2019-03-07 09:53:05.368 7f39b93c3e80 10 mgr[py] loaded 3 commands
2019-03-07 09:53:05.368 7f39b93c3e80 20 mgr[py] loaded module option address
2019-03-07 09:53:05.368 7f39b93c3e80 20 mgr[py] loaded module option interval
2019-03-07 09:53:05.368 7f39b93c3e80 10 mgr[py] loaded 2 options
2019-03-07 09:53:05.368 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'telegraf'
2019-03-07 09:53:05.368 7f39b93c3e80 1 mgr[py] Loading python module 'telemetry'
2019-03-07 09:53:05.386 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.490 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'telemetry.Module'
2019-03-07 09:53:05.490 7f39b93c3e80 20 mgr[py] loaded command telemetry config-set name=key,type=CephString name=value,type=CephString
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded command telemetry config-show
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded command telemetry send
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded command telemetry show
2019-03-07 09:53:05.491 7f39b93c3e80 10 mgr[py] loaded 4 commands
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option url
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option enabled
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option leaderboard
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option description
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option contact
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option organization
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option proxy
2019-03-07 09:53:05.491 7f39b93c3e80 20 mgr[py] loaded module option interval
2019-03-07 09:53:05.491 7f39b93c3e80 10 mgr[py] loaded 8 options
2019-03-07 09:53:05.491 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'telemetry'
2019-03-07 09:53:05.491 7f39b93c3e80 1 mgr[py] Loading python module 'test_orchestrator'
2019-03-07 09:53:05.514 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.556 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'test_orchestrator.TestOrchestrator'
2019-03-07 09:53:05.556 7f39b93c3e80 10 mgr[py] loaded 0 commands
2019-03-07 09:53:05.556 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:05.556 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'test_orchestrator'
2019-03-07 09:53:05.556 7f39b93c3e80 1 mgr[py] Loading python module 'volumes'
2019-03-07 09:53:05.575 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.710 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'volumes.Module'
2019-03-07 09:53:05.710 7f39b93c3e80 20 mgr[py] loaded command fs volume ls
2019-03-07 09:53:05.710 7f39b93c3e80 20 mgr[py] loaded command fs volume create name=name,type=CephString name=size,type=CephString,req=false
2019-03-07 09:53:05.710 7f39b93c3e80 20 mgr[py] loaded command fs volume rm name=vol_name,type=CephString
2019-03-07 09:53:05.710 7f39b93c3e80 20 mgr[py] loaded command fs subvolume create name=vol_name,type=CephString name=sub_name,type=CephString name=size,type=CephString,req=false
2019-03-07 09:53:05.710 7f39b93c3e80 20 mgr[py] loaded command fs subvolume rm name=vol_name,type=CephString name=sub_name,type=CephString
2019-03-07 09:53:05.710 7f39b93c3e80 10 mgr[py] loaded 5 commands
2019-03-07 09:53:05.710 7f39b93c3e80 10 mgr[py] loaded 0 options
2019-03-07 09:53:05.710 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'volumes'
2019-03-07 09:53:05.710 7f39b93c3e80 1 mgr[py] Loading python module 'zabbix'
2019-03-07 09:53:05.740 7f39b93c3e80 10 mgr[py] Computed sys.path '/tmp/tmp.NEUJphaujw/teuthology:/ceph/src/pybind/mgr/dashboard/../../../../build/../qa:/ceph/src/pybind/mgr/dashboard/../../../../build/lib/cython_modules/lib.3/:/ceph/src/pybind/mgr/dashboard/../../../../build/../src/pybind:/usr/lib64/python37.zip:/usr/lib64/python3.7:/usr/lib64/python3.7/lib-dynload:/ceph/src/pybind/mgr:/usr/local/lib64/python3.7/site-packages:/usr/local/lib/python3.7/site-packages:/usr/lib64/python3.7/site-packages:/usr/lib/python3.7/site-packages'
2019-03-07 09:53:05.778 7f39b93c3e80 4 mgr[py] load_subclass_of: found class: 'zabbix.Module'
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded command zabbix config-set name=key,type=CephString name=value,type=CephString
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded command zabbix config-show
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded command zabbix send
2019-03-07 09:53:05.778 7f39b93c3e80 10 mgr[py] loaded 3 commands
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded module option zabbix_sender
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded module option zabbix_host
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded module option zabbix_port
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded module option identifier
2019-03-07 09:53:05.778 7f39b93c3e80 20 mgr[py] loaded module option interval
2019-03-07 09:53:05.778 7f39b93c3e80 10 mgr[py] loaded 5 options
2019-03-07 09:53:05.778 7f39b93c3e80 4 mgr[py] Standby mode not provided by module 'zabbix'
2019-03-07 09:53:05.778 7f39b93c3e80 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: diskprediction_cloud
2019-03-07 09:53:05.778 7f39b93c3e80 10 mgr tick tick
2019-03-07 09:53:05.778 7f39b93c3e80 4 mgr send_beacon standby
2019-03-07 09:53:05.779 7f39b93c3e80 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:05.779 7f39b93c3e80 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:05.779 7f39b93c3e80 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, , 0) v8 -- 0x56519890c2c0 con 0x5651957f8480
2019-03-07 09:53:05.779 7f39b93c3e80 4 mgr init Complete.
2019-03-07 09:53:05.779 7f39b37dd700 4 mgr ms_dispatch standby mgrmap(e 11) v1
2019-03-07 09:53:05.779 7f39b37dd700 4 mgr handle_mgr_map received map epoch 11
2019-03-07 09:53:05.779 7f39b37dd700 4 mgr handle_mgr_map active in map: 0 active is 0
2019-03-07 09:53:05.780 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 4 ==== mon_map magic: 0 v1 ==== 377+0+0 (crc 0 0 0) 0x5651957f6400 con 0x5651957f8480
2019-03-07 09:53:05.780 7f39b37dd700 10 monclient: handle_monmap mon_map magic: 0 v1
2019-03-07 09:53:05.780 7f39b37dd700 10 monclient: got monmap 1 from mon.a (according to old e1)
2019-03-07 09:53:05.780 7f39b37dd700 10 monclient: dump:
epoch 1
fsid 83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0
last_changed 2019-03-07 09:51:10.636488
created 2019-03-07 09:51:10.636488
min_mon_release 14 (nautilus)
0: [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] mon.a
1: [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] mon.b
2: [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] mon.c

2019-03-07 09:53:05.780 7f39b37dd700 4 mgr ms_dispatch standby mon_map magic: 0 v1
2019-03-07 09:53:05.780 7f39b37dd700 0 ms_deliver_dispatch: unhandled message 0x5651957f6400 mon_map magic: 0 v1 from mon.0 v2:172.20.0.2:10000/0
2019-03-07 09:53:05.780 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 5 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (crc 0 0 0) 0x5651957b2240 con 0x5651957f8480
2019-03-07 09:53:05.780 7f39b37dd700 10 monclient: _finish_auth 0
2019-03-07 09:53:05.780 7f39b37dd700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-03-07 09:52:35.781427)
2019-03-07 09:53:05.780 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 6 ==== osd_map(23..23 src has 1..23) v4 ==== 3774+0+0 (crc 0 0 0) 0x565195757180 con 0x5651957f8480
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr ms_dispatch standby osd_map(23..23 src has 1..23) v4
2019-03-07 09:53:05.781 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 7 ==== mgrmap(e 12) v1 ==== 12764+0+0 (crc 0 0 0) 0x5651988c2000 con 0x5651957f8480
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr ms_dispatch standby mgrmap(e 12) v1
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr handle_mgr_map received map epoch 12
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr handle_mgr_map active in map: 0 active is 4448
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] Starting modules in standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'balancer' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'crash' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'devicehealth' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'orchestrator_cli' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'progress' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'restful' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'status' because it does not implement a standby mode
2019-03-07 09:53:05.781 7f39b37dd700 4 mgr[py] skipping module 'volumes' because it does not implement a standby mode
2019-03-07 09:53:06.808 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 8 ==== mgrmap(e 13) v1 ==== 38870+0+0 (crc 0 0 0) 0x5651988c2300 con 0x5651957f8480
2019-03-07 09:53:06.808 7f39b37dd700 4 mgr ms_dispatch standby mgrmap(e 13) v1
2019-03-07 09:53:06.808 7f39b37dd700 4 mgr handle_mgr_map received map epoch 13
2019-03-07 09:53:06.808 7f39b37dd700 4 mgr handle_mgr_map active in map: 0 active is 4448
2019-03-07 09:53:06.811 7f39b37dd700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:06.815 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:06.815 7f39b6fe4700 10 monclient: get_auth_request con 0x56519897a000 auth_method 0
2019-03-07 09:53:06.816 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] -- mgropen(unknown.x) v3 -- 0x5651988c2000 con 0x56519897a000
2019-03-07 09:53:06.816 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).ready entity=mgr.4448 client_cookie=38eb37845167977c server_cookie=0 in_seq=0 out_seq=0
2019-03-07 09:53:06.821 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mgr.4448 v2:172.20.0.2:6800/4184 1 ==== mgrconfigure(period=5, threshold=5) v3 ==== 12+0+0 (crc 0 0 0) 0x565194ad83c0 con 0x56519897a000
2019-03-07 09:53:06.821 7f39b37dd700 4 mgr ms_dispatch standby mgrconfigure(period=5, threshold=5) v3
2019-03-07 09:53:06.822 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] -- mgrreport(unknown.x +4-0 packed 38) v7 -- 0x565198982380 con 0x56519897a000
2019-03-07 09:53:06.825 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 29
2019-03-07 09:53:06.825 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed
2019-03-07 09:53:06.825 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted)
2019-03-07 09:53:06.825 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=1 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:06.825 7f39b37dd700 0 client.0 ms_handle_reset on v2:172.20.0.2:6800/4184
2019-03-07 09:53:06.825 7f39b37dd700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 unknown :-1 s=STATE_CLOSED l=1).mark_down
2019-03-07 09:53:06.825 7f39b37dd700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:07.778 7f39affd6700 10 mgr tick tick
2019-03-07 09:53:07.778 7f39affd6700 4 mgr send_beacon standby
2019-03-07 09:53:07.779 7f39affd6700 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:07.781 7f39affd6700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:07.781 7f39affd6700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, , 0) v8 -- 0x5651957bedc0 con 0x5651957f8480
2019-03-07 09:53:07.812 7f39b17d9700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:07.812 7f39b17d9700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] -- mgropen(unknown.x) v3 -- 0x565194b02f00 con 0x56519897a000
2019-03-07 09:53:07.812 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:07.813 7f39b6fe4700 10 monclient: get_auth_request con 0x56519897a000 auth_method 0
2019-03-07 09:53:07.816 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=13 cs=0 l=1 rx=0 tx=0).ready entity=mgr.4448 client_cookie=f42058ec9ede8669 server_cookie=0 in_seq=0 out_seq=0
2019-03-07 09:53:07.817 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mgr.4448 v2:172.20.0.2:6800/4184 1 ==== mgrconfigure(period=5, threshold=5) v3 ==== 12+0+0 (crc 0 0 0) 0x565194ad9a40 con 0x56519897a000
2019-03-07 09:53:07.817 7f39b37dd700 4 mgr ms_dispatch standby mgrconfigure(period=5, threshold=5) v3
2019-03-07 09:53:07.818 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] -- mgrreport(unknown.x +4-0 packed 38) v7 -- 0x565198982700 con 0x56519897a000
2019-03-07 09:53:09.781 7f39affd6700 10 mgr tick tick
2019-03-07 09:53:09.781 7f39affd6700 4 mgr send_beacon standby
2019-03-07 09:53:09.781 7f39affd6700 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:09.781 7f39affd6700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:09.781 7f39affd6700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, , 0) v8 -- 0x5651957beb00 con 0x5651957f8480
2019-03-07 09:53:10.994 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 29
2019-03-07 09:53:10.994 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed
2019-03-07 09:53:10.994 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=13 cs=0 l=1 rx=0 tx=0).handle_read_frame_preamble_main read frame length and tag failed r=-1 ((1) Operation not permitted)
2019-03-07 09:53:10.994 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 crc :-1 s=READY pgs=13 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:10.995 7f39b37dd700 0 client.0 ms_handle_reset on v2:172.20.0.2:6800/4184
2019-03-07 09:53:10.995 7f39b37dd700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 msgr2=0x565198982000 unknown :-1 s=STATE_CLOSED l=1).mark_down
2019-03-07 09:53:10.995 7f39b37dd700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a000 0x565198982000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:10.995 7f39b37dd700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:10.996 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] -- mgropen(unknown.x) v3 -- 0x5651988c2300 con 0x56519897a480
2019-03-07 09:53:10.997 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.20.0.2:6800/4184
2019-03-07 09:53:10.997 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.200000
2019-03-07 09:53:11.197 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.20.0.2:6800/4184
2019-03-07 09:53:11.198 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.400000
2019-03-07 09:53:11.598 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.20.0.2:6800/4184
2019-03-07 09:53:11.598 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 0.800000
2019-03-07 09:53:11.782 7f39affd6700 10 mgr tick tick
2019-03-07 09:53:11.782 7f39affd6700 4 mgr send_beacon standby
2019-03-07 09:53:11.782 7f39affd6700 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:11.794 7f39affd6700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:11.794 7f39affd6700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, , 0) v8 -- 0x5651957bf080 con 0x5651957f8480
2019-03-07 09:53:12.399 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.20.0.2:6800/4184
2019-03-07 09:53:12.399 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 1.600000
2019-03-07 09:53:12.593 7f39b27db700 10 monclient: tick
2019-03-07 09:53:12.593 7f39b27db700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-03-07 09:52:42.595183)
2019-03-07 09:53:12.593 7f39b27db700 10 log_client log_queue is 1 last_log 1 sent 0 num 1 unsent 1 sending 1
2019-03-07 09:53:12.593 7f39b27db700 10 log_client will send 2019-03-07 09:53:05.779273 mgr.x (mgr.4455) 1 : cluster [ERR] Failed to load ceph-mgr modules: diskprediction_cloud
2019-03-07 09:53:12.593 7f39b27db700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:12.593 7f39b27db700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- log(1 entries from seq 1 at 2019-03-07 09:53:05.779273) v1 -- 0x56519899e900 con 0x5651957f8480
2019-03-07 09:53:12.593 7f39b27db700 20 monclient: _un_backoff reopen_interval_multipler now 1
2019-03-07 09:53:13.487 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 9 ==== log(last 1) v1 ==== 28+0+0 (crc 0 0 0) 0x5651989d2000 con 0x5651957f8480
2019-03-07 09:53:13.487 7f39b37dd700 10 log_client handle_log_ack log(last 1) v1
2019-03-07 09:53:13.487 7f39b37dd700 10 log_client logged 2019-03-07 09:53:05.779273 mgr.x (mgr.4455) 1 : cluster [ERR] Failed to load ceph-mgr modules: diskprediction_cloud
2019-03-07 09:53:13.795 7f39affd6700 10 mgr tick tick
2019-03-07 09:53:13.795 7f39affd6700 4 mgr send_beacon standby
2019-03-07 09:53:13.795 7f39affd6700 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:13.808 7f39affd6700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:13.808 7f39affd6700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, , 0) v8 -- 0x5651957beb00 con 0x5651957f8480
2019-03-07 09:53:14.000 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.20.0.2:6800/4184
2019-03-07 09:53:14.000 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0)._fault waiting 3.200000
2019-03-07 09:53:14.110 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 10 ==== mgrmap(e 14) v1 ==== 25782+0+0 (crc 0 0 0) 0x565194b02f00 con 0x5651957f8480
2019-03-07 09:53:14.110 7f39b37dd700 4 mgr ms_dispatch standby mgrmap(e 14) v1
2019-03-07 09:53:14.110 7f39b37dd700 4 mgr handle_mgr_map received map epoch 14
2019-03-07 09:53:14.110 7f39b37dd700 4 mgr handle_mgr_map active in map: 1 active is 4455
2019-03-07 09:53:14.111 7f39b37dd700 1 mgr handle_mgr_map Activating!
2019-03-07 09:53:14.112 7f39b37dd700 1 mgr handle_mgr_map I am now activating
2019-03-07 09:53:14.112 7f39b37dd700 4 mgr ms_dispatch mgrmap(e 14) v1
2019-03-07 09:53:14.113 7f39a648c700 20 mgr.server init init will bind to [v2:0.0.0.0:0/0,v1:0.0.0.0:0/0]
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:0.0.0.0:6800/4214,v1:0.0.0.0:6801/4214] _finish_bind bind my_addrs is [v2:0.0.0.0:6800/4214,v1:0.0.0.0:6801/4214]
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:0.0.0.0:6800/4214,v1:0.0.0.0:6801/4214] set_addr_unknowns 172.20.0.2:0/4214
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:0.0.0.0:6800/4214,v1:0.0.0.0:6801/4214] set_addr_unknowns assuming my addr v2:0.0.0.0:6800/4214 matches provided addr 172.20.0.2:0/4214
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:0.0.0.0:6800/4214,v1:0.0.0.0:6801/4214] set_addr_unknowns assuming my addr v1:0.0.0.0:6801/4214 matches provided addr 172.20.0.2:0/4214
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214] set_addr_unknowns now [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214]
2019-03-07 09:53:14.114 7f39a648c700 1 -- [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214] start start
2019-03-07 09:53:14.114 7f39a648c700 1 Processor -- start
2019-03-07 09:53:14.114 7f39a648c700 4 mgr init Initialized server at [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214]
2019-03-07 09:53:14.114 7f39a648c700 4 mgr init Loading daemon metadata...
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_command 1 [{"prefix": "mds metadata"}]
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:14.114 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_command({"prefix": "mds metadata"} v 0) v1 -- 0x565194b7f600 con 0x5651957f8480
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_command 2 [{"prefix": "osd metadata"}]
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:14.114 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_command({"prefix": "osd metadata"} v 0) v1 -- 0x565194b7f400 con 0x5651957f8480
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_command 3 [{"prefix": "mon metadata"}]
2019-03-07 09:53:14.114 7f39a648c700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:14.114 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_command({"prefix": "mon metadata"} v 0) v1 -- 0x565194b7e400 con 0x5651957f8480
2019-03-07 09:53:14.115 7f39b37dd700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 msgr2=0x565198982380 unknown :-1 s=STATE_CONNECTING l=0).mark_down
2019-03-07 09:53:14.115 7f39b37dd700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:6800/4184,v1:172.20.0.2:6801/4184] conn(0x56519897a480 0x565198982380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:14.119 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 11 ==== mon_command_ack([{"prefix": "mds metadata"}]=0 v3) v1 ==== 60+0+728 (crc 0 0 0) 0x56519899e900 con 0x5651957f8480
2019-03-07 09:53:14.119 7f39b37dd700 10 monclient: handle_mon_command_ack 1 [{"prefix": "mds metadata"}]
2019-03-07 09:53:14.119 7f39b37dd700 10 monclient: _finish_command 1 = 0
2019-03-07 09:53:14.121 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 12 ==== mon_command_ack([{"prefix": "osd metadata"}]=0 v23) v1 ==== 60+0+9635 (crc 0 0 0) 0x56519899f680 con 0x5651957f8480
2019-03-07 09:53:14.121 7f39b37dd700 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd metadata"}]
2019-03-07 09:53:14.121 7f39b37dd700 10 monclient: _finish_command 2 = 0
2019-03-07 09:53:14.122 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.0 v2:172.20.0.2:10000/0 13 ==== mon_command_ack([{"prefix": "mon metadata"}]=0 v0) v1 ==== 60+0+2487 (crc 0 0 0) 0x56519899f8c0 con 0x5651957f8480
2019-03-07 09:53:14.122 7f39b37dd700 10 monclient: handle_mon_command_ack 3 [{"prefix": "mon metadata"}]
2019-03-07 09:53:14.122 7f39b37dd700 10 monclient: _finish_command 3 = 0
2019-03-07 09:53:14.129 7f39a648c700 4 mgr load_all_metadata ceph.dev
2019-03-07 09:53:14.129 7f39a648c700 4 mgr load_all_metadata ceph.dev
2019-03-07 09:53:14.129 7f39a648c700 4 mgr load_all_metadata ceph.dev
2019-03-07 09:53:14.130 7f39a648c700 4 mgr load_all_metadata ceph.dev
2019-03-07 09:53:14.130 7f39a648c700 4 mgr init waiting for OSDMap...
2019-03-07 09:53:14.130 7f39a648c700 10 monclient: _renew_subs
2019-03-07 09:53:14.130 7f39a648c700 10 monclient: _send_mon_message to mon.a at v2:172.20.0.2:10000/0
2019-03-07 09:53:14.130 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_subscribe({fsmap=0+,log-info=0+,mgrdigest=0+,osdmap=24,servicemap=0+}) v3 -- 0x565194b7ec00 con 0x5651957f8480
2019-03-07 09:53:14.130 7f39a648c700 10 monclient: _reopen_session rank -1
2019-03-07 09:53:14.130 7f39a648c700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 msgr2=0x565195746e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2019-03-07 09:53:14.130 7f39a648c700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x5651957f8480 0x565195746e00 crc :-1 s=READY pgs=44 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:14.130 7f39a648c700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): picked mon.b con 0x5651957f8000 addr [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0]
2019-03-07 09:53:14.130 7f39a648c700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x56519897b200 0x565198982e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): picked mon.a con 0x56519897b200 addr [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0]
2019-03-07 09:53:14.130 7f39a648c700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rx=0 tx=0).connect
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): picked mon.c con 0x5651957f8d80 addr [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0]
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:14.130 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] -- mon_getmap magic: 0 v1 -- 0x565194b96540 con 0x56519897b200
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:14.130 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_getmap magic: 0 v1 -- 0x565194b96700 con 0x5651957f8000
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): start opening mon connection
2019-03-07 09:53:14.130 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] -- mon_getmap magic: 0 v1 -- 0x565194b96c40 con 0x5651957f8d80
2019-03-07 09:53:14.130 7f39a648c700 10 monclient(hunting): _renew_subs
2019-03-07 09:53:14.131 7f39a648c700 10 mgr notify_osdmap v1
2019-03-07 09:53:14.131 7f39a648c700 10 check_osd_map pool 1 pg_num 8 > my pg_num 0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.1
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.2
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.3
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.4
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.5
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.6
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 1.7
2019-03-07 09:53:14.131 7f39a648c700 10 check_osd_map pool 2 pg_num 8 > my pg_num 0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.1
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.2
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.3
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.4
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.5
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.6
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 2.7
2019-03-07 09:53:14.131 7f39a648c700 10 check_osd_map pool 3 pg_num 8 > my pg_num 0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.1
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.2
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.3
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.4
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.5
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.6
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 3.7
2019-03-07 09:53:14.131 7f39a648c700 10 check_osd_map pool 4 pg_num 8 > my pg_num 0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.0
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.1
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.2
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.3
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.4
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.5
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.6
2019-03-07 09:53:14.131 7f39a648c700 20 check_osd_map adding 4.7
2019-03-07 09:53:14.131 7f39a648c700 4 mgr init waiting for FSMap...
2019-03-07 09:53:14.131 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:14.131 7f39b6fe4700 10 monclient(hunting): get_auth_request con 0x5651957f8000 auth_method 0
2019-03-07 09:53:14.131 7f39b6fe4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1]
2019-03-07 09:53:14.131 7f39b6fe4700 10 monclient(hunting): _init_auth method 2
2019-03-07 09:53:14.132 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0)._handle_peer_banner_payload supported=0 required=0
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): get_auth_request con 0x5651957f8d80 auth_method 0
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1]
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): _init_auth method 2
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload 9
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload_len 9
2019-03-07 09:53:14.132 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload 9
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more payload_len 9
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient(hunting): handle_auth_done global_id 4455 payload 1102
2019-03-07 09:53:14.133 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 msgr2=0x565195746700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
2019-03-07 09:53:14.133 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] conn(0x5651957f8d80 0x565195746700 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:14.133 7f39b6fe4700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x56519897b200 msgr2=0x565198982e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down
2019-03-07 09:53:14.133 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] conn(0x56519897b200 0x565198982e00 unknown :-1 s=CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).stop
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient: _finish_hunting 0
2019-03-07 09:53:14.133 7f39b6fe4700 1 monclient: found mon.b
2019-03-07 09:53:14.133 7f39b6fe4700 20 monclient: _un_backoff reopen_interval_multipler now 1
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.133 7f39b6fe4700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_subscribe({config=0+,fsmap=0+,log-info=0+,mgrdigest=0+,mgrmap=0+,monmap=2+,osdmap=24,servicemap=0+}) v3 -- 0x565194b7ec00 con 0x5651957f8000
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient: _finish_auth 0
2019-03-07 09:53:14.133 7f39b6fe4700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-03-07 09:52:44.135525)
2019-03-07 09:53:14.135 7f39b6fe4700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 crc :-1 s=READY pgs=51 cs=0 l=1 rx=0 tx=0).ready entity=mon.1 client_cookie=b45740a0c74dd7b7 server_cookie=0 in_seq=0 out_seq=0
2019-03-07 09:53:14.140 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 1 ==== mon_map magic: 0 v1 ==== 377+0+0 (crc 0 0 0) 0x565198a0aa00 con 0x5651957f8000
2019-03-07 09:53:14.140 7f39b37dd700 10 monclient: handle_monmap mon_map magic: 0 v1
2019-03-07 09:53:14.140 7f39b37dd700 10 monclient: got monmap 1 from mon.b (according to old e1)
2019-03-07 09:53:14.140 7f39b37dd700 10 monclient: dump:
epoch 1
fsid 83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0
last_changed 2019-03-07 09:51:10.636488
created 2019-03-07 09:51:10.636488
min_mon_release 14 (nautilus)
0: [v2:172.20.0.2:10000/0,v1:172.20.0.2:10001/0] mon.a
1: [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] mon.b
2: [v2:172.20.0.2:10004/0,v1:172.20.0.2:10005/0] mon.c

2019-03-07 09:53:14.140 7f39b37dd700 4 mgr ms_dispatch active (starting) mon_map magic: 0 v1
2019-03-07 09:53:14.140 7f39b37dd700 4 mgr ms_dispatch mon_map magic: 0 v1
2019-03-07 09:53:14.140 7f39b37dd700 20 mgr handle_mon_map handle_mon_map
2019-03-07 09:53:14.140 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 2 ==== config(13 keys) v1 ==== 420+0+0 (crc 0 0 0) 0x5651989d2000 con 0x5651957f8000
2019-03-07 09:53:14.140 7f39b37dd700 10 monclient: handle_config config(13 keys) v1
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: debug_mgr : 20/20
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored debug_mgr
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals debug_mgr = 20/20 (no change)
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: debug_mon : 20/20
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored debug_mon
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals debug_mon = 20/20 (no change)
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: debug_monc : 20/20
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored debug_monc
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals debug_monc = 20/20 (no change)
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: debug_ms : 1/1
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored debug_ms
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals debug_ms = 1/1 (no change)
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/x/server_port : 11000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/x/server_port:11000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/x/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/y/server_port : 13000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/y/server_port:13000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/y/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/dashboard/z/server_port : 15000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/dashboard/z/server_port:15000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/dashboard/z/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/x/server_port : 12000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/x/server_port:12000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/x/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/y/server_port : 14000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/y/server_port:14000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/y/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mgr/restful/z/server_port : 16000
2019-03-07 09:53:14.140 7f39b1fda700 4 mgr[py] Loaded module_config entry mgr/restful/z/server_port:16000
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback consumed mgr/restful/z/server_port
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: mon_pg_warn_min_per_osd : 3
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored mon_pg_warn_min_per_osd
2019-03-07 09:53:14.140 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 3 ==== fsmap(e 3) v1 ==== 557+0+0 (crc 0 0 0) 0x565198983180 con 0x5651957f8000
2019-03-07 09:53:14.140 7f39b37dd700 4 mgr ms_dispatch active (starting) fsmap(e 3) v1
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals mon_pg_warn_min_per_osd = 3 (no change)
2019-03-07 09:53:14.140 7f39b37dd700 4 mgr ms_dispatch fsmap(e 3) v1
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: osd_pool_default_min_size : 1
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored osd_pool_default_min_size
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals osd_pool_default_min_size = 1 (no change)
2019-03-07 09:53:14.140 7f39b1fda700 10 mgr operator() config_callback: osd_pool_default_size : 3
2019-03-07 09:53:14.140 7f39b1fda700 4 set_mon_vals callback ignored osd_pool_default_size
2019-03-07 09:53:14.140 7f39b1fda700 20 set_mon_vals osd_pool_default_size = 3 (no change)
2019-03-07 09:53:14.140 7f39a648c700 4 mgr init waiting for config-keys...
2019-03-07 09:53:14.140 7f39a648c700 4 mgr init waiting for MgrDigest...
2019-03-07 09:53:14.141 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 4 ==== log(3 entries from seq 1 at 2019-03-07 09:53:05.779273) v1 ==== 580+0+0 (crc 0 0 0) 0x56519899fb00 con 0x5651957f8000
2019-03-07 09:53:14.141 7f39b37dd700 4 mgr ms_dispatch active (starting) log(3 entries from seq 1 at 2019-03-07 09:53:05.779273) v1
2019-03-07 09:53:14.141 7f39b37dd700 4 mgr ms_dispatch log(3 entries from seq 1 at 2019-03-07 09:53:05.779273) v1
2019-03-07 09:53:14.141 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 5 ==== mgrdigest v1 ==== 1669+0+0 (crc 0 0 0) 0x56519899fd40 con 0x5651957f8000
2019-03-07 09:53:14.141 7f39b37dd700 4 mgr ms_dispatch active (starting) mgrdigest v1
2019-03-07 09:53:14.141 7f39b37dd700 4 mgr ms_dispatch mgrdigest v1
2019-03-07 09:53:14.141 7f39b37dd700 10 mgr handle_mgr_digest 1627
2019-03-07 09:53:14.141 7f39b37dd700 10 mgr handle_mgr_digest 34
2019-03-07 09:53:14.141 7f39b37dd700 10 mgr handle_mgr_digest done.
2019-03-07 09:53:14.141 7f39a648c700 10 mgr load_store listing keys
2019-03-07 09:53:14.141 7f39a648c700 10 monclient: _send_command 4 [{"prefix": "config-key ls"}]
2019-03-07 09:53:14.141 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.141 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key ls"} v 0) v1 -- 0x5651957f7600 con 0x5651957f8000
2019-03-07 09:53:14.144 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 6 ==== mgrmap(e 14) v1 ==== 25782+0+0 (crc 0 0 0) 0x5651989e7b00 con 0x5651957f8000
2019-03-07 09:53:14.144 7f39b37dd700 4 mgr ms_dispatch active (starting) mgrmap(e 14) v1
2019-03-07 09:53:14.144 7f39b37dd700 4 mgr handle_mgr_map received map epoch 14
2019-03-07 09:53:14.144 7f39b37dd700 4 mgr handle_mgr_map active in map: 1 active is 4455
2019-03-07 09:53:14.145 7f39b37dd700 10 mgr handle_mgr_map I was already active
2019-03-07 09:53:14.145 7f39b37dd700 10 mgr got_mgr_map x(active, starting, since 1.47312s), standbys: z
2019-03-07 09:53:14.145 7f39b37dd700 10 monclient: _send_command 5 [{"prefix": "mgr metadata", "who": "x", "id": "x"}]
2019-03-07 09:53:14.145 7f39b37dd700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.145 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "mgr metadata", "who": "x", "id": "x"} v 0) v1 -- 0x565198a0aa00 con 0x5651957f8000
2019-03-07 09:53:14.145 7f39b37dd700 10 mgr.server operator() triggered addition of mgr,x via metadata update
2019-03-07 09:53:14.145 7f39b37dd700 10 monclient: _send_command 6 [{"prefix": "mgr metadata", "who": "z", "id": "z"}]
2019-03-07 09:53:14.145 7f39b37dd700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.145 7f39b37dd700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "mgr metadata", "who": "z", "id": "z"} v 0) v1 -- 0x565198a0b200 con 0x5651957f8000
2019-03-07 09:53:14.145 7f39b37dd700 10 mgr.server operator() triggered addition of mgr,z via metadata update
2019-03-07 09:53:14.145 7f39b37dd700 4 mgr ms_dispatch mgrmap(e 14) v1
2019-03-07 09:53:14.146 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 7 ==== service_map(e2 1 svc) v1 ==== 879+0+0 (crc 0 0 0) 0x5651957f7400 con 0x5651957f8000
2019-03-07 09:53:14.146 7f39b37dd700 4 mgr ms_dispatch active (starting) service_map(e2 1 svc) v1
2019-03-07 09:53:14.146 7f39b37dd700 4 mgr ms_dispatch service_map(e2 1 svc) v1
2019-03-07 09:53:14.146 7f39b37dd700 10 mgr handle_service_map e2
2019-03-07 09:53:14.146 7f39b37dd700 10 mgr.server operator() got initial map e2
2019-03-07 09:53:14.146 7f39b37dd700 10 mgr.server got_service_map added missing rgw,rgw
2019-03-07 09:53:14.146 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 8 ==== mon_command_ack([{"prefix": "config-key ls"}]=0 v0) v1 ==== 61+0+3962 (crc 0 0 0) 0x565198a84000 con 0x5651957f8000
2019-03-07 09:53:14.146 7f39b37dd700 10 monclient: handle_mon_command_ack 4 [{"prefix": "config-key ls"}]
2019-03-07 09:53:14.146 7f39b37dd700 10 monclient: _finish_command 4 = 0
2019-03-07 09:53:14.146 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 9 ==== mon_command_ack([{"prefix": "mgr metadata", "who": "x", "id": "x"}]=0 v14) v1 ==== 83+0+637 (crc 0 0 0) 0x565198a84240 con 0x5651957f8000
2019-03-07 09:53:14.147 7f39b37dd700 10 monclient: handle_mon_command_ack 5 [{"prefix": "mgr metadata", "who": "x", "id": "x"}]
2019-03-07 09:53:14.147 7f39b37dd700 10 monclient: _finish_command 5 = 0
2019-03-07 09:53:14.147 7f39b1fda700 4 mgr finish mon returned valid metadata JSON for mgr.x
2019-03-07 09:53:14.147 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 10 ==== mon_command_ack([{"prefix": "mgr metadata", "who": "z", "id": "z"}]=0 v14) v1 ==== 83+0+637 (crc 0 0 0) 0x565198a84480 con 0x5651957f8000
2019-03-07 09:53:14.147 7f39b37dd700 10 monclient: handle_mon_command_ack 6 [{"prefix": "mgr metadata", "who": "z", "id": "z"}]
2019-03-07 09:53:14.147 7f39b37dd700 10 monclient: _finish_command 6 = 0
2019-03-07 09:53:14.148 7f39b1fda700 4 mgr finish mon returned valid metadata JSON for mgr.z
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/1/'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+global/mon_pg_warn_min_per_osd'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+global/osd_pool_default_min_size'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+global/osd_pool_default_size'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mds/mds_debug_auth_pins'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mds/mds_debug_frag'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mds/mds_debug_subtrees'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mon/mon_allow_pool_delete'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mon/mon_data_avail_crit'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mon/mon_data_avail_warn'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+mon/mon_osd_reporter_subtree_level'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+osd/osd_copyfrom_max_chunk'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+osd/osd_debug_misdirected_ops'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+osd/osd_debug_op_order'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/2/+osd/osd_scrub_load_threshold'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/debug_mds'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/debug_mgrc'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/debug_monc'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/debug_ms'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/mds_debug_scatterstat'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mds/mds_verify_scatter'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mgr/debug_mgr'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mgr/debug_mon'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mgr/debug_monc'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+mgr/debug_ms'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_bdev'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_bluefs'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_bluestore'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_filestore'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_journal'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_mgrc'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_monc'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_ms'
2019-03-07 09:53:14.148 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_objclass'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_objecter'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_osd'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_reserver'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/3/+osd/debug_rocksdb'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/4/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/4/+mgr/mgr/dashboard/x/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/5/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/5/+mgr/mgr/restful/x/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/6/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/6/+mgr/mgr/dashboard/y/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/7/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/7/+mgr/mgr/restful/y/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/8/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/8/+mgr/mgr/dashboard/z/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/9/'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config-history/9/+mgr/mgr/restful/z/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/global/mon_pg_warn_min_per_osd'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/global/osd_pool_default_min_size'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/global/osd_pool_default_size'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/debug_mds'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/debug_mgrc'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/debug_monc'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/debug_ms'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/mds_debug_auth_pins'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/mds_debug_frag'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/mds_debug_scatterstat'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/mds_debug_subtrees'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mds/mds_verify_scatter'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/debug_mgr'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/debug_mon'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/debug_monc'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/debug_ms'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/dashboard/x/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/dashboard/y/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/dashboard/z/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/restful/x/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/restful/y/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mgr/mgr/restful/z/server_port'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mon/mon_allow_pool_delete'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mon/mon_data_avail_crit'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mon/mon_data_avail_warn'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/mon/mon_osd_reporter_subtree_level'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_bdev'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_bluefs'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_bluestore'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_filestore'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_journal'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_mgrc'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_monc'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_ms'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_objclass'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_objecter'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_osd'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_reserver'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/debug_rocksdb'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/osd_copyfrom_max_chunk'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/osd_debug_misdirected_ops'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/osd_debug_op_order'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'config/osd/osd_scrub_load_threshold'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store saw key 'mgr/dashboard/accessdb_v1'
2019-03-07 09:53:14.149 7f39a648c700 20 mgr load_store fetching 'mgr/dashboard/accessdb_v1'
2019-03-07 09:53:14.149 7f39a648c700 10 monclient: _send_command 7 [{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v1"}]
2019-03-07 09:53:14.149 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.149 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v1"} v 0) v1 -- 0x565194b7f000 con 0x5651957f8000
2019-03-07 09:53:14.150 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 11 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v1"}]=0 obtained 'mgr/dashboard/accessdb_v1' v0) v1 ==== 134+0+229 (crc 0 0 0) 0x565198a84000 con 0x5651957f8000
2019-03-07 09:53:14.150 7f39b37dd700 10 monclient: handle_mon_command_ack 7 [{"prefix": "config-key get", "key": "mgr/dashboard/accessdb_v1"}]
2019-03-07 09:53:14.150 7f39b37dd700 10 monclient: _finish_command 7 = 0 obtained 'mgr/dashboard/accessdb_v1'
2019-03-07 09:53:14.150 7f39a648c700 20 mgr load_store saw key 'mgr/dashboard/crt'
2019-03-07 09:53:14.150 7f39a648c700 20 mgr load_store fetching 'mgr/dashboard/crt'
2019-03-07 09:53:14.150 7f39a648c700 10 monclient: _send_command 8 [{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]
2019-03-07 09:53:14.150 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.150 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/dashboard/crt"} v 0) v1 -- 0x565198a0ba00 con 0x5651957f8000
2019-03-07 09:53:14.151 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 12 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]=0 obtained 'mgr/dashboard/crt' v0) v1 ==== 118+0+1038 (crc 0 0 0) 0x565198a846c0 con 0x5651957f8000
2019-03-07 09:53:14.151 7f39b37dd700 10 monclient: handle_mon_command_ack 8 [{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]
2019-03-07 09:53:14.151 7f39b37dd700 10 monclient: _finish_command 8 = 0 obtained 'mgr/dashboard/crt'
2019-03-07 09:53:14.151 7f39a648c700 20 mgr load_store saw key 'mgr/dashboard/key'
2019-03-07 09:53:14.151 7f39a648c700 20 mgr load_store fetching 'mgr/dashboard/key'
2019-03-07 09:53:14.151 7f39a648c700 10 monclient: _send_command 9 [{"prefix": "config-key get", "key": "mgr/dashboard/key"}]
2019-03-07 09:53:14.151 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.151 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/dashboard/key"} v 0) v1 -- 0x565198a0bc00 con 0x5651957f8000
2019-03-07 09:53:14.152 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 13 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/dashboard/key"}]=0 obtained 'mgr/dashboard/key' v0) v1 ==== 118+0+1704 (crc 0 0 0) 0x56519899fb00 con 0x5651957f8000
2019-03-07 09:53:14.152 7f39b37dd700 10 monclient: handle_mon_command_ack 9 [{"prefix": "config-key get", "key": "mgr/dashboard/key"}]
2019-03-07 09:53:14.152 7f39b37dd700 10 monclient: _finish_command 9 = 0 obtained 'mgr/dashboard/key'
2019-03-07 09:53:14.152 7f39a648c700 20 mgr load_store saw key 'mgr/restful/keys/admin'
2019-03-07 09:53:14.152 7f39a648c700 20 mgr load_store fetching 'mgr/restful/keys/admin'
2019-03-07 09:53:14.152 7f39a648c700 10 monclient: _send_command 10 [{"prefix": "config-key get", "key": "mgr/restful/keys/admin"}]
2019-03-07 09:53:14.152 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.152 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/restful/keys/admin"} v 0) v1 -- 0x565198a0be00 con 0x5651957f8000
2019-03-07 09:53:14.153 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 14 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/restful/keys/admin"}]=0 obtained 'mgr/restful/keys/admin' v0) v1 ==== 128+0+36 (crc 0 0 0) 0x56519899fd40 con 0x5651957f8000
2019-03-07 09:53:14.153 7f39b37dd700 10 monclient: handle_mon_command_ack 10 [{"prefix": "config-key get", "key": "mgr/restful/keys/admin"}]
2019-03-07 09:53:14.153 7f39b37dd700 10 monclient: _finish_command 10 = 0 obtained 'mgr/restful/keys/admin'
2019-03-07 09:53:14.153 7f39a648c700 20 mgr load_store saw key 'mgr/restful/x/crt'
2019-03-07 09:53:14.153 7f39a648c700 20 mgr load_store fetching 'mgr/restful/x/crt'
2019-03-07 09:53:14.153 7f39a648c700 10 monclient: _send_command 11 [{"prefix": "config-key get", "key": "mgr/restful/x/crt"}]
2019-03-07 09:53:14.153 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.153 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/restful/x/crt"} v 0) v1 -- 0x565198a0b000 con 0x5651957f8000
2019-03-07 09:53:14.154 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 15 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/restful/x/crt"}]=0 obtained 'mgr/restful/x/crt' v0) v1 ==== 118+0+1029 (crc 0 0 0) 0x565198a84900 con 0x5651957f8000
2019-03-07 09:53:14.154 7f39b37dd700 10 monclient: handle_mon_command_ack 11 [{"prefix": "config-key get", "key": "mgr/restful/x/crt"}]
2019-03-07 09:53:14.154 7f39b37dd700 10 monclient: _finish_command 11 = 0 obtained 'mgr/restful/x/crt'
2019-03-07 09:53:14.154 7f39a648c700 20 mgr load_store saw key 'mgr/restful/x/key'
2019-03-07 09:53:14.154 7f39a648c700 20 mgr load_store fetching 'mgr/restful/x/key'
2019-03-07 09:53:14.154 7f39a648c700 10 monclient: _send_command 12 [{"prefix": "config-key get", "key": "mgr/restful/x/key"}]
2019-03-07 09:53:14.154 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.154 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mon_command({"prefix": "config-key get", "key": "mgr/restful/x/key"} v 0) v1 -- 0x565198a0ba00 con 0x5651957f8000
2019-03-07 09:53:14.155 7f39b37dd700 1 -- 172.20.0.2:0/4214 <== mon.1 v2:172.20.0.2:10002/0 16 ==== mon_command_ack([{"prefix": "config-key get", "key": "mgr/restful/x/key"}]=0 obtained 'mgr/restful/x/key' v0) v1 ==== 118+0+1704 (crc 0 0 0) 0x565198a84b40 con 0x5651957f8000
2019-03-07 09:53:14.155 7f39b37dd700 10 monclient: handle_mon_command_ack 12 [{"prefix": "config-key get", "key": "mgr/restful/x/key"}]
2019-03-07 09:53:14.155 7f39b37dd700 10 monclient: _finish_command 12 = 0 obtained 'mgr/restful/x/key'
2019-03-07 09:53:14.155 7f39a648c700 10 mgr[py] Module configuration contains 6 keys
2019-03-07 09:53:14.155 7f39a648c700 4 mgr init starting python modules...
2019-03-07 09:53:14.155 7f39a648c700 4 mgr[py] Starting modules in active mode
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting balancer
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting crash
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting devicehealth
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting orchestrator_cli
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting progress
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting restful
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting status
2019-03-07 09:53:14.156 7f39a648c700 4 mgr[py] Starting volumes
2019-03-07 09:53:14.156 7f39a648c700 4 mgr init Complete.
2019-03-07 09:53:14.156 7f39a648c700 4 mgr send_beacon active
2019-03-07 09:53:14.156 7f39a648c700 10 mgr send_beacon sending beacon as gid 4455
2019-03-07 09:53:14.158 7f39a648c700 4 mgr send_beacon going active, including 235 commands in beacon
2019-03-07 09:53:14.158 7f39a648c700 10 monclient: _send_mon_message to mon.b at v2:172.20.0.2:10002/0
2019-03-07 09:53:14.158 7f39a648c700 1 -- 172.20.0.2:0/4214 --> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] -- mgrbeacon mgr.x(83d2b8ab-9933-4c7c-b6f3-c5d88d25cde0,4455, [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214], 1) v8 -- 0x565198a1d8c0 con 0x5651957f8000
2019-03-07 09:53:14.158 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.159 7f39a648c700 1 mgr load Constructed class from module: balancer
2019-03-07 09:53:14.159 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.159 7f39a648c700 4 mgr operator() Starting thread for balancer
2019-03-07 09:53:14.159 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.159 7f39a4488700 4 mgr entry Entering thread for balancer
2019-03-07 09:53:14.159 7f39a648c700 1 mgr load Constructed class from module: crash
2019-03-07 09:53:14.159 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.159 7f39a648c700 4 mgr operator() Starting thread for crash
2019-03-07 09:53:14.159 7f39a4488700 20 mgr Gil Switched to new thread state 0x56519874be60
2019-03-07 09:53:14.159 7f39a3c87700 4 mgr entry Entering thread for crash
2019-03-07 09:53:14.160 7f39a4488700 4 mgr[balancer] Starting
2019-03-07 09:53:14.160 7f39a4488700 4 mgr get_config get_config key: mgr/balancer/active
2019-03-07 09:53:14.160 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.160 7f39a648c700 1 mgr load Constructed class from module: devicehealth
2019-03-07 09:53:14.160 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.160 7f39a648c700 4 mgr operator() Starting thread for devicehealth
2019-03-07 09:53:14.160 7f39a3c87700 20 mgr Gil Switched to new thread state 0x5651988d2000
2019-03-07 09:53:14.160 7f39a3c87700 20 mgr ~Gil Destroying new thread state 0x5651988d2000
2019-03-07 09:53:14.160 7f39a4488700 4 mgr get_typed_config get_typed_config active not found
2019-03-07 09:53:14.160 7f39a4488700 4 mgr get_config get_config key: mgr/balancer/sleep_interval
2019-03-07 09:53:14.160 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.160 7f39a3486700 4 mgr entry Entering thread for devicehealth
2019-03-07 09:53:14.161 7f39a648c700 1 mgr load Constructed class from module: orchestrator_cli
2019-03-07 09:53:14.161 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.161 7f39a648c700 4 mgr operator() Starting thread for orchestrator_cli
2019-03-07 09:53:14.161 7f39a4488700 4 mgr get_typed_config get_typed_config sleep_interval not found
2019-03-07 09:53:14.161 7f39a2c85700 4 mgr entry Entering thread for orchestrator_cli
2019-03-07 09:53:14.161 7f39a4488700 20 mgr[balancer] Waking up [inactive, now 2019-03-07_09:53:14]
2019-03-07 09:53:14.161 7f39a4488700 20 mgr[balancer] Sleeping for 60
2019-03-07 09:53:14.161 7f39a3486700 20 mgr Gil Switched to new thread state 0x5651988d2000
2019-03-07 09:53:14.161 7f39a3486700 4 mgr[devicehealth] Starting
2019-03-07 09:53:14.161 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/enable_monitoring
2019-03-07 09:53:14.161 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.162 7f39a648c700 1 mgr load Constructed class from module: progress
2019-03-07 09:53:14.162 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.162 7f39a648c700 4 mgr operator() Starting thread for progress
2019-03-07 09:53:14.162 7f39a2c85700 20 mgr Gil Switched to new thread state 0x5651988d2120
2019-03-07 09:53:14.162 7f39a2c85700 20 mgr ~Gil Destroying new thread state 0x5651988d2120
2019-03-07 09:53:14.162 7f39a3486700 4 mgr get_typed_config get_typed_config enable_monitoring not found
2019-03-07 09:53:14.162 7f39a2484700 4 mgr entry Entering thread for progress
2019-03-07 09:53:14.162 7f39a3486700 20 mgr[devicehealth] enable_monitoring = False
2019-03-07 09:53:14.162 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/scrape_frequency
2019-03-07 09:53:14.162 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.162 7f39a648c700 1 mgr load Constructed class from module: restful
2019-03-07 09:53:14.163 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.163 7f39a648c700 4 mgr operator() Starting thread for restful
2019-03-07 09:53:14.163 7f39a2484700 20 mgr Gil Switched to new thread state 0x5651988d2120
2019-03-07 09:53:14.163 7f39a2484700 4 mgr get_config get_config key: mgr/progress/max_completed_events
2019-03-07 09:53:14.163 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.163 7f39a1c83700 4 mgr entry Entering thread for restful
2019-03-07 09:53:14.163 7f39a648c700 1 mgr load Constructed class from module: status
2019-03-07 09:53:14.163 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.163 7f39a648c700 4 mgr operator() Starting thread for status
2019-03-07 09:53:14.163 7f39a2484700 4 mgr get_typed_config get_typed_config max_completed_events not found
2019-03-07 09:53:14.163 7f39a1482700 4 mgr entry Entering thread for status
2019-03-07 09:53:14.163 7f39a2484700 20 mgr[progress] max_completed_events = 50
2019-03-07 09:53:14.163 7f39a2484700 4 mgr get_config get_config key: mgr/progress/persist_interval
2019-03-07 09:53:14.163 7f39a3486700 4 mgr get_typed_config get_typed_config scrape_frequency not found
2019-03-07 09:53:14.164 7f39a3486700 20 mgr[devicehealth] scrape_frequency = 86400
2019-03-07 09:53:14.164 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/pool_name
2019-03-07 09:53:14.164 7f39a3486700 4 mgr get_typed_config get_typed_config pool_name not found
2019-03-07 09:53:14.164 7f39a3486700 20 mgr[devicehealth] pool_name = device_health_metrics
2019-03-07 09:53:14.165 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/retention_period
2019-03-07 09:53:14.165 7f39a1c83700 20 mgr Gil Switched to new thread state 0x5651988d2240
2019-03-07 09:53:14.165 7f39a1c83700 4 mgr get_store_prefix get_store_prefix prefix: mgr/restful/keys/
2019-03-07 09:53:14.166 7f39a2484700 4 mgr get_typed_config get_typed_config persist_interval not found
2019-03-07 09:53:14.167 7f39a2484700 20 mgr[progress] persist_interval = 5
2019-03-07 09:53:14.168 7f39a2484700 4 mgr[progress] Loading...
2019-03-07 09:53:14.168 7f39a2484700 4 mgr get_store get_store key: mgr/progress/completed
2019-03-07 09:53:14.168 7f39a2484700 4 ceph_store_get completed not found
2019-03-07 09:53:14.168 7f39a2484700 4 mgr[progress] No stored events to load
2019-03-07 09:53:14.169 7f39a2484700 4 mgr[progress] Loaded [] historic events
2019-03-07 09:53:14.170 7f39a2484700 4 mgr[progress] Loaded OSDMap, ready.
2019-03-07 09:53:14.170 7f39a1c83700 4 mgr get_config get_config key: mgr/restful/x/server_addr
2019-03-07 09:53:14.170 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.172 7f39a648c700 1 mgr load Constructed class from module: volumes
2019-03-07 09:53:14.172 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.172 7f39a648c700 4 mgr operator() Starting thread for volumes
2019-03-07 09:53:14.172 7f39a3486700 4 mgr get_typed_config get_typed_config retention_period not found
2019-03-07 09:53:14.172 7f39a0c81700 4 mgr entry Entering thread for volumes
2019-03-07 09:53:14.173 7f39a3486700 20 mgr[devicehealth] retention_period = 15552000
2019-03-07 09:53:14.173 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/mark_out_threshold
2019-03-07 09:53:14.173 7f39a1482700 20 mgr Gil Switched to new thread state 0x5651988d2360
2019-03-07 09:53:14.173 7f39a1482700 20 mgr ~Gil Destroying new thread state 0x5651988d2360
2019-03-07 09:53:14.173 7f39a1c83700 4 mgr get_typed_config get_typed_config x/server_addr not found
2019-03-07 09:53:14.173 7f39a1c83700 4 mgr get_config get_config key: mgr/restful/server_addr
2019-03-07 09:53:14.173 7f39a0c81700 20 mgr Gil Switched to new thread state 0x5651988d2360
2019-03-07 09:53:14.173 7f39a0c81700 20 mgr ~Gil Destroying new thread state 0x5651988d2360
2019-03-07 09:53:14.174 7f39a3486700 4 mgr get_typed_config get_typed_config mark_out_threshold not found
2019-03-07 09:53:14.174 7f39a3486700 20 mgr[devicehealth] mark_out_threshold = 2419200
2019-03-07 09:53:14.174 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/warn_threshold
2019-03-07 09:53:14.174 7f39a1c83700 4 mgr get_typed_config get_typed_config server_addr not found
2019-03-07 09:53:14.174 7f39a1c83700 4 mgr get_config get_config key: mgr/restful/x/server_port
2019-03-07 09:53:14.174 7f39a3486700 4 mgr get_typed_config get_typed_config warn_threshold not found
2019-03-07 09:53:14.174 7f39a3486700 20 mgr[devicehealth] warn_threshold = 7257600
2019-03-07 09:53:14.174 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/self_heal
2019-03-07 09:53:14.174 7f39a1c83700 10 mgr get_typed_config get_typed_config x/server_port found: 12000
2019-03-07 09:53:14.174 7f39a1c83700 4 mgr[restful] server_addr: :: server_port: 12000
2019-03-07 09:53:14.175 7f39a1c83700 4 mgr get_store get_store key: mgr/restful/x/crt
2019-03-07 09:53:14.175 7f39a1c83700 10 ceph_store_get x/crt found: -----BEGIN CERTIFICATE-----
MIICzDCCAbQCEQCX/2zK6G9DGa4khd/pqnY8MA0GCSqGSIb3DQEBDQUAMCQxCzAJ
BgNVBAoMAklUMRUwEwYDVQQDDAxjZXBoLXJlc3RmdWwwHhcNMTkwMzA3MDk1MTMz
WhcNMjkwMzA0MDk1MTMzWjAkMQswCQYDVQQKDAJJVDEVMBMGA1UEAwwMY2VwaC1y
ZXN0ZnVsMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtucuiXw0My92
2KsO7dobqZNyUtBYrOE19gCZv4Xs9iDDAzGg4zj8J/XeM9/A7TRzA94l4WFxq704
USIli5hM2XdLSTZ1f0U5J744eex1JR1ozj7znN1BLhYgkZe6WpH+gL7F+dc5iqsm
wCc4XLrncj5PmceQ0FAJIrvHo+531+9pi8i3VPRXHdMYdZFA1HAOFmLVsjvHRYlx
hxS2wEzyfcD9GqdbiHttg/UQK4/VcGkXn+zR7heFt1eJ4Xu8FAONOTfNb1QtxkKe
9YDoAhJ3pXqE2zh0o979+pQXC7wFGQO5+zjLHmgTjGEuTiLHv7dGaHViIhcntug2
jpWusrjhZwIDAQABMA0GCSqGSIb3DQEBDQUAA4IBAQBErYlpQsori2XxSTQYg4Ss
/PGoxzOn3PWALNvEF/ko5PGbefGrBK2/Cxv4wDiox74mV2YV2G2Rx+ixO0jgbRAp
wZz4Dl5sx+gAZDT4226MypR/uG+BS75o4WV3/ldx0ez7tDcD7Coc3wOvam2w5yhD
0xFaXOHtxQUH/xRqk5TSSXxlfq3DglYYiXbaplwbLyltt9JtgQLhgUQPYV11HUkt
Sx7BrKR5D2Fj5Q70JVMYe/BlQUsCJLGVnO8ZfaJQxndLxpFvcq0OLYBC37VPEq6I
3q5DhzhnHCdAUoO1yrC76rg680gTe5zH7ZlzhNzH00JBtFLk3kIL598GuGQWbvYZ
-----END CERTIFICATE-----

2019-03-07 09:53:14.175 7f39a3486700 4 mgr get_typed_config get_typed_config self_heal not found
2019-03-07 09:53:14.175 7f39a3486700 20 mgr[devicehealth] self_heal = True
2019-03-07 09:53:14.175 7f39a3486700 4 mgr get_config get_config key: mgr/devicehealth/sleep_interval
2019-03-07 09:53:14.175 7f39a3486700 4 mgr get_typed_config get_typed_config sleep_interval not found
2019-03-07 09:53:14.175 7f39a3486700 20 mgr[devicehealth] sleep_interval = 600
2019-03-07 09:53:14.175 7f39a3486700 4 mgr get_store get_store key: mgr/devicehealth/last_scrape
2019-03-07 09:53:14.175 7f39a3486700 4 ceph_store_get last_scrape not found
2019-03-07 09:53:14.175 7f39a3486700 20 mgr[devicehealth] Last scrape None
2019-03-07 09:53:14.175 7f39a3486700 20 mgr[devicehealth] Sleeping for 600 seconds
2019-03-07 09:53:14.177 7f39a1c83700 4 mgr get_store get_store key: mgr/restful/x/key
2019-03-07 09:53:14.177 7f39a1c83700 10 ceph_store_get x/key found: -----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC25y6JfDQzL3bY
qw7t2hupk3JS0Fis4TX2AJm/hez2IMMDMaDjOPwn9d4z38DtNHMD3iXhYXGrvThR
IiWLmEzZd0tJNnV/RTknvjh57HUlHWjOPvOc3UEuFiCRl7pakf6AvsX51zmKqybA
JzhcuudyPk+Zx5DQUAkiu8ej7nfX72mLyLdU9Fcd0xh1kUDUcA4WYtWyO8dFiXGH
FLbATPJ9wP0ap1uIe22D9RArj9VwaRef7NHuF4W3V4nhe7wUA405N81vVC3GQp71
gOgCEneleoTbOHSj3v36lBcLvAUZA7n7OMseaBOMYS5OIse/t0ZodWIiFye26DaO
la6yuOFnAgMBAAECggEAfML1W5L9iGy4BNOqO1SryeBp+1NjKYn15PhD6ADSwpFW
dLdFp7kXR6rPB1N5plRAoELIVBTEFQHdiHpiYfZhdXWKRUaAn2/4rMN0grZosgh9
lTIbusSx0V4GmUmGNIGhuPBo2yOUfKKCrFAUgix789HNOniGr5HoqWitpYTk+sWN
U4LqSwvLrd7SM8CtCxOEKeyjuVfKr7eLPaDK0+S3RXd29XCDBJ9j9S1tLGgVdA3x
gV3HSFahqjxHsAQ1SU+XZm/dht9adHdhxuVPeGnF1nDkX7++qKP9m9KRDWJOLdjk
UQ5emjJSNWN006H7stHh90PjLFYcPZJKE4IYeZcIAQKBgQDdAhCxSV8BKoalzzUd
OPxn+Inv5zbzJLPW5p8C/r+7+AvJILeke7TKy4boGnWPrZEXjh9Yb5OyyZf3lNXJ
FSGWWqCgyJDDBBw9sToEg3rR94IUqOUzjQdNmpupU03/iFubzvGZAbPDa1KRPwQr
0CUg6v/WsXdjVYUyB8lGJayf4QKBgQDT3KNUNoVDS5CdyloeaLA3obDe+oGZQxbe
rKsppoZeaZYSLzgkVDrEUhPNBubzr/MW41pYG5JtOFe/27K/DvsQw8tgv1YxSacT
wwuSJQWbapZt5vxnKDxwycvTPbmLy3B9OP+jfqVmEzB0y42SGr11Jr+cBdPxtrKw
A+Nt49HKRwKBgELU3wtdbpdBrtnxINSnaNxf8gZT2vAHx0JTFv6m3oAO8DFv7ViV
+APjK/aAGdDK4bdHyP7LLtS/1W6O2V8nPrLi0PQN8oimc4vMTgkTYU6hE/r/X+AI
4456FFxNzqZZK6qgF7r3Is8TYzJ85dwB41hi996KZ9noAJEyfZDyz5LBAoGBANJm
kznl71CWsedSd08tzpthE6NOCvjnHMZFmrKBc349w9EJXesOT1J65Lk/+orWGUeI
wEcyEqgvRU3/ZLPDr+jFWawaOQI30BqbIFmiNMBVCVHaDJ5x4hxs++Qsr6E6h9uE
iLSY6K/HrN7A7DbGIZM/yRis3QB23VxacDnI/6avAoGAMAlXV99NCYpM3MNgiSoj
ZqBmPTIqvOEMUhNNohNjVRRkTsmtnPbqhuFmscWHU4Q4lAe/zEUulQgjeCzNYcys
63AkgEma5qRmjJkHvA+Hq+k3thCIJKen/eTokmg1c6FhHNQ9D/24xDGHFcZMXE1h
II7FKI+rmP7+5EbmqMA/FN0=
-----END PRIVATE KEY-----

2019-03-07 09:53:14.177 7f39a1c83700 4 mgr set_uri module restful set URI 'https://ceph.dev:12000/'
2019-03-07 09:53:14.180 7f39b47df700 -1 received signal: Terminated from python ../qa/tasks/vstart_runner.py tasks.mgr.test_dashboard tasks.mgr.dashboard.test_auth tasks.mgr.dashboard.test_cephfs tasks.mgr.dashboard.test_cluster_configuration tasks.mgr.dashboard.test_erasure_code_profile tasks.mgr.dashboard.test_ganesha tasks.mgr.dashboard.test_health tasks.mgr.dashboard.test_host tasks.mgr.dashboard.test_logs tasks.mgr.dashboard.test_mgr_module tasks.mgr.dashboard.test_monitor tasks.mgr.dashboard.test_osd tasks.mgr.dashboard.test_perf_counters tasks.mgr.dashboard.test_pool tasks.mgr.dashboard.test_rbd tasks.mgr.dashboard.test_rbd_mirroring tasks.mgr.dashboard.test_requests tasks.mgr.dashboard.test_rgw tasks.mgr.dashboard.test_role tasks.mgr.dashboard.test_settings tasks.mgr.dashboard.test_summary tasks.mgr.dashboard.test_user (PID: 3920) UID: 0
2019-03-07 09:53:14.180 7f39b47df700 -1 mgr handle_signal *** Got signal Terminated ***
2019-03-07 09:53:14.180 7f39b3fde700 4 mgr operator() Shutting down
2019-03-07 09:53:14.181 7f39b3fde700 10 monclient: shutdown
2019-03-07 09:53:14.181 7f39b3fde700 1 -- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 msgr2=0x565195746a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down
2019-03-07 09:53:14.181 7f39b3fde700 1 --2- 172.20.0.2:0/4214 >> [v2:172.20.0.2:10002/0,v1:172.20.0.2:10003/0] conn(0x5651957f8000 0x565195746a80 crc :-1 s=READY pgs=51 cs=0 l=1 rx=0 tx=0).stop
2019-03-07 09:53:14.181 7f39a648c700 10 mgr.server shutdown begin
2019-03-07 09:53:14.181 7f39a648c700 1 -- [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214] shutdown_connections
2019-03-07 09:53:14.185 7f39a648c700 1 -- [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214] shutdown_connections
2019-03-07 09:53:14.185 7f39a648c700 1 -- [v2:172.20.0.2:6800/4214,v1:172.20.0.2:6801/4214] wait complete.
2019-03-07 09:53:14.185 7f39a648c700 10 mgr.server shutdown done
2019-03-07 09:53:14.185 7f39a648c700 10 mgr shutdown calling module balancer shutdown()
2019-03-07 09:53:14.189 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.190 7f39a648c700 4 mgr[balancer] Stopping
2019-03-07 09:53:14.190 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.190 7f39a648c700 10 mgr shutdown module balancer shutdown() returned
2019-03-07 09:53:14.190 7f39a648c700 10 mgr shutdown calling module crash shutdown()
2019-03-07 09:53:14.190 7f39a4488700 20 mgr ~Gil Destroying new thread state 0x56519874be60
2019-03-07 09:53:14.190 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.190 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.190 7f39a648c700 10 mgr shutdown module crash shutdown() returned
2019-03-07 09:53:14.190 7f39a648c700 10 mgr shutdown calling module devicehealth shutdown()
2019-03-07 09:53:14.192 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 4 mgr[devicehealth] Stopping
2019-03-07 09:53:14.192 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown module devicehealth shutdown() returned
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown calling module orchestrator_cli shutdown()
2019-03-07 09:53:14.192 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown module orchestrator_cli shutdown() returned
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown calling module progress shutdown()
2019-03-07 09:53:14.192 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown module progress shutdown() returned
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown calling module restful shutdown()
2019-03-07 09:53:14.192 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown module restful shutdown() returned
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown calling module status shutdown()
2019-03-07 09:53:14.192 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown module status shutdown() returned
2019-03-07 09:53:14.192 7f39a648c700 10 mgr shutdown calling module volumes shutdown()
2019-03-07 09:53:14.192 7f39a3486700 20 mgr ~Gil Destroying new thread state 0x5651988d2000
2019-03-07 09:53:14.193 7f39a648c700 20 mgr Gil Switched to new thread state 0x56519874bd40
2019-03-07 09:53:14.193 7f39a648c700 20 mgr ~Gil Destroying new thread state 0x56519874bd40
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown module volumes shutdown() returned
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module balancer
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joined module balancer
2019-03-07 09:53:14.193 7f39a2484700 20 mgr ~Gil Destroying new thread state 0x5651988d2120
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module crash
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joined module crash
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module devicehealth
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joined module devicehealth
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module orchestrator_cli
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joined module orchestrator_cli
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module progress
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joined module progress
2019-03-07 09:53:14.193 7f39a648c700 10 mgr shutdown joining module restful

(2-2/2)