Project

General

Profile

Bug #43407 » ceph-mds.ceph1.log

Marco Savoca, 12/22/2019 11:36 PM

 
2019-12-22 12:20:17.347 7fce967872c0 0 set uid:gid to 64045:64045 (ceph:ceph)
2019-12-22 12:20:17.347 7fce967872c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7040
2019-12-22 12:20:17.347 7fce967872c0 0 pidfile_write: ignore empty --pid-file
2019-12-22 12:20:17.351 7fce848d7700 1 mds.ceph1 Updating MDS map to version 331 from mon.0
2019-12-22 12:20:21.811 7fce848d7700 1 mds.ceph1 Updating MDS map to version 332 from mon.0
2019-12-22 12:20:21.811 7fce848d7700 1 mds.ceph1 Map has assigned me to become a standby
2019-12-22 12:20:21.815 7fce848d7700 1 mds.ceph1 Updating MDS map to version 333 from mon.0
2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 handle_mds_map i am now mds.0.333
2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 handle_mds_map state change up:boot --> up:replay
2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 replay_start
2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 recovery set is
2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 waiting for osdmap 6044 (which blacklists prior instance)
2019-12-22 12:20:21.819 7fce7d8c9700 0 mds.0.cache creating system inode with ino:0x100
2019-12-22 12:20:21.819 7fce7d8c9700 0 mds.0.cache creating system inode with ino:0x1
2019-12-22 12:20:22.055 7fce7c8c7700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
2019-12-22 12:20:22.055 7fce7c8c7700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
2019-12-22 12:20:22.055 7fce7c8c7700 0 mds.0.journal journal ambig_subtrees:
2019-12-22 12:20:22.059 7fce7c8c7700 -1 *** Caught signal (Segmentation fault) **
in thread 7fce7c8c7700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7fce8d26d890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-272> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command assert hook 0x562d33cf4500
-271> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command abort hook 0x562d33cf4500
-270> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perfcounters_dump hook 0x562d33cf4500
-269> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command 1 hook 0x562d33cf4500
-268> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perf dump hook 0x562d33cf4500
-267> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perfcounters_schema hook 0x562d33cf4500
-266> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perf histogram dump hook 0x562d33cf4500
-265> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command 2 hook 0x562d33cf4500
-264> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perf schema hook 0x562d33cf4500
-263> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perf histogram schema hook 0x562d33cf4500
-262> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command perf reset hook 0x562d33cf4500
-261> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config show hook 0x562d33cf4500
-260> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config help hook 0x562d33cf4500
-259> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config set hook 0x562d33cf4500
-258> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config unset hook 0x562d33cf4500
-257> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config get hook 0x562d33cf4500
-256> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config diff hook 0x562d33cf4500
-255> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command config diff get hook 0x562d33cf4500
-254> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command log flush hook 0x562d33cf4500
-253> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command log dump hook 0x562d33cf4500
-252> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command log reopen hook 0x562d33cf4500
-251> 2019-12-22 12:20:17.331 7fce967872c0 5 asok(0x562d33d82000) register_command dump_mempools hook 0x562d3498c068
-250> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient: get_monmap_and_config
-249> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient: build_initial_monmap
-248> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:17.343610
created 2019-12-22 12:20:17.343610
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-247> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding auth protocol: cephx
-246> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding auth protocol: cephx
-245> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding auth protocol: cephx
-244> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-243> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-242> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-241> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-240> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-239> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-238> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-237> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-236> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-235> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-234> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: crc
-233> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x562d33dc6140) adding con mode: secure
-232> 2019-12-22 12:20:17.339 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-231> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient: init
-230> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding auth protocol: cephx
-229> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding auth protocol: cephx
-228> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding auth protocol: cephx
-227> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-226> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-225> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-224> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-223> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-222> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-221> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-220> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-219> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-218> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-217> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: crc
-216> 2019-12-22 12:20:17.339 7fce967872c0 5 AuthRegistry(0x7ffd444b6bc8) adding con mode: secure
-215> 2019-12-22 12:20:17.339 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-214> 2019-12-22 12:20:17.339 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-213> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient: _reopen_session rank -1
-212> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x562d33e1d680 addr v1:10.10.0.20:6789/0
-211> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): picked mon.noname-c-legacy con 0x562d33e1cd80 addr v1:10.10.0.30:6789/0
-210> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): picked mon.noname-c con 0x562d33e1db00 addr v2:10.10.0.30:3300/0
-209> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): start opening mon connection
-208> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): _renew_subs
-207> 2019-12-22 12:20:17.339 7fce967872c0 10 monclient(hunting): authenticate will time out at 2019-12-22 12:25:17.344924
-206> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): get_auth_request con 0x562d33e1db00 auth_method 0
-205> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-204> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): _init_auth method 2
-203> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more payload 9
-202> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-201> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-200> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient(hunting): handle_auth_done global_id 554753 payload 386
-199> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient: _finish_hunting 0
-198> 2019-12-22 12:20:17.343 7fce88ac7700 1 monclient: found mon.noname-c
-197> 2019-12-22 12:20:17.343 7fce88ac7700 10 monclient: _send_mon_message to mon.noname-c at v2:10.10.0.30:3300/0
-196> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: handle_monmap mon_map magic: 0 v1
-195> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: got monmap 1 from mon.noname-c (according to old e1)
-194> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-193> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: _finish_auth 0
-192> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:47.347388)
-191> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: _send_mon_message to mon.ceph3 at v2:10.10.0.30:3300/0
-190> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: handle_config config(2 keys) v1
-189> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: handle_monmap mon_map magic: 0 v1
-188> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: got monmap 1 from mon.ceph3 (according to old e1)
-187> 2019-12-22 12:20:17.343 7fce872c4700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-186> 2019-12-22 12:20:17.343 7fce967872c0 5 monclient: authenticate success, global_id 554753
-185> 2019-12-22 12:20:17.343 7fce967872c0 10 monclient: get_monmap_and_config success
-184> 2019-12-22 12:20:17.343 7fce967872c0 10 monclient: shutdown
-183> 2019-12-22 12:20:17.343 7fce85ac1700 4 set_mon_vals no callback set
-182> 2019-12-22 12:20:17.343 7fce85ac1700 10 set_mon_vals mon_target_pg_per_osd = 50
-181> 2019-12-22 12:20:17.347 7fce967872c0 0 set uid:gid to 64045:64045 (ceph:ceph)
-180> 2019-12-22 12:20:17.347 7fce967872c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7040
-179> 2019-12-22 12:20:17.347 7fce967872c0 0 pidfile_write: ignore empty --pid-file
-178> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) init /var/run/ceph/ceph-mds.ceph1.asok
-177> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) bind_and_listen /var/run/ceph/ceph-mds.ceph1.asok
-176> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) register_command 0 hook 0x562d33cf2ac8
-175> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) register_command version hook 0x562d33cf2ac8
-174> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) register_command git_version hook 0x562d33cf2ac8
-173> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) register_command help hook 0x562d33cf4380
-172> 2019-12-22 12:20:17.347 7fce967872c0 5 asok(0x562d33d82000) register_command get_command_descriptions hook 0x562d33cf4460
-171> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding auth protocol: cephx
-170> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding auth protocol: cephx
-169> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding auth protocol: cephx
-168> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-167> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-166> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-165> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-164> 2019-12-22 12:20:17.347 7fce862c2700 5 asok(0x562d33d82000) entry start
-163> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-162> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-161> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-160> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-159> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-158> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-157> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: crc
-156> 2019-12-22 12:20:17.347 7fce967872c0 5 AuthRegistry(0x562d33dc6a40) adding con mode: secure
-155> 2019-12-22 12:20:17.347 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-154> 2019-12-22 12:20:17.347 7fce967872c0 10 monclient: build_initial_monmap
-153> 2019-12-22 12:20:17.347 7fce967872c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:17.352920
created 2019-12-22 12:20:17.352920
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-152> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient: init
-151> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding auth protocol: cephx
-150> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding auth protocol: cephx
-149> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding auth protocol: cephx
-148> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-147> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-146> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-145> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-144> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-143> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-142> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-141> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-140> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-139> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-138> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: crc
-137> 2019-12-22 12:20:17.351 7fce967872c0 5 AuthRegistry(0x7ffd444b8958) adding con mode: secure
-136> 2019-12-22 12:20:17.351 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-135> 2019-12-22 12:20:17.351 7fce967872c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-134> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient: _reopen_session rank -1
-133> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): picked mon.noname-c con 0x562d33e1d680 addr v2:10.10.0.30:3300/0
-132> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): picked mon.noname-a-legacy con 0x562d33e1cd80 addr v1:10.10.0.10:6789/0
-131> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): picked mon.noname-a con 0x562d33e1df80 addr v2:10.10.0.10:3300/0
-130> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): start opening mon connection
-129> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): start opening mon connection
-128> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient(hunting): _renew_subs
-127> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): get_auth_request con 0x562d33e1df80 auth_method 0
-126> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-125> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): _init_auth method 2
-124> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more payload 9
-123> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-122> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-121> 2019-12-22 12:20:17.351 7fce882c6700 10 monclient(hunting): get_auth_request con 0x562d33e1d680 auth_method 0
-120> 2019-12-22 12:20:17.351 7fce882c6700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-119> 2019-12-22 12:20:17.351 7fce882c6700 10 monclient(hunting): _init_auth method 2
-118> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient(hunting): handle_auth_done global_id 554631 payload 1102
-117> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient: _finish_hunting 0
-116> 2019-12-22 12:20:17.351 7fce88ac7700 1 monclient: found mon.noname-a
-115> 2019-12-22 12:20:17.351 7fce88ac7700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.10:3300/0
-114> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1
-113> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: handle_monmap mon_map magic: 0 v1
-112> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: got monmap 1 from mon.noname-a (according to old e1)
-111> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-110> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: _finish_auth 0
-109> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:47.354648)
-108> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-107> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: handle_config config(2 keys) v1
-106> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: handle_monmap mon_map magic: 0 v1
-105> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: got monmap 1 from mon.ceph1 (according to old e1)
-104> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-103> 2019-12-22 12:20:17.351 7fce830d4700 4 set_mon_vals no callback set
-102> 2019-12-22 12:20:17.351 7fce967872c0 5 monclient: authenticate success, global_id 554631
-101> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: _finish_auth 0
-100> 2019-12-22 12:20:17.351 7fce848d7700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-22 12:19:47.355045)
-99> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient: wait_auth_rotating done
-98> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient: _renew_subs
-97> 2019-12-22 12:20:17.351 7fce967872c0 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-96> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command status hook 0x562d33cf47c0
-95> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump_ops_in_flight hook 0x562d33cf47c0
-94> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command ops hook 0x562d33cf47c0
-93> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump_blocked_ops hook 0x562d33cf47c0
-92> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump_historic_ops hook 0x562d33cf47c0
-91> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump_historic_ops_by_duration hook 0x562d33cf47c0
-90> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command scrub_path hook 0x562d33cf47c0
-89> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command tag path hook 0x562d33cf47c0
-88> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command flush_path hook 0x562d33cf47c0
-87> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command export dir hook 0x562d33cf47c0
-86> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump cache hook 0x562d33cf47c0
-85> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command cache status hook 0x562d33cf47c0
-84> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump tree hook 0x562d33cf47c0
-83> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump loads hook 0x562d33cf47c0
-82> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump snaps hook 0x562d33cf47c0
-81> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command session evict hook 0x562d33cf47c0
-80> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command osdmap barrier hook 0x562d33cf47c0
-79> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command session ls hook 0x562d33cf47c0
-78> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command flush journal hook 0x562d33cf47c0
-77> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command force_readonly hook 0x562d33cf47c0
-76> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command get subtrees hook 0x562d33cf47c0
-75> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dirfrag split hook 0x562d33cf47c0
-74> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dirfrag merge hook 0x562d33cf47c0
-73> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dirfrag ls hook 0x562d33cf47c0
-72> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command openfiles ls hook 0x562d33cf47c0
-71> 2019-12-22 12:20:17.351 7fce967872c0 5 asok(0x562d33d82000) register_command dump inode hook 0x562d33cf47c0
-70> 2019-12-22 12:20:17.351 7fce818d1700 5 mds.beacon.ceph1 Sending beacon up:boot seq 1
-69> 2019-12-22 12:20:17.351 7fce818d1700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-68> 2019-12-22 12:20:17.351 7fce848d7700 1 mds.ceph1 Updating MDS map to version 331 from mon.0
-67> 2019-12-22 12:20:17.355 7fce848d7700 4 mgrc handle_mgr_map Got map version 130
-66> 2019-12-22 12:20:17.355 7fce848d7700 4 mgrc handle_mgr_map Active mgr is now [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-65> 2019-12-22 12:20:17.355 7fce848d7700 4 mgrc reconnect Starting new session with [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-64> 2019-12-22 12:20:17.355 7fce882c6700 10 monclient: get_auth_request con 0x562d33e1f600 auth_method 0
-63> 2019-12-22 12:20:17.355 7fce848d7700 4 mgrc handle_mgr_configure stats_period=5
-62> 2019-12-22 12:20:17.355 7fce848d7700 4 mgrc handle_mgr_configure updated stats threshold: 5
-61> 2019-12-22 12:20:21.351 7fce818d1700 5 mds.beacon.ceph1 Sending beacon up:boot seq 2
-60> 2019-12-22 12:20:21.371 7fce818d1700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-59> 2019-12-22 12:20:21.811 7fce848d7700 1 mds.ceph1 Updating MDS map to version 332 from mon.0
-58> 2019-12-22 12:20:21.811 7fce848d7700 5 mds.beacon.ceph1 set_want_state: up:boot -> up:standby
-57> 2019-12-22 12:20:21.811 7fce848d7700 1 mds.ceph1 Map has assigned me to become a standby
-56> 2019-12-22 12:20:21.811 7fce88ac7700 5 mds.beacon.ceph1 received beacon reply up:boot seq 2 rtt 0.459994
-55> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.ceph1 Updating MDS map to version 333 from mon.0
-54> 2019-12-22 12:20:21.815 7fce848d7700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-53> 2019-12-22 12:20:21.815 7fce848d7700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-52> 2019-12-22 12:20:21.815 7fce848d7700 5 asok(0x562d33d82000) register_command objecter_requests hook 0x562d33cf4760
-51> 2019-12-22 12:20:21.815 7fce848d7700 10 monclient: _renew_subs
-50> 2019-12-22 12:20:21.815 7fce848d7700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-49> 2019-12-22 12:20:21.815 7fce848d7700 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
-48> 2019-12-22 12:20:21.815 7fce848d7700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-47> 2019-12-22 12:20:21.815 7fce848d7700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-46> 2019-12-22 12:20:21.815 7fce848d7700 4 mds.0.0 handle_osd_map epoch 0, 0 new blacklist entries
-45> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 handle_mds_map i am now mds.0.333
-44> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 handle_mds_map state change up:boot --> up:replay
-43> 2019-12-22 12:20:21.815 7fce848d7700 5 mds.beacon.ceph1 set_want_state: up:standby -> up:replay
-42> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 replay_start
-41> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 recovery set is
-40> 2019-12-22 12:20:21.815 7fce848d7700 1 mds.0.333 waiting for osdmap 6044 (which blacklists prior instance)
-39> 2019-12-22 12:20:21.819 7fce848d7700 4 mds.0.333 handle_osd_map epoch 6044, 0 new blacklist entries
-38> 2019-12-22 12:20:21.819 7fce848d7700 10 monclient: _renew_subs
-37> 2019-12-22 12:20:21.819 7fce848d7700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-36> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: opening inotable
-35> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: opening sessionmap
-34> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: opening mds log
-33> 2019-12-22 12:20:21.819 7fce7e0ca700 5 mds.0.log open discovering log bounds
-32> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: opening purge queue (async)
-31> 2019-12-22 12:20:21.819 7fce7e0ca700 4 mds.0.purge_queue open: opening
-30> 2019-12-22 12:20:21.819 7fce7e0ca700 1 mds.0.journaler.pq(ro) recover start
-29> 2019-12-22 12:20:21.819 7fce7e0ca700 1 mds.0.journaler.pq(ro) read_head
-28> 2019-12-22 12:20:21.819 7fce7d8c9700 4 mds.0.journalpointer Reading journal pointer '400.00000000'
-27> 2019-12-22 12:20:21.819 7fce882c6700 10 monclient: get_auth_request con 0x562d34add200 auth_method 0
-26> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: loading open file table (async)
-25> 2019-12-22 12:20:21.819 7fce7e0ca700 2 mds.0.333 Booting: 0: opening snap table
-24> 2019-12-22 12:20:21.819 7fce87ac5700 10 monclient: get_auth_request con 0x562d34add680 auth_method 0
-23> 2019-12-22 12:20:21.819 7fce87ac5700 10 monclient: get_auth_request con 0x562d34adcd80 auth_method 0
-22> 2019-12-22 12:20:21.819 7fce88ac7700 10 monclient: get_auth_request con 0x562d34addb00 auth_method 0
-21> 2019-12-22 12:20:21.819 7fce7f0cc700 1 mds.0.journaler.pq(ro) _finish_read_head loghead(trim 4194304, expire 4320033, write 4320033, stream_format 1). probing for end of log (from 4320033)...
-20> 2019-12-22 12:20:21.819 7fce7f0cc700 1 mds.0.journaler.pq(ro) probing for end of the log
-19> 2019-12-22 12:20:21.819 7fce7d8c9700 1 mds.0.journaler.mdlog(ro) recover start
-18> 2019-12-22 12:20:21.819 7fce7d8c9700 1 mds.0.journaler.mdlog(ro) read_head
-17> 2019-12-22 12:20:21.819 7fce7f0cc700 1 mds.0.journaler.pq(ro) _finish_probe_end write_pos = 4320033 (header had 4320033). recovered.
-16> 2019-12-22 12:20:21.819 7fce7f0cc700 4 mds.0.purge_queue operator(): open complete
-15> 2019-12-22 12:20:21.819 7fce7f0cc700 1 mds.0.journaler.pq(ro) set_writeable
-14> 2019-12-22 12:20:21.819 7fce7d8c9700 4 mds.0.log Waiting for journal 0x200 to recover...
-13> 2019-12-22 12:20:21.819 7fce7e0ca700 1 mds.0.journaler.mdlog(ro) _finish_read_head loghead(trim 805306368, expire 808964827, write 812539373, stream_format 1). probing for end of log (from 812539373)...
-12> 2019-12-22 12:20:21.819 7fce7e0ca700 1 mds.0.journaler.mdlog(ro) probing for end of the log
-11> 2019-12-22 12:20:21.819 7fce7e0ca700 1 mds.0.journaler.mdlog(ro) _finish_probe_end write_pos = 812539516 (header had 812539373). recovered.
-10> 2019-12-22 12:20:21.819 7fce7d8c9700 4 mds.0.log Journal 0x200 recovered.
-9> 2019-12-22 12:20:21.819 7fce7d8c9700 4 mds.0.log Recovered journal 0x200 in format 1
-8> 2019-12-22 12:20:21.819 7fce7d8c9700 2 mds.0.333 Booting: 1: loading/discovering base inodes
-7> 2019-12-22 12:20:21.819 7fce7d8c9700 0 mds.0.cache creating system inode with ino:0x100
-6> 2019-12-22 12:20:21.819 7fce7d8c9700 0 mds.0.cache creating system inode with ino:0x1
-5> 2019-12-22 12:20:21.823 7fce7e0ca700 2 mds.0.333 Booting: 2: replaying mds log
-4> 2019-12-22 12:20:21.823 7fce7e0ca700 2 mds.0.333 Booting: 2: waiting for purge queue recovered
-3> 2019-12-22 12:20:22.055 7fce7c8c7700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
-2> 2019-12-22 12:20:22.055 7fce7c8c7700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
-1> 2019-12-22 12:20:22.055 7fce7c8c7700 0 mds.0.journal journal ambig_subtrees:
0> 2019-12-22 12:20:22.059 7fce7c8c7700 -1 *** Caught signal (Segmentation fault) **
in thread 7fce7c8c7700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7fce8d26d890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mds.ceph1.log
--- end dump of recent events ---
2019-12-22 12:20:22.607 7f550c06c2c0 0 set uid:gid to 64045:64045 (ceph:ceph)
2019-12-22 12:20:22.607 7f550c06c2c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7073
2019-12-22 12:20:22.611 7f550c06c2c0 0 pidfile_write: ignore empty --pid-file
2019-12-22 12:20:22.615 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 333 from mon.1
2019-12-22 12:20:22.819 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 334 from mon.1
2019-12-22 12:20:22.819 7f54fa1bc700 1 mds.ceph1 Map has assigned me to become a standby
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 335 from mon.1
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 handle_mds_map i am now mds.0.335
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 handle_mds_map state change up:boot --> up:replay
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 replay_start
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 recovery set is
2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 waiting for osdmap 6045 (which blacklists prior instance)
2019-12-22 12:20:22.867 7f54f39af700 0 mds.0.cache creating system inode with ino:0x100
2019-12-22 12:20:22.867 7f54f39af700 0 mds.0.cache creating system inode with ino:0x1
2019-12-22 12:20:22.903 7f54f21ac700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
2019-12-22 12:20:22.903 7f54f21ac700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
2019-12-22 12:20:22.903 7f54f21ac700 0 mds.0.journal journal ambig_subtrees:
2019-12-22 12:20:22.907 7f54f21ac700 -1 *** Caught signal (Segmentation fault) **
in thread 7f54f21ac700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7f5502b52890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-272> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command assert hook 0x55d06f8ac500
-271> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command abort hook 0x55d06f8ac500
-270> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perfcounters_dump hook 0x55d06f8ac500
-269> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command 1 hook 0x55d06f8ac500
-268> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perf dump hook 0x55d06f8ac500
-267> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perfcounters_schema hook 0x55d06f8ac500
-266> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perf histogram dump hook 0x55d06f8ac500
-265> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command 2 hook 0x55d06f8ac500
-264> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perf schema hook 0x55d06f8ac500
-263> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perf histogram schema hook 0x55d06f8ac500
-262> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command perf reset hook 0x55d06f8ac500
-261> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config show hook 0x55d06f8ac500
-260> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config help hook 0x55d06f8ac500
-259> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config set hook 0x55d06f8ac500
-258> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config unset hook 0x55d06f8ac500
-257> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config get hook 0x55d06f8ac500
-256> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config diff hook 0x55d06f8ac500
-255> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command config diff get hook 0x55d06f8ac500
-254> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command log flush hook 0x55d06f8ac500
-253> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command log dump hook 0x55d06f8ac500
-252> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command log reopen hook 0x55d06f8ac500
-251> 2019-12-22 12:20:22.595 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump_mempools hook 0x55d070544068
-250> 2019-12-22 12:20:22.603 7f550c06c2c0 10 monclient: get_monmap_and_config
-249> 2019-12-22 12:20:22.603 7f550c06c2c0 10 monclient: build_initial_monmap
-248> 2019-12-22 12:20:22.603 7f550c06c2c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:22.607767
created 2019-12-22 12:20:22.607767
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-247> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding auth protocol: cephx
-246> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding auth protocol: cephx
-245> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding auth protocol: cephx
-244> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-243> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-242> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-241> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-240> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-239> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-238> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-237> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-236> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-235> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-234> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: crc
-233> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x55d06f97e140) adding con mode: secure
-232> 2019-12-22 12:20:22.603 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-231> 2019-12-22 12:20:22.603 7f550c06c2c0 10 monclient: init
-230> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding auth protocol: cephx
-229> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding auth protocol: cephx
-228> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding auth protocol: cephx
-227> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-226> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-225> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-224> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-223> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-222> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-221> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-220> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-219> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-218> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-217> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: crc
-216> 2019-12-22 12:20:22.603 7f550c06c2c0 5 AuthRegistry(0x7ffe15922db8) adding con mode: secure
-215> 2019-12-22 12:20:22.603 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-214> 2019-12-22 12:20:22.603 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-213> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient: _reopen_session rank -1
-212> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x55d06f9d5680 addr v1:10.10.0.20:6789/0
-211> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): picked mon.noname-a con 0x55d06f9d4d80 addr v2:10.10.0.10:3300/0
-210> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): picked mon.noname-b con 0x55d06f9d5b00 addr v2:10.10.0.20:3300/0
-209> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): start opening mon connection
-208> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): start opening mon connection
-207> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): _renew_subs
-206> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient(hunting): authenticate will time out at 2019-12-22 12:25:22.609116
-205> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): get_auth_request con 0x55d06f9d4d80 auth_method 0
-204> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-203> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): _init_auth method 2
-202> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): handle_auth_reply_more payload 9
-201> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-200> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-199> 2019-12-22 12:20:22.607 7f54fe3ac700 10 monclient(hunting): get_auth_request con 0x55d06f9d5b00 auth_method 0
-198> 2019-12-22 12:20:22.607 7f54fe3ac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-197> 2019-12-22 12:20:22.607 7f54fe3ac700 10 monclient(hunting): _init_auth method 2
-196> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient(hunting): handle_auth_done global_id 554637 payload 386
-195> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient: _finish_hunting 0
-194> 2019-12-22 12:20:22.607 7f54fd3aa700 1 monclient: found mon.noname-a
-193> 2019-12-22 12:20:22.607 7f54fd3aa700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.10:3300/0
-192> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: handle_monmap mon_map magic: 0 v1
-191> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: got monmap 1 from mon.noname-a (according to old e1)
-190> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-189> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: _finish_auth 0
-188> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:52.610435)
-187> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-186> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: handle_config config(2 keys) v1
-185> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: handle_monmap mon_map magic: 0 v1
-184> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: got monmap 1 from mon.ceph1 (according to old e1)
-183> 2019-12-22 12:20:22.607 7f54fb3a6700 4 set_mon_vals no callback set
-182> 2019-12-22 12:20:22.607 7f54fcba9700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-181> 2019-12-22 12:20:22.607 7f54fb3a6700 10 set_mon_vals mon_target_pg_per_osd = 50
-180> 2019-12-22 12:20:22.607 7f550c06c2c0 5 monclient: authenticate success, global_id 554637
-179> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient: get_monmap_and_config success
-178> 2019-12-22 12:20:22.607 7f550c06c2c0 10 monclient: shutdown
-177> 2019-12-22 12:20:22.607 7f550c06c2c0 0 set uid:gid to 64045:64045 (ceph:ceph)
-176> 2019-12-22 12:20:22.607 7f550c06c2c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7073
-175> 2019-12-22 12:20:22.611 7f550c06c2c0 0 pidfile_write: ignore empty --pid-file
-174> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) init /var/run/ceph/ceph-mds.ceph1.asok
-173> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) bind_and_listen /var/run/ceph/ceph-mds.ceph1.asok
-172> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) register_command 0 hook 0x55d06f8aaac8
-171> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) register_command version hook 0x55d06f8aaac8
-170> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) register_command git_version hook 0x55d06f8aaac8
-169> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) register_command help hook 0x55d06f8ac480
-168> 2019-12-22 12:20:22.611 7f550c06c2c0 5 asok(0x55d06f93a000) register_command get_command_descriptions hook 0x55d06f8ac490
-167> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding auth protocol: cephx
-166> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding auth protocol: cephx
-165> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding auth protocol: cephx
-164> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-163> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-162> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-161> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-160> 2019-12-22 12:20:22.611 7f54fbba7700 5 asok(0x55d06f93a000) entry start
-159> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-158> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-157> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-156> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-155> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-154> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-153> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: crc
-152> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x55d06f97ea40) adding con mode: secure
-151> 2019-12-22 12:20:22.611 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-150> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient: build_initial_monmap
-149> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:22.615334
created 2019-12-22 12:20:22.615334
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-148> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient: init
-147> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding auth protocol: cephx
-146> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding auth protocol: cephx
-145> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding auth protocol: cephx
-144> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-143> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-142> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-141> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-140> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-139> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-138> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-137> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-136> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-135> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-134> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: crc
-133> 2019-12-22 12:20:22.611 7f550c06c2c0 5 AuthRegistry(0x7ffe15924b48) adding con mode: secure
-132> 2019-12-22 12:20:22.611 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-131> 2019-12-22 12:20:22.611 7f550c06c2c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-130> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient: _reopen_session rank -1
-129> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient(hunting): picked mon.noname-c-legacy con 0x55d06f9d5680 addr v1:10.10.0.30:6789/0
-128> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x55d06f9d4d80 addr v1:10.10.0.20:6789/0
-127> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient(hunting): picked mon.noname-b con 0x55d06f9d5f80 addr v2:10.10.0.20:3300/0
-126> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient(hunting): start opening mon connection
-125> 2019-12-22 12:20:22.611 7f550c06c2c0 10 monclient(hunting): _renew_subs
-124> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): get_auth_request con 0x55d06f9d5f80 auth_method 0
-123> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-122> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): _init_auth method 2
-121> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): handle_auth_reply_more payload 9
-120> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-119> 2019-12-22 12:20:22.611 7f54fe3ac700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-118> 2019-12-22 12:20:22.611 7f54fa1bc700 10 monclient(hunting): _init_auth method 2
-117> 2019-12-22 12:20:22.611 7f54fa1bc700 10 monclient(hunting): my global_id is 554762
-116> 2019-12-22 12:20:22.615 7f54fe3ac700 10 monclient(hunting): handle_auth_done global_id 554743 payload 1102
-115> 2019-12-22 12:20:22.615 7f54fe3ac700 10 monclient: _finish_hunting 0
-114> 2019-12-22 12:20:22.615 7f54fe3ac700 1 monclient: found mon.noname-b
-113> 2019-12-22 12:20:22.615 7f54fe3ac700 10 monclient: _send_mon_message to mon.noname-b at v2:10.10.0.20:3300/0
-112> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: handle_monmap mon_map magic: 0 v1
-111> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1)
-110> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-109> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: _finish_auth 0
-108> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:52.617728)
-107> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: _send_mon_message to mon.ceph2 at v2:10.10.0.20:3300/0
-106> 2019-12-22 12:20:22.615 7f550c06c2c0 5 monclient: authenticate success, global_id 554743
-105> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: handle_config config(2 keys) v1
-104> 2019-12-22 12:20:22.615 7f550c06c2c0 10 monclient: wait_auth_rotating waiting (until 2019-12-22 12:20:52.617799)
-103> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: handle_monmap mon_map magic: 0 v1
-102> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: got monmap 1 from mon.ceph2 (according to old e1)
-101> 2019-12-22 12:20:22.615 7f54f89b9700 4 set_mon_vals no callback set
-100> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-99> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: _finish_auth 0
-98> 2019-12-22 12:20:22.615 7f54fa1bc700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-22 12:19:52.618067)
-97> 2019-12-22 12:20:22.615 7f550c06c2c0 10 monclient: wait_auth_rotating done
-96> 2019-12-22 12:20:22.615 7f550c06c2c0 10 monclient: _renew_subs
-95> 2019-12-22 12:20:22.615 7f550c06c2c0 10 monclient: _send_mon_message to mon.ceph2 at v2:10.10.0.20:3300/0
-94> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command status hook 0x55d06f8ac7c0
-93> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump_ops_in_flight hook 0x55d06f8ac7c0
-92> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command ops hook 0x55d06f8ac7c0
-91> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump_blocked_ops hook 0x55d06f8ac7c0
-90> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump_historic_ops hook 0x55d06f8ac7c0
-89> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump_historic_ops_by_duration hook 0x55d06f8ac7c0
-88> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command scrub_path hook 0x55d06f8ac7c0
-87> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command tag path hook 0x55d06f8ac7c0
-86> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command flush_path hook 0x55d06f8ac7c0
-85> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command export dir hook 0x55d06f8ac7c0
-84> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump cache hook 0x55d06f8ac7c0
-83> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command cache status hook 0x55d06f8ac7c0
-82> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump tree hook 0x55d06f8ac7c0
-81> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump loads hook 0x55d06f8ac7c0
-80> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump snaps hook 0x55d06f8ac7c0
-79> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command session evict hook 0x55d06f8ac7c0
-78> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command osdmap barrier hook 0x55d06f8ac7c0
-77> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command session ls hook 0x55d06f8ac7c0
-76> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command flush journal hook 0x55d06f8ac7c0
-75> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command force_readonly hook 0x55d06f8ac7c0
-74> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command get subtrees hook 0x55d06f8ac7c0
-73> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dirfrag split hook 0x55d06f8ac7c0
-72> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dirfrag merge hook 0x55d06f8ac7c0
-71> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dirfrag ls hook 0x55d06f8ac7c0
-70> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command openfiles ls hook 0x55d06f8ac7c0
-69> 2019-12-22 12:20:22.615 7f550c06c2c0 5 asok(0x55d06f93a000) register_command dump inode hook 0x55d06f8ac7c0
-68> 2019-12-22 12:20:22.615 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 333 from mon.1
-67> 2019-12-22 12:20:22.615 7f54f71b6700 5 mds.beacon.ceph1 Sending beacon up:boot seq 1
-66> 2019-12-22 12:20:22.615 7f54fa1bc700 4 mgrc handle_mgr_map Got map version 130
-65> 2019-12-22 12:20:22.615 7f54fa1bc700 4 mgrc handle_mgr_map Active mgr is now [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-64> 2019-12-22 12:20:22.615 7f54fa1bc700 4 mgrc reconnect Starting new session with [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-63> 2019-12-22 12:20:22.619 7f54fdbab700 10 monclient: get_auth_request con 0x55d06f9d7600 auth_method 0
-62> 2019-12-22 12:20:22.619 7f54fa1bc700 4 mgrc handle_mgr_configure stats_period=5
-61> 2019-12-22 12:20:22.619 7f54fa1bc700 4 mgrc handle_mgr_configure updated stats threshold: 5
-60> 2019-12-22 12:20:22.635 7f54f71b6700 10 monclient: _send_mon_message to mon.ceph2 at v2:10.10.0.20:3300/0
-59> 2019-12-22 12:20:22.819 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 334 from mon.1
-58> 2019-12-22 12:20:22.819 7f54fa1bc700 5 mds.beacon.ceph1 set_want_state: up:boot -> up:standby
-57> 2019-12-22 12:20:22.819 7f54fa1bc700 1 mds.ceph1 Map has assigned me to become a standby
-56> 2019-12-22 12:20:22.823 7f54fe3ac700 5 mds.beacon.ceph1 received beacon reply up:boot seq 1 rtt 0.207997
-55> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.ceph1 Updating MDS map to version 335 from mon.1
-54> 2019-12-22 12:20:22.827 7f54fa1bc700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-53> 2019-12-22 12:20:22.827 7f54fa1bc700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-52> 2019-12-22 12:20:22.827 7f54fa1bc700 5 asok(0x55d06f93a000) register_command objecter_requests hook 0x55d06f8ac320
-51> 2019-12-22 12:20:22.827 7f54fa1bc700 10 monclient: _renew_subs
-50> 2019-12-22 12:20:22.827 7f54fa1bc700 10 monclient: _send_mon_message to mon.ceph2 at v2:10.10.0.20:3300/0
-49> 2019-12-22 12:20:22.827 7f54fa1bc700 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
-48> 2019-12-22 12:20:22.827 7f54fa1bc700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-47> 2019-12-22 12:20:22.827 7f54fa1bc700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-46> 2019-12-22 12:20:22.827 7f54fa1bc700 4 mds.0.0 handle_osd_map epoch 0, 0 new blacklist entries
-45> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 handle_mds_map i am now mds.0.335
-44> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 handle_mds_map state change up:boot --> up:replay
-43> 2019-12-22 12:20:22.827 7f54fa1bc700 5 mds.beacon.ceph1 set_want_state: up:standby -> up:replay
-42> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 replay_start
-41> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 recovery set is
-40> 2019-12-22 12:20:22.827 7f54fa1bc700 1 mds.0.335 waiting for osdmap 6045 (which blacklists prior instance)
-39> 2019-12-22 12:20:22.831 7f54fa1bc700 4 mds.0.335 handle_osd_map epoch 6045, 0 new blacklist entries
-38> 2019-12-22 12:20:22.831 7f54fa1bc700 10 monclient: _renew_subs
-37> 2019-12-22 12:20:22.831 7f54fa1bc700 10 monclient: _send_mon_message to mon.ceph2 at v2:10.10.0.20:3300/0
-36> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: opening inotable
-35> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: opening sessionmap
-34> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: opening mds log
-33> 2019-12-22 12:20:22.831 7f54f39af700 5 mds.0.log open discovering log bounds
-32> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: opening purge queue (async)
-31> 2019-12-22 12:20:22.831 7f54f39af700 4 mds.0.purge_queue open: opening
-30> 2019-12-22 12:20:22.831 7f54f31ae700 4 mds.0.journalpointer Reading journal pointer '400.00000000'
-29> 2019-12-22 12:20:22.831 7f54f39af700 1 mds.0.journaler.pq(ro) recover start
-28> 2019-12-22 12:20:22.831 7f54f39af700 1 mds.0.journaler.pq(ro) read_head
-27> 2019-12-22 12:20:22.831 7f54fd3aa700 10 monclient: get_auth_request con 0x55d070698d80 auth_method 0
-26> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: loading open file table (async)
-25> 2019-12-22 12:20:22.831 7f54fdbab700 10 monclient: get_auth_request con 0x55d070699200 auth_method 0
-24> 2019-12-22 12:20:22.831 7f54f39af700 2 mds.0.335 Booting: 0: opening snap table
-23> 2019-12-22 12:20:22.831 7f54fd3aa700 10 monclient: get_auth_request con 0x55d070699680 auth_method 0
-22> 2019-12-22 12:20:22.831 7f54fe3ac700 10 monclient: get_auth_request con 0x55d070699b00 auth_method 0
-21> 2019-12-22 12:20:22.831 7f54f31ae700 1 mds.0.journaler.mdlog(ro) recover start
-20> 2019-12-22 12:20:22.831 7f54f31ae700 1 mds.0.journaler.mdlog(ro) read_head
-19> 2019-12-22 12:20:22.831 7f54f31ae700 4 mds.0.log Waiting for journal 0x200 to recover...
-18> 2019-12-22 12:20:22.831 7f54f39af700 1 mds.0.journaler.mdlog(ro) _finish_read_head loghead(trim 805306368, expire 808964827, write 812539373, stream_format 1). probing for end of log (from 812539373)...
-17> 2019-12-22 12:20:22.831 7f54f39af700 1 mds.0.journaler.mdlog(ro) probing for end of the log
-16> 2019-12-22 12:20:22.855 7f54f49b1700 1 mds.0.journaler.pq(ro) _finish_read_head loghead(trim 4194304, expire 4320033, write 4320033, stream_format 1). probing for end of log (from 4320033)...
-15> 2019-12-22 12:20:22.855 7f54f49b1700 1 mds.0.journaler.pq(ro) probing for end of the log
-14> 2019-12-22 12:20:22.855 7f54f49b1700 1 mds.0.journaler.pq(ro) _finish_probe_end write_pos = 4320033 (header had 4320033). recovered.
-13> 2019-12-22 12:20:22.855 7f54f49b1700 4 mds.0.purge_queue operator(): open complete
-12> 2019-12-22 12:20:22.855 7f54f49b1700 1 mds.0.journaler.pq(ro) set_writeable
-11> 2019-12-22 12:20:22.867 7f54f39af700 1 mds.0.journaler.mdlog(ro) _finish_probe_end write_pos = 812539516 (header had 812539373). recovered.
-10> 2019-12-22 12:20:22.867 7f54f31ae700 4 mds.0.log Journal 0x200 recovered.
-9> 2019-12-22 12:20:22.867 7f54f31ae700 4 mds.0.log Recovered journal 0x200 in format 1
-8> 2019-12-22 12:20:22.867 7f54f39af700 2 mds.0.335 Booting: 1: loading/discovering base inodes
-7> 2019-12-22 12:20:22.867 7f54f39af700 0 mds.0.cache creating system inode with ino:0x100
-6> 2019-12-22 12:20:22.867 7f54f39af700 0 mds.0.cache creating system inode with ino:0x1
-5> 2019-12-22 12:20:22.867 7f54f39af700 2 mds.0.335 Booting: 2: replaying mds log
-4> 2019-12-22 12:20:22.867 7f54f39af700 2 mds.0.335 Booting: 2: waiting for purge queue recovered
-3> 2019-12-22 12:20:22.903 7f54f21ac700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
-2> 2019-12-22 12:20:22.903 7f54f21ac700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
-1> 2019-12-22 12:20:22.903 7f54f21ac700 0 mds.0.journal journal ambig_subtrees:
0> 2019-12-22 12:20:22.907 7f54f21ac700 -1 *** Caught signal (Segmentation fault) **
in thread 7f54f21ac700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7f5502b52890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mds.ceph1.log
--- end dump of recent events ---
2019-12-22 12:20:23.391 7f48efc612c0 0 set uid:gid to 64045:64045 (ceph:ceph)
2019-12-22 12:20:23.391 7f48efc612c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7106
2019-12-22 12:20:23.391 7f48efc612c0 0 pidfile_write: ignore empty --pid-file
2019-12-22 12:20:23.395 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 335 from mon.0
2019-12-22 12:20:27.803 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 336 from mon.0
2019-12-22 12:20:27.803 7f48dddb1700 1 mds.ceph1 Map has assigned me to become a standby
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 337 from mon.0
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 handle_mds_map i am now mds.0.337
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 handle_mds_map state change up:boot --> up:replay
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 replay_start
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 recovery set is
2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 waiting for osdmap 6046 (which blacklists prior instance)
2019-12-22 12:20:27.819 7f48d6da3700 0 mds.0.cache creating system inode with ino:0x100
2019-12-22 12:20:27.819 7f48d6da3700 0 mds.0.cache creating system inode with ino:0x1
2019-12-22 12:20:27.895 7f48d5da1700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
2019-12-22 12:20:27.895 7f48d5da1700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
2019-12-22 12:20:27.895 7f48d5da1700 0 mds.0.journal journal ambig_subtrees:
2019-12-22 12:20:27.899 7f48d5da1700 -1 *** Caught signal (Segmentation fault) **
in thread 7f48d5da1700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7f48e6747890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-270> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command assert hook 0x56188bbc8500
-269> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command abort hook 0x56188bbc8500
-268> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perfcounters_dump hook 0x56188bbc8500
-267> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command 1 hook 0x56188bbc8500
-266> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perf dump hook 0x56188bbc8500
-265> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perfcounters_schema hook 0x56188bbc8500
-264> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perf histogram dump hook 0x56188bbc8500
-263> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command 2 hook 0x56188bbc8500
-262> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perf schema hook 0x56188bbc8500
-261> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perf histogram schema hook 0x56188bbc8500
-260> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command perf reset hook 0x56188bbc8500
-259> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config show hook 0x56188bbc8500
-258> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config help hook 0x56188bbc8500
-257> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config set hook 0x56188bbc8500
-256> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config unset hook 0x56188bbc8500
-255> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config get hook 0x56188bbc8500
-254> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config diff hook 0x56188bbc8500
-253> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command config diff get hook 0x56188bbc8500
-252> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command log flush hook 0x56188bbc8500
-251> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command log dump hook 0x56188bbc8500
-250> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command log reopen hook 0x56188bbc8500
-249> 2019-12-22 12:20:23.379 7f48efc612c0 5 asok(0x56188bc56000) register_command dump_mempools hook 0x56188c860068
-248> 2019-12-22 12:20:23.383 7f48efc612c0 10 monclient: get_monmap_and_config
-247> 2019-12-22 12:20:23.383 7f48efc612c0 10 monclient: build_initial_monmap
-246> 2019-12-22 12:20:23.383 7f48efc612c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:23.388937
created 2019-12-22 12:20:23.388937
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-245> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding auth protocol: cephx
-244> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding auth protocol: cephx
-243> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding auth protocol: cephx
-242> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-241> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-240> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-239> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-238> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-237> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-236> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-235> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-234> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-233> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-232> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: crc
-231> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x56188bc9a140) adding con mode: secure
-230> 2019-12-22 12:20:23.387 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-229> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient: init
-228> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding auth protocol: cephx
-227> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding auth protocol: cephx
-226> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding auth protocol: cephx
-225> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-224> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-223> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-222> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-221> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-220> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-219> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-218> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-217> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-216> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-215> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: crc
-214> 2019-12-22 12:20:23.387 7f48efc612c0 5 AuthRegistry(0x7ffc3226a458) adding con mode: secure
-213> 2019-12-22 12:20:23.387 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-212> 2019-12-22 12:20:23.387 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-211> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient: _reopen_session rank -1
-210> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x56188bcf1680 addr v1:10.10.0.20:6789/0
-209> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): picked mon.noname-a-legacy con 0x56188bcf0d80 addr v1:10.10.0.10:6789/0
-208> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): picked mon.noname-a con 0x56188bcf1b00 addr v2:10.10.0.10:3300/0
-207> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): start opening mon connection
-206> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): _renew_subs
-205> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient(hunting): authenticate will time out at 2019-12-22 12:25:23.390276
-204> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): get_auth_request con 0x56188bcf1b00 auth_method 0
-203> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-202> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): _init_auth method 2
-201> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more payload 9
-200> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-199> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-198> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient(hunting): handle_auth_done global_id 554640 payload 386
-197> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient: _finish_hunting 0
-196> 2019-12-22 12:20:23.387 7f48e1fa1700 1 monclient: found mon.noname-a
-195> 2019-12-22 12:20:23.387 7f48e1fa1700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.10:3300/0
-194> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: discarding stray monitor message auth_reply(proto 2 0 (0) Success) v1
-193> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: handle_monmap mon_map magic: 0 v1
-192> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: got monmap 1 from mon.noname-a (according to old e1)
-191> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-190> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: _finish_auth 0
-189> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:53.391286)
-188> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-187> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: handle_config config(2 keys) v1
-186> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: handle_monmap mon_map magic: 0 v1
-185> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: got monmap 1 from mon.ceph1 (according to old e1)
-184> 2019-12-22 12:20:23.387 7f48e079e700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-183> 2019-12-22 12:20:23.387 7f48efc612c0 5 monclient: authenticate success, global_id 554640
-182> 2019-12-22 12:20:23.387 7f48def9b700 4 set_mon_vals no callback set
-181> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient: get_monmap_and_config success
-180> 2019-12-22 12:20:23.387 7f48efc612c0 10 monclient: shutdown
-179> 2019-12-22 12:20:23.387 7f48def9b700 10 set_mon_vals mon_target_pg_per_osd = 50
-178> 2019-12-22 12:20:23.391 7f48efc612c0 0 set uid:gid to 64045:64045 (ceph:ceph)
-177> 2019-12-22 12:20:23.391 7f48efc612c0 0 ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable), process ceph-mds, pid 7106
-176> 2019-12-22 12:20:23.391 7f48efc612c0 0 pidfile_write: ignore empty --pid-file
-175> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) init /var/run/ceph/ceph-mds.ceph1.asok
-174> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) bind_and_listen /var/run/ceph/ceph-mds.ceph1.asok
-173> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) register_command 0 hook 0x56188bbc6ac8
-172> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) register_command version hook 0x56188bbc6ac8
-171> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) register_command git_version hook 0x56188bbc6ac8
-170> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) register_command help hook 0x56188bbc8380
-169> 2019-12-22 12:20:23.391 7f48efc612c0 5 asok(0x56188bc56000) register_command get_command_descriptions hook 0x56188bbc8460
-168> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding auth protocol: cephx
-167> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding auth protocol: cephx
-166> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding auth protocol: cephx
-165> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-164> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-163> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-162> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-161> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-160> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-159> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-158> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-157> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-156> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-155> 2019-12-22 12:20:23.391 7f48df79c700 5 asok(0x56188bc56000) entry start
-154> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: crc
-153> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x56188bc9aa40) adding con mode: secure
-152> 2019-12-22 12:20:23.391 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-151> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient: build_initial_monmap
-150> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient: monmap:
epoch 0
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-12-22 12:20:23.396016
created 2019-12-22 12:20:23.396016
min_mon_release 0 (unknown)
0: v1:10.10.0.10:6789/0 mon.noname-a-legacy
1: v1:10.10.0.20:6789/0 mon.noname-b-legacy
2: v1:10.10.0.30:6789/0 mon.noname-c-legacy
3: v2:10.10.0.10:3300/0 mon.noname-a
4: v2:10.10.0.20:3300/0 mon.noname-b
5: v2:10.10.0.30:3300/0 mon.noname-c

-149> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient: init
-148> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding auth protocol: cephx
-147> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding auth protocol: cephx
-146> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding auth protocol: cephx
-145> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-144> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-143> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-142> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-141> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-140> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-139> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-138> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-137> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-136> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-135> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: crc
-134> 2019-12-22 12:20:23.391 7f48efc612c0 5 AuthRegistry(0x7ffc3226c1e8) adding con mode: secure
-133> 2019-12-22 12:20:23.391 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-132> 2019-12-22 12:20:23.391 7f48efc612c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/mds/ceph-ceph1/keyring
-131> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient: _reopen_session rank -1
-130> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient(hunting): picked mon.noname-a-legacy con 0x56188bcf1680 addr v1:10.10.0.10:6789/0
-129> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient(hunting): picked mon.noname-b-legacy con 0x56188bcf0d80 addr v1:10.10.0.20:6789/0
-128> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient(hunting): picked mon.noname-a con 0x56188bcf1f80 addr v2:10.10.0.10:3300/0
-127> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient(hunting): start opening mon connection
-126> 2019-12-22 12:20:23.391 7f48efc612c0 10 monclient(hunting): _renew_subs
-125> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): get_auth_request con 0x56188bcf1f80 auth_method 0
-124> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2]
-123> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): _init_auth method 2
-122> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more payload 9
-121> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-120> 2019-12-22 12:20:23.391 7f48e1fa1700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-119> 2019-12-22 12:20:23.391 7f48dddb1700 10 monclient(hunting): _init_auth method 2
-118> 2019-12-22 12:20:23.391 7f48dddb1700 10 monclient(hunting): my global_id is 554649
-117> 2019-12-22 12:20:23.395 7f48e1fa1700 10 monclient(hunting): handle_auth_done global_id 554646 payload 1102
-116> 2019-12-22 12:20:23.395 7f48e1fa1700 10 monclient: _finish_hunting 0
-115> 2019-12-22 12:20:23.395 7f48e1fa1700 1 monclient: found mon.noname-a
-114> 2019-12-22 12:20:23.395 7f48e1fa1700 10 monclient: _send_mon_message to mon.noname-a at v2:10.10.0.10:3300/0
-113> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: handle_monmap mon_map magic: 0 v1
-112> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: got monmap 1 from mon.noname-a (according to old e1)
-111> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-110> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: _finish_auth 0
-109> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2019-12-22 12:19:53.397436)
-108> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-107> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: handle_config config(2 keys) v1
-106> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: handle_monmap mon_map magic: 0 v1
-105> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: got monmap 1 from mon.ceph1 (according to old e1)
-104> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: dump:
epoch 1
fsid 3ac4d7e5-98ca-4b75-bb88-02386caa5793
last_changed 2019-10-20 21:47:34.775628
created 2019-10-20 21:47:34.775628
min_mon_release 14 (nautilus)
0: [v2:10.10.0.10:3300/0,v1:10.10.0.10:6789/0] mon.ceph1
1: [v2:10.10.0.20:3300/0,v1:10.10.0.20:6789/0] mon.ceph2
2: [v2:10.10.0.30:3300/0,v1:10.10.0.30:6789/0] mon.ceph3

-103> 2019-12-22 12:20:23.395 7f48dc5ae700 4 set_mon_vals no callback set
-102> 2019-12-22 12:20:23.395 7f48efc612c0 5 monclient: authenticate success, global_id 554646
-101> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: _finish_auth 0
-100> 2019-12-22 12:20:23.395 7f48dddb1700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-22 12:19:53.397749)
-99> 2019-12-22 12:20:23.395 7f48efc612c0 10 monclient: wait_auth_rotating done
-98> 2019-12-22 12:20:23.395 7f48efc612c0 10 monclient: _renew_subs
-97> 2019-12-22 12:20:23.395 7f48efc612c0 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-96> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command status hook 0x56188bbc87d0
-95> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump_ops_in_flight hook 0x56188bbc87d0
-94> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command ops hook 0x56188bbc87d0
-93> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump_blocked_ops hook 0x56188bbc87d0
-92> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump_historic_ops hook 0x56188bbc87d0
-91> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump_historic_ops_by_duration hook 0x56188bbc87d0
-90> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command scrub_path hook 0x56188bbc87d0
-89> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command tag path hook 0x56188bbc87d0
-88> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command flush_path hook 0x56188bbc87d0
-87> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command export dir hook 0x56188bbc87d0
-86> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump cache hook 0x56188bbc87d0
-85> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command cache status hook 0x56188bbc87d0
-84> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump tree hook 0x56188bbc87d0
-83> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump loads hook 0x56188bbc87d0
-82> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump snaps hook 0x56188bbc87d0
-81> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command session evict hook 0x56188bbc87d0
-80> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command osdmap barrier hook 0x56188bbc87d0
-79> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command session ls hook 0x56188bbc87d0
-78> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command flush journal hook 0x56188bbc87d0
-77> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command force_readonly hook 0x56188bbc87d0
-76> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command get subtrees hook 0x56188bbc87d0
-75> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dirfrag split hook 0x56188bbc87d0
-74> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dirfrag merge hook 0x56188bbc87d0
-73> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dirfrag ls hook 0x56188bbc87d0
-72> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command openfiles ls hook 0x56188bbc87d0
-71> 2019-12-22 12:20:23.395 7f48efc612c0 5 asok(0x56188bc56000) register_command dump inode hook 0x56188bbc87d0
-70> 2019-12-22 12:20:23.395 7f48dadab700 5 mds.beacon.ceph1 Sending beacon up:boot seq 1
-69> 2019-12-22 12:20:23.395 7f48dadab700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-68> 2019-12-22 12:20:23.395 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 335 from mon.0
-67> 2019-12-22 12:20:23.395 7f48dddb1700 4 mgrc handle_mgr_map Got map version 130
-66> 2019-12-22 12:20:23.395 7f48dddb1700 4 mgrc handle_mgr_map Active mgr is now [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-65> 2019-12-22 12:20:23.395 7f48dddb1700 4 mgrc reconnect Starting new session with [v2:10.10.0.20:6800/1434,v1:10.10.0.20:6801/1434]
-64> 2019-12-22 12:20:23.395 7f48e17a0700 10 monclient: get_auth_request con 0x56188bcf3600 auth_method 0
-63> 2019-12-22 12:20:23.399 7f48dddb1700 4 mgrc handle_mgr_configure stats_period=5
-62> 2019-12-22 12:20:23.399 7f48dddb1700 4 mgrc handle_mgr_configure updated stats threshold: 5
-61> 2019-12-22 12:20:27.395 7f48dadab700 5 mds.beacon.ceph1 Sending beacon up:boot seq 2
-60> 2019-12-22 12:20:27.395 7f48dadab700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-59> 2019-12-22 12:20:27.803 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 336 from mon.0
-58> 2019-12-22 12:20:27.803 7f48dddb1700 5 mds.beacon.ceph1 set_want_state: up:boot -> up:standby
-57> 2019-12-22 12:20:27.803 7f48dddb1700 1 mds.ceph1 Map has assigned me to become a standby
-56> 2019-12-22 12:20:27.803 7f48e1fa1700 5 mds.beacon.ceph1 received beacon reply up:boot seq 2 rtt 0.407994
-55> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.ceph1 Updating MDS map to version 337 from mon.0
-54> 2019-12-22 12:20:27.811 7f48dddb1700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-53> 2019-12-22 12:20:27.811 7f48dddb1700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-52> 2019-12-22 12:20:27.811 7f48dddb1700 5 asok(0x56188bc56000) register_command objecter_requests hook 0x56188bbc8980
-51> 2019-12-22 12:20:27.811 7f48dddb1700 10 monclient: _renew_subs
-50> 2019-12-22 12:20:27.811 7f48dddb1700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-49> 2019-12-22 12:20:27.811 7f48dddb1700 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
-48> 2019-12-22 12:20:27.811 7f48dddb1700 4 mds.0.purge_queue operator(): data pool 11 not found in OSDMap
-47> 2019-12-22 12:20:27.811 7f48dddb1700 4 mds.0.purge_queue operator(): data pool 12 not found in OSDMap
-46> 2019-12-22 12:20:27.811 7f48dddb1700 4 mds.0.0 handle_osd_map epoch 0, 0 new blacklist entries
-45> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 handle_mds_map i am now mds.0.337
-44> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 handle_mds_map state change up:boot --> up:replay
-43> 2019-12-22 12:20:27.811 7f48dddb1700 5 mds.beacon.ceph1 set_want_state: up:standby -> up:replay
-42> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 replay_start
-41> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 recovery set is
-40> 2019-12-22 12:20:27.811 7f48dddb1700 1 mds.0.337 waiting for osdmap 6046 (which blacklists prior instance)
-39> 2019-12-22 12:20:27.815 7f48dddb1700 4 mds.0.337 handle_osd_map epoch 6046, 0 new blacklist entries
-38> 2019-12-22 12:20:27.815 7f48dddb1700 10 monclient: _renew_subs
-37> 2019-12-22 12:20:27.815 7f48dddb1700 10 monclient: _send_mon_message to mon.ceph1 at v2:10.10.0.10:3300/0
-36> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: opening inotable
-35> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: opening sessionmap
-34> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: opening mds log
-33> 2019-12-22 12:20:27.815 7f48d75a4700 5 mds.0.log open discovering log bounds
-32> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: opening purge queue (async)
-31> 2019-12-22 12:20:27.815 7f48d75a4700 4 mds.0.purge_queue open: opening
-30> 2019-12-22 12:20:27.815 7f48d75a4700 1 mds.0.journaler.pq(ro) recover start
-29> 2019-12-22 12:20:27.815 7f48d75a4700 1 mds.0.journaler.pq(ro) read_head
-28> 2019-12-22 12:20:27.815 7f48d6da3700 4 mds.0.journalpointer Reading journal pointer '400.00000000'
-27> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: loading open file table (async)
-26> 2019-12-22 12:20:27.815 7f48e17a0700 10 monclient: get_auth_request con 0x56188c9ab200 auth_method 0
-25> 2019-12-22 12:20:27.815 7f48d75a4700 2 mds.0.337 Booting: 0: opening snap table
-24> 2019-12-22 12:20:27.815 7f48e0f9f700 10 monclient: get_auth_request con 0x56188c9aad80 auth_method 0
-23> 2019-12-22 12:20:27.815 7f48e0f9f700 10 monclient: get_auth_request con 0x56188c9ab680 auth_method 0
-22> 2019-12-22 12:20:27.815 7f48e1fa1700 10 monclient: get_auth_request con 0x56188c9abb00 auth_method 0
-21> 2019-12-22 12:20:27.815 7f48d85a6700 1 mds.0.journaler.pq(ro) _finish_read_head loghead(trim 4194304, expire 4320033, write 4320033, stream_format 1). probing for end of log (from 4320033)...
-20> 2019-12-22 12:20:27.815 7f48d85a6700 1 mds.0.journaler.pq(ro) probing for end of the log
-19> 2019-12-22 12:20:27.815 7f48d6da3700 1 mds.0.journaler.mdlog(ro) recover start
-18> 2019-12-22 12:20:27.815 7f48d6da3700 1 mds.0.journaler.mdlog(ro) read_head
-17> 2019-12-22 12:20:27.815 7f48d6da3700 4 mds.0.log Waiting for journal 0x200 to recover...
-16> 2019-12-22 12:20:27.815 7f48d85a6700 1 mds.0.journaler.pq(ro) _finish_probe_end write_pos = 4320033 (header had 4320033). recovered.
-15> 2019-12-22 12:20:27.815 7f48d85a6700 4 mds.0.purge_queue operator(): open complete
-14> 2019-12-22 12:20:27.815 7f48d85a6700 1 mds.0.journaler.pq(ro) set_writeable
-13> 2019-12-22 12:20:27.815 7f48d75a4700 1 mds.0.journaler.mdlog(ro) _finish_read_head loghead(trim 805306368, expire 808964827, write 812539373, stream_format 1). probing for end of log (from 812539373)...
-12> 2019-12-22 12:20:27.815 7f48d75a4700 1 mds.0.journaler.mdlog(ro) probing for end of the log
-11> 2019-12-22 12:20:27.819 7f48d75a4700 1 mds.0.journaler.mdlog(ro) _finish_probe_end write_pos = 812539516 (header had 812539373). recovered.
-10> 2019-12-22 12:20:27.819 7f48d6da3700 4 mds.0.log Journal 0x200 recovered.
-9> 2019-12-22 12:20:27.819 7f48d6da3700 4 mds.0.log Recovered journal 0x200 in format 1
-8> 2019-12-22 12:20:27.819 7f48d6da3700 2 mds.0.337 Booting: 1: loading/discovering base inodes
-7> 2019-12-22 12:20:27.819 7f48d6da3700 0 mds.0.cache creating system inode with ino:0x100
-6> 2019-12-22 12:20:27.819 7f48d6da3700 0 mds.0.cache creating system inode with ino:0x1
-5> 2019-12-22 12:20:27.819 7f48d75a4700 2 mds.0.337 Booting: 2: replaying mds log
-4> 2019-12-22 12:20:27.819 7f48d75a4700 2 mds.0.337 Booting: 2: waiting for purge queue recovered
-3> 2019-12-22 12:20:27.895 7f48d5da1700 -1 log_channel(cluster) log [ERR] : replayed ESubtreeMap at 809504954 subtree root 0x1 is not mine in cache (it's -2,-2)
-2> 2019-12-22 12:20:27.895 7f48d5da1700 0 mds.0.journal journal subtrees: {0x1=[],0x100=[]}
-1> 2019-12-22 12:20:27.895 7f48d5da1700 0 mds.0.journal journal ambig_subtrees:
0> 2019-12-22 12:20:27.899 7f48d5da1700 -1 *** Caught signal (Segmentation fault) **
in thread 7f48d5da1700 thread_name:md_log_replay

ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nautilus (stable)
1: (()+0x12890) [0x7f48e6747890]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mds.ceph1.log
--- end dump of recent events ---
(1-1/2)