Project

General

Profile

Bug #23421 ยป client.admin.269603.log

crash - Jos Collin, 06/19/2020 09:26 AM

 
2020-06-19T14:49:24.193+0530 7fbad58b7080 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-19T14:49:24.198+0530 7fbad58b7080 -1 init, newargv = 0x55fb8fee6c80 newargc=9
2020-06-19T14:49:24.230+0530 7fba997fa700 -1 client.4311 failed to remount (to trim kernel dentries): return code = 1
2020-06-19T14:49:24.230+0530 7fba997fa700 -1 client.4311 failed to remount for kernel dentry trimming; quitting!
2020-06-19T14:49:24.234+0530 7fba997fa700 -1 /home/jcollin/workspace/ceph/src/client/Client.cc: In function 'int Client::_do_remount(bool)' thread 7fba997fa700 time 2020-06-19T14:49:24.230486+0530
/home/jcollin/workspace/ceph/src/client/Client.cc: 4245: ceph_abort_msg("abort() called")

ceph version 16.0.0-2623-gfe183904a2c (fe183904a2caece314d8e167b4a20a455535197e) pacific (dev)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xf6) [0x7fbad790b646]
2: (Client::_do_remount(bool)+0x564) [0x55fb8e890fd2]
3: (Client::test_dentry_handling(bool)+0x28f) [0x55fb8e8c8803]
4: (()+0x289877) [0x55fb8e83a877]
5: (Thread::entry_wrapper()+0x78) [0x7fbad78a6f1e]
6: (Thread::_entry_func(void*)+0x18) [0x7fbad78a6e9c]
7: (()+0x94e2) [0x7fbad5f164e2]
8: (clone()+0x43) [0x7fbad5a6a643]

2020-06-19T14:49:24.238+0530 7fba997fa700 -1 *** Caught signal (Aborted) **
in thread 7fba997fa700 thread_name:tester

ceph version 16.0.0-2623-gfe183904a2c (fe183904a2caece314d8e167b4a20a455535197e) pacific (dev)
1: (()+0x414512) [0x55fb8e9c5512]
2: (()+0x14b20) [0x7fbad5f21b20]
3: (gsignal()+0x145) [0x7fbad59a5625]
4: (abort()+0x12b) [0x7fbad598e8d9]
5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x37a) [0x7fbad790b8ca]
6: (Client::_do_remount(bool)+0x564) [0x55fb8e890fd2]
7: (Client::test_dentry_handling(bool)+0x28f) [0x55fb8e8c8803]
8: (()+0x289877) [0x55fb8e83a877]
9: (Thread::entry_wrapper()+0x78) [0x7fbad78a6f1e]
10: (Thread::_entry_func(void*)+0x18) [0x7fbad78a6e9c]
11: (()+0x94e2) [0x7fbad5f164e2]
12: (clone()+0x43) [0x7fbad5a6a643]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-243> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command assert hook 0x55fb8fe3d3a0
-242> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command abort hook 0x55fb8fe3d3a0
-241> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command leak_some_memory hook 0x55fb8fe3d3a0
-240> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perfcounters_dump hook 0x55fb8fe3d3a0
-239> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command 1 hook 0x55fb8fe3d3a0
-238> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perf dump hook 0x55fb8fe3d3a0
-237> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perfcounters_schema hook 0x55fb8fe3d3a0
-236> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perf histogram dump hook 0x55fb8fe3d3a0
-235> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command 2 hook 0x55fb8fe3d3a0
-234> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perf schema hook 0x55fb8fe3d3a0
-233> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perf histogram schema hook 0x55fb8fe3d3a0
-232> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command perf reset hook 0x55fb8fe3d3a0
-231> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config show hook 0x55fb8fe3d3a0
-230> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config help hook 0x55fb8fe3d3a0
-229> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config set hook 0x55fb8fe3d3a0
-228> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config unset hook 0x55fb8fe3d3a0
-227> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config get hook 0x55fb8fe3d3a0
-226> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config diff hook 0x55fb8fe3d3a0
-225> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command config diff get hook 0x55fb8fe3d3a0
-224> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command injectargs hook 0x55fb8fe3d3a0
-223> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command log flush hook 0x55fb8fe3d3a0
-222> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command log dump hook 0x55fb8fe3d3a0
-221> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command log reopen hook 0x55fb8fe3d3a0
-220> 2020-06-19T14:49:24.136+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command dump_mempools hook 0x55fb8fe3f7d8
-219> 2020-06-19T14:49:24.176+0530 7fbad58b7080 1 lockdep start
-218> 2020-06-19T14:49:24.176+0530 7fbad58b7080 -1 WARNING: all dangerous and experimental features are enabled.
-217> 2020-06-19T14:49:24.176+0530 7fbad58b7080 1 lockdep using id 0
-216> 2020-06-19T14:49:24.176+0530 7fbad58b7080 1 lockdep using id 1
-215> 2020-06-19T14:49:24.176+0530 7fbad58b7080 1 lockdep using id 2
-214> 2020-06-19T14:49:24.176+0530 7fbad58b7080 1 lockdep using id 3
-213> 2020-06-19T14:49:24.176+0530 7fbad58b7080 10 monclient: get_monmap_and_config
-212> 2020-06-19T14:49:24.176+0530 7fbad58b7080 10 monclient: build_initial_monmap
-211> 2020-06-19T14:49:24.177+0530 7fbad58b7080 10 monclient: monmap:
epoch 0
fsid 10e9ad9d-95b2-471c-9aeb-a7c3b37bc6c1
last_changed 2020-06-19T14:49:24.177553+0530
created 2020-06-19T14:49:24.177553+0530
min_mon_release 0 (unknown)
0: v1:127.0.0.1:40749/0 mon.noname-b-legacy
1: v1:[::1]:40749/0 mon.noname-a-legacy
2: v2:127.0.0.1:40749/0 mon.noname-b
3: v2:[::1]:40749/0 mon.noname-a

-210> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding auth protocol: cephx
-209> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding auth protocol: cephx
-208> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding auth protocol: cephx
-207> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-206> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-205> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-204> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-203> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-202> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-201> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-200> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-199> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-198> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-197> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: crc
-196> 2020-06-19T14:49:24.177+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe41780) adding con mode: secure
-195> 2020-06-19T14:49:24.178+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-194> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 4
-193> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 5
-192> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 6
-191> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 7
-190> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 8
-189> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 9
-188> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 10
-187> 2020-06-19T14:49:24.178+0530 7fbad58b7080 1 lockdep using id 11
-186> 2020-06-19T14:49:24.179+0530 7fbad58b7080 1 lockdep using id 12
-185> 2020-06-19T14:49:24.179+0530 7fbad58b7080 1 lockdep using id 13
-184> 2020-06-19T14:49:24.179+0530 7fbad58b7080 1 lockdep using id 14
-183> 2020-06-19T14:49:24.179+0530 7fbad58b7080 10 monclient: init
-182> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding auth protocol: cephx
-181> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding auth protocol: cephx
-180> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding auth protocol: cephx
-179> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-178> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-177> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-176> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-175> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-174> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-173> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-172> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-171> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-170> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-169> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: crc
-168> 2020-06-19T14:49:24.179+0530 7fbad58b7080 5 AuthRegistry(0x7fff8529abb0) adding con mode: secure
-167> 2020-06-19T14:49:24.181+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-166> 2020-06-19T14:49:24.182+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-165> 2020-06-19T14:49:24.182+0530 7fbad58b7080 1 lockdep using id 15
-164> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient: _reopen_session rank -1
-163> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient: _add_conns ranks=[3,2,0,1]
-162> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-a con 0x55fb8fee5b10 addr v2:[::1]:40749/0
-161> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-b con 0x55fb8fee6380 addr v2:127.0.0.1:40749/0
-160> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-b-legacy con 0x55fb8feed0f0 addr v1:127.0.0.1:40749/0
-159> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): start opening mon connection
-158> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): start opening mon connection
-157> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): _renew_subs
-156> 2020-06-19T14:49:24.182+0530 7fbad58b7080 10 monclient(hunting): authenticate will time out at 2020-06-19T14:54:24.182992+0530
-155> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): get_auth_request con 0x55fb8fee6380 auth_method 0
-154> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-153> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): _init_auth method 2
-152> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): handle_auth_reply_more payload 9
-151> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-150> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-149> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient(hunting): handle_auth_done global_id 4308 payload 450
-148> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient: _finish_hunting 0
-147> 2020-06-19T14:49:24.183+0530 7fbaceffd700 1 monclient: found mon.noname-b
-146> 2020-06-19T14:49:24.183+0530 7fbaceffd700 10 monclient: _send_mon_message to mon.noname-b at v2:127.0.0.1:40749/0
-145> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: handle_monmap mon_map magic: 0 v1
-144> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: got monmap 1 from mon.noname-b (according to old e1)
-143> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: dump:
epoch 1
fsid 10e9ad9d-95b2-471c-9aeb-a7c3b37bc6c1
last_changed 2020-06-19T14:47:12.312173+0530
created 2020-06-19T14:47:12.312173+0530
min_mon_release 16 (pacific)
0: [v2:127.0.0.1:40749/0,v1:127.0.0.1:40750/0] mon.a
1: [v2:127.0.0.1:40751/0,v1:127.0.0.1:40752/0] mon.b
2: [v2:127.0.0.1:40753/0,v1:127.0.0.1:40754/0] mon.c

-142> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: _finish_auth 0
-141> 2020-06-19T14:49:24.184+0530 7fbad58b7080 5 monclient: authenticate success, global_id 4308
-140> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: handle_config config(2 keys) v1
-139> 2020-06-19T14:49:24.184+0530 7fbad58b7080 10 monclient: get_monmap_and_config success
-138> 2020-06-19T14:49:24.184+0530 7fbad58b7080 10 monclient: shutdown
-137> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: handle_monmap mon_map magic: 0 v1
-136> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: got monmap 1 from mon.a (according to old e1)
-135> 2020-06-19T14:49:24.184+0530 7fbacc4f9700 4 set_mon_vals no callback set
-134> 2020-06-19T14:49:24.184+0530 7fbace7fc700 10 monclient: dump:
epoch 1
fsid 10e9ad9d-95b2-471c-9aeb-a7c3b37bc6c1
last_changed 2020-06-19T14:47:12.312173+0530
created 2020-06-19T14:47:12.312173+0530
min_mon_release 16 (pacific)
0: [v2:127.0.0.1:40749/0,v1:127.0.0.1:40750/0] mon.a
1: [v2:127.0.0.1:40751/0,v1:127.0.0.1:40752/0] mon.b
2: [v2:127.0.0.1:40753/0,v1:127.0.0.1:40754/0] mon.c

-133> 2020-06-19T14:49:24.184+0530 7fbacc4f9700 10 set_mon_vals osd_pool_default_min_size = 1
-132> 2020-06-19T14:49:24.184+0530 7fbacc4f9700 10 set_mon_vals osd_pool_default_size = 3
-131> 2020-06-19T14:49:24.185+0530 7fbad58b7080 1 lockdep using id 16
-130> 2020-06-19T14:49:24.185+0530 7fbad58b7080 1 lockdep reusing last freed id 16
-129> 2020-06-19T14:49:24.187+0530 7fbad58b7080 1 lockdep reusing last freed id 1
-128> 2020-06-19T14:49:24.187+0530 7fbad58b7080 1 lockdep using id 2
-127> 2020-06-19T14:49:24.187+0530 7fbad58b7080 -1 WARNING: all dangerous and experimental features are enabled.
-126> 2020-06-19T14:49:24.187+0530 7fbad58b7080 1 lockdep using id 3
-125> 2020-06-19T14:49:24.187+0530 7fbad58b7080 0 ceph version 16.0.0-2623-gfe183904a2c (fe183904a2caece314d8e167b4a20a455535197e) pacific (dev), process ceph-fuse, pid 269594
-124> 2020-06-19T14:49:24.187+0530 7fbad58b7080 1 lockdep using id 4
-123> 2020-06-19T14:49:24.188+0530 7fbad58b7080 0 pidfile_write: ignore empty --pid-file
-122> 2020-06-19T14:49:24.192+0530 7fbad58b7080 1 lockdep reusing last freed id 13
-121> 2020-06-19T14:49:24.192+0530 7fbad58b7080 1 lockdep using id 5
-120> 2020-06-19T14:49:24.193+0530 7fbad58b7080 1 lockdep using id 6
-119> 2020-06-19T14:49:24.193+0530 7fbad58b7080 -1 WARNING: all dangerous and experimental features are enabled.
-118> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) init /tmp/ceph-asok.3kjhqv/client.admin.269603.asok
-117> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) bind_and_listen /tmp/ceph-asok.3kjhqv/client.admin.269603.asok
-116> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command 0 hook 0x55fb8fe45790
-115> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command version hook 0x55fb8fe45790
-114> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command git_version hook 0x55fb8fe45790
-113> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command help hook 0x55fb8fe44d10
-112> 2020-06-19T14:49:24.193+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command get_command_descriptions hook 0x55fb8fe45d90
-111> 2020-06-19T14:49:24.193+0530 7fbad58b7080 1 lockdep using id 7
-110> 2020-06-19T14:49:24.193+0530 7fbacf7fe700 5 asok(0x55fb8fde0160) entry start
-109> 2020-06-19T14:49:24.193+0530 7fbad58b7080 1 lockdep using id 9
-108> 2020-06-19T14:49:24.194+0530 7fbad58b7080 1 lockdep using id 10
-107> 2020-06-19T14:49:24.194+0530 7fbad58b7080 1 lockdep using id 11
-106> 2020-06-19T14:49:24.194+0530 7fbad58b7080 10 monclient: build_initial_monmap
-105> 2020-06-19T14:49:24.194+0530 7fbad58b7080 10 monclient: monmap:
epoch 0
fsid 10e9ad9d-95b2-471c-9aeb-a7c3b37bc6c1
last_changed 2020-06-19T14:49:24.194968+0530
created 2020-06-19T14:49:24.194968+0530
min_mon_release 0 (unknown)
0: v1:127.0.0.1:40749/0 mon.noname-b-legacy
1: v1:[::1]:40749/0 mon.noname-a-legacy
2: v2:127.0.0.1:40749/0 mon.noname-b
3: v2:[::1]:40749/0 mon.noname-a

-104> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding auth protocol: cephx
-103> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding auth protocol: cephx
-102> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding auth protocol: cephx
-101> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-100> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-99> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-98> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-97> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-96> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-95> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-94> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-93> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-92> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-91> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: crc
-90> 2020-06-19T14:49:24.194+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fee2650) adding con mode: secure
-89> 2020-06-19T14:49:24.196+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-88> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 12
-87> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 14
-86> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 15
-85> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 16
-84> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 17
-83> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 18
-82> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 19
-81> 2020-06-19T14:49:24.196+0530 7fbad58b7080 1 lockdep using id 20
-80> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 21
-79> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 22
-78> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 23
-77> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 24
-76> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 25
-75> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 26
-74> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 27
-73> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 28
-72> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 29
-71> 2020-06-19T14:49:24.197+0530 7fbad58b7080 1 lockdep using id 30
-70> 2020-06-19T14:49:24.198+0530 7fbad58b7080 -1 init, newargv = 0x55fb8fee6c80 newargc=9
-69> 2020-06-19T14:49:24.198+0530 7fbad58b7080 1 lockdep using id 31
-68> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command objecter_requests hook 0x55fb8fef4ac0
-67> 2020-06-19T14:49:24.198+0530 7fbad58b7080 10 monclient: init
-66> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding auth protocol: cephx
-65> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding auth protocol: cephx
-64> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding auth protocol: cephx
-63> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-62> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-61> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-60> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-59> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-58> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-57> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-56> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-55> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-54> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-53> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: crc
-52> 2020-06-19T14:49:24.198+0530 7fbad58b7080 5 AuthRegistry(0x55fb8fe57a20) adding con mode: secure
-51> 2020-06-19T14:49:24.200+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-50> 2020-06-19T14:49:24.201+0530 7fbad58b7080 2 auth: KeyRing::load: loaded key file /home/jcollin/workspace/ceph/build/keyring
-49> 2020-06-19T14:49:24.201+0530 7fbad58b7080 1 lockdep using id 32
-48> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient: _renew_subs
-47> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient: _reopen_session rank -1
-46> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient: _add_conns ranks=[3,0,2,1]
-45> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-a con 0x55fb8fee5b10 addr v2:[::1]:40749/0
-44> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-b-legacy con 0x55fb8feed0f0 addr v1:127.0.0.1:40749/0
-43> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): picked mon.noname-b con 0x55fb8fef5f70 addr v2:127.0.0.1:40749/0
-42> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): start opening mon connection
-41> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): start opening mon connection
-40> 2020-06-19T14:49:24.201+0530 7fbad58b7080 10 monclient(hunting): _renew_subs
-39> 2020-06-19T14:49:24.201+0530 7fbac97f2700 1 client.0 ms_handle_refused on v2:[::1]:40749/0
-38> 2020-06-19T14:49:24.201+0530 7fbad58b7080 1 lockdep using id 33
-37> 2020-06-19T14:49:24.202+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command mds_requests hook 0x55fb8feebae8
-36> 2020-06-19T14:49:24.202+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command mds_sessions hook 0x55fb8feebae8
-35> 2020-06-19T14:49:24.202+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command dump_cache hook 0x55fb8feebae8
-34> 2020-06-19T14:49:24.202+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command kick_stale_sessions hook 0x55fb8feebae8
-33> 2020-06-19T14:49:24.202+0530 7fbad58b7080 5 asok(0x55fb8fde0160) register_command status hook 0x55fb8feebae8
-32> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): get_auth_request con 0x55fb8fef5f70 auth_method 0
-31> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-30> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): _init_auth method 2
-29> 2020-06-19T14:49:24.202+0530 7fbad58b7080 10 monclient(hunting): authenticate will time out at 2020-06-19T14:54:24.202411+0530
-28> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): handle_auth_reply_more payload 9
-27> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-26> 2020-06-19T14:49:24.202+0530 7fbacd7fa700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-25> 2020-06-19T14:49:24.203+0530 7fbacd7fa700 10 monclient(hunting): handle_auth_done global_id 4311 payload 824
-24> 2020-06-19T14:49:24.203+0530 7fbacd7fa700 10 monclient: _finish_hunting 0
-23> 2020-06-19T14:49:24.203+0530 7fbacd7fa700 1 monclient: found mon.noname-b
-22> 2020-06-19T14:49:24.203+0530 7fbacd7fa700 10 monclient: _send_mon_message to mon.noname-b at v2:127.0.0.1:40749/0
-21> 2020-06-19T14:49:24.203+0530 7fbac97f2700 10 monclient: handle_monmap mon_map magic: 0 v1
-20> 2020-06-19T14:49:24.204+0530 7fbac97f2700 10 monclient: got monmap 1 from mon.noname-b (according to old e1)
-19> 2020-06-19T14:49:24.204+0530 7fbac97f2700 10 monclient: dump:
epoch 1
fsid 10e9ad9d-95b2-471c-9aeb-a7c3b37bc6c1
last_changed 2020-06-19T14:47:12.312173+0530
created 2020-06-19T14:47:12.312173+0530
min_mon_release 16 (pacific)
0: [v2:127.0.0.1:40749/0,v1:127.0.0.1:40750/0] mon.a
1: [v2:127.0.0.1:40751/0,v1:127.0.0.1:40752/0] mon.b
2: [v2:127.0.0.1:40753/0,v1:127.0.0.1:40754/0] mon.c

-18> 2020-06-19T14:49:24.204+0530 7fbac97f2700 10 monclient: _finish_auth 0
-17> 2020-06-19T14:49:24.204+0530 7fbad58b7080 5 monclient: authenticate success, global_id 4311
-16> 2020-06-19T14:49:24.204+0530 7fbad58b7080 10 monclient: _renew_subs
-15> 2020-06-19T14:49:24.204+0530 7fbad58b7080 10 monclient: _send_mon_message to mon.a at v2:127.0.0.1:40749/0
-14> 2020-06-19T14:49:24.204+0530 7fbac97f2700 10 monclient: handle_config config(2 keys) v1
-13> 2020-06-19T14:49:24.204+0530 7fbac97f2700 1 client.4311 handle_mds_map epoch 6
-12> 2020-06-19T14:49:24.204+0530 7fbacdffb700 4 set_mon_vals no callback set
-11> 2020-06-19T14:49:24.205+0530 7fbaccff9700 10 monclient: get_auth_request con 0x55fb8ff050c0 auth_method 0
-10> 2020-06-19T14:49:24.224+0530 7fba997fa700 1 client.4311 using remount_cb
-9> 2020-06-19T14:49:24.225+0530 7fba7bfff700 3 client.4311 ll_lookup 0x1.head .Trash
-8> 2020-06-19T14:49:24.225+0530 7fba7bfff700 3 client.4311 may_lookup 0x7fbaa0005a10 = 0
-7> 2020-06-19T14:49:24.225+0530 7fba7bfff700 3 client.4311 ll_lookup 0x1.head .Trash -> -2 (0)
-6> 2020-06-19T14:49:24.225+0530 7fba98ff9700 3 client.4311 ll_lookup 0x1.head .Trash-1000
-5> 2020-06-19T14:49:24.225+0530 7fba98ff9700 3 client.4311 may_lookup 0x7fbaa0005a10 = 0
-4> 2020-06-19T14:49:24.225+0530 7fba98ff9700 3 client.4311 ll_lookup 0x1.head .Trash-1000 -> -2 (0)
-3> 2020-06-19T14:49:24.230+0530 7fba997fa700 -1 client.4311 failed to remount (to trim kernel dentries): return code = 1
-2> 2020-06-19T14:49:24.230+0530 7fba997fa700 -1 client.4311 failed to remount for kernel dentry trimming; quitting!
-1> 2020-06-19T14:49:24.234+0530 7fba997fa700 -1 /home/jcollin/workspace/ceph/src/client/Client.cc: In function 'int Client::_do_remount(bool)' thread 7fba997fa700 time 2020-06-19T14:49:24.230486+0530
/home/jcollin/workspace/ceph/src/client/Client.cc: 4245: ceph_abort_msg("abort() called")

ceph version 16.0.0-2623-gfe183904a2c (fe183904a2caece314d8e167b4a20a455535197e) pacific (dev)
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xf6) [0x7fbad790b646]
2: (Client::_do_remount(bool)+0x564) [0x55fb8e890fd2]
3: (Client::test_dentry_handling(bool)+0x28f) [0x55fb8e8c8803]
4: (()+0x289877) [0x55fb8e83a877]
5: (Thread::entry_wrapper()+0x78) [0x7fbad78a6f1e]
6: (Thread::_entry_func(void*)+0x18) [0x7fbad78a6e9c]
7: (()+0x94e2) [0x7fbad5f164e2]
8: (clone()+0x43) [0x7fbad5a6a643]

0> 2020-06-19T14:49:24.238+0530 7fba997fa700 -1 *** Caught signal (Aborted) **
in thread 7fba997fa700 thread_name:tester

ceph version 16.0.0-2623-gfe183904a2c (fe183904a2caece314d8e167b4a20a455535197e) pacific (dev)
1: (()+0x414512) [0x55fb8e9c5512]
2: (()+0x14b20) [0x7fbad5f21b20]
3: (gsignal()+0x145) [0x7fbad59a5625]
4: (abort()+0x12b) [0x7fbad598e8d9]
5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x37a) [0x7fbad790b8ca]
6: (Client::_do_remount(bool)+0x564) [0x55fb8e890fd2]
7: (Client::test_dentry_handling(bool)+0x28f) [0x55fb8e8c8803]
8: (()+0x289877) [0x55fb8e83a877]
9: (Thread::entry_wrapper()+0x78) [0x7fbad78a6f1e]
10: (Thread::_entry_func(void*)+0x18) [0x7fbad78a6e9c]
11: (()+0x94e2) [0x7fbad5f164e2]
12: (clone()+0x43) [0x7fbad5a6a643]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 rbd_rwl
0/ 5 journaler
0/ 5 objectcacher
0/ 5 immutable_obj_cache
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
0/ 5 test
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
--- pthread ID / name mapping for recent threads ---
7fba7bfff700 / ceph-fuse
7fba98ff9700 / ceph-fuse
7fba997fa700 / tester
7fbac97f2700 / ms_dispatch
7fbacc4f9700 /
7fbaccff9700 / msgr-worker-1
7fbacd7fa700 / msgr-worker-0
7fbacdffb700 / io_context_pool
7fbace7fc700 / io_context_pool
7fbaceffd700 / service
7fbacf7fe700 / admin_socket
7fbad58b7080 / ceph-fuse
max_recent 10000
max_new 1000
log_file /home/jcollin/workspace/ceph/build/out/client.admin.269603.log
--- end dump of recent events ---
    (1-1/1)