2019-02-15 10:58:08.272 7f3bba04dec0 1 Processor -- start 2019-02-15 10:58:08.272 7f3bba04dec0 1 -- start start 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=NONE pgs=0 cs=0 l=0).connect 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=NONE pgs=0 cs=0 l=0).connect 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=NONE pgs=0 cs=0 l=0).connect 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] -- mon_getmap magic: 0 v1 -- 0x35da540 con 0x420cc00 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] -- mon_getmap magic: 0 v1 -- 0x35da700 con 0x3615c00 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] -- mon_getmap magic: 0 v1 -- 0x35da8c0 con 0x420c400 2019-02-15 10:58:08.274 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 2019-02-15 10:58:08.274 7f3ba552c700 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 2019-02-15 10:58:08.274 7f3ba652e700 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 msgr2=0x41dd200 :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CONNECTING pgs=0 cs=0 l=0).stop 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 msgr2=0x41dd680 :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CONNECTING pgs=0 cs=0 l=0).stop 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- --> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x35dc780 con 0x420c400 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CONNECTING pgs=0 cs=0 l=0).send_client_ident getsockname reveals I am 127.0.0.1:38680 when talking to v2:127.0.0.1:40623/0 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- 127.0.0.1:0/351556750 learned_addr learned my addr 127.0.0.1:0/351556750 (peer_addr_for_me v2:127.0.0.1:0/0) 2019-02-15 10:58:08.276 7f3ba5d2d700 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=READY pgs=34 cs=0 l=1).ready entity=mon.2 cookie=0 in_seq=0 out_seq=0 2019-02-15 10:58:08.277 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 1 ==== mon_map magic: 0 v1 ==== 376+0+0 (3611191555 0 0) 0x42e4200 con 0x420c400 2019-02-15 10:58:08.277 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 2 ==== config(3 keys) v1 ==== 100+0+0 (40072250 0 0) 0x35dcb40 con 0x420c400 2019-02-15 10:58:08.278 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 3 ==== mon_map magic: 0 v1 ==== 376+0+0 (3611191555 0 0) 0x42e4a00 con 0x420c400 2019-02-15 10:58:08.278 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 msgr2=0x41dcd80 :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2019-02-15 10:58:08.278 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=READY pgs=34 cs=0 l=1).stop 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 shutdown_connections 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CLOSED pgs=0 cs=0 l=0).stop 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CLOSED pgs=0 cs=0 l=0).stop 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CLOSED pgs=34 cs=0 l=1).stop 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 shutdown_connections 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 wait complete. 2019-02-15 10:58:08.289 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 >> 127.0.0.1:0/351556750 conn(0x3615400 msgr2=0x41fcb00 :-1 s=STATE_NONE l=0).mark_down 2019-02-15 10:58:08.289 7f3bba04dec0 0 framework: civetweb 2019-02-15 10:58:08.289 7f3bba04dec0 0 framework conf key: port, val: 8000 2019-02-15 10:58:08.290 7f3bba04dec0 0 ceph version Development (no_version) nautilus (dev), process radosgw, pid 21605 2019-02-15 10:58:08.324 7f3bba04dec0 0 ext_mime_map_init failed to open file=/etc/mime.types : (2) No such file or directory 2019-02-15 10:58:08.324 7f3bba04dec0 -1 ERROR: unable to initialize rgw tools 2019-02-15 10:58:08.330 7f3bba04dec0 -1 /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/common/Timer.cc: In function 'virtual SafeTimer::~SafeTimer()' thread 7f3bba04dec0 time 2019-02-15 10:58:08.325416 /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/common/Timer.cc: 50: FAILED ceph_assert(thread == __null) ceph version Development (no_version) nautilus (dev) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1aa) [0x7f3baf50483e] 2: (()+0x126eac0) [0x7f3baf504ac0] 3: (SafeTimer::~SafeTimer()+0x37) [0x7f3baf4b1f6d] 4: (main()+0x39e6) [0xf181d0] 5: (__libc_start_main()+0xf5) [0x7f3baacf23d5] 6: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0xf135a7] 2019-02-15 10:58:08.336 7f3bba04dec0 -1 *** Caught signal (Aborted) ** in thread 7f3bba04dec0 thread_name:radosgw ceph version Development (no_version) nautilus (dev) 1: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0x108fccc] 2: (()+0xf5d0) [0x7f3bade855d0] 3: (gsignal()+0x37) [0x7f3baad06207] 4: (abort()+0x148) [0x7f3baad078f8] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x379) [0x7f3baf504a0d] 6: (()+0x126eac0) [0x7f3baf504ac0] 7: (SafeTimer::~SafeTimer()+0x37) [0x7f3baf4b1f6d] 8: (main()+0x39e6) [0xf181d0] 9: (__libc_start_main()+0xf5) [0x7f3baacf23d5] 10: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0xf135a7] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -173> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command assert hook 0x35c04c0 -172> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command abort hook 0x35c04c0 -171> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perfcounters_dump hook 0x35c04c0 -170> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command 1 hook 0x35c04c0 -169> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perf dump hook 0x35c04c0 -168> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perfcounters_schema hook 0x35c04c0 -167> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perf histogram dump hook 0x35c04c0 -166> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command 2 hook 0x35c04c0 -165> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perf schema hook 0x35c04c0 -164> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perf histogram schema hook 0x35c04c0 -163> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command perf reset hook 0x35c04c0 -162> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config show hook 0x35c04c0 -161> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config help hook 0x35c04c0 -160> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config set hook 0x35c04c0 -159> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config unset hook 0x35c04c0 -158> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config get hook 0x35c04c0 -157> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config diff hook 0x35c04c0 -156> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command config diff get hook 0x35c04c0 -155> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command log flush hook 0x35c04c0 -154> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command log dump hook 0x35c04c0 -153> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command log reopen hook 0x35c04c0 -152> 2019-02-15 10:58:08.220 7f3bba04dec0 5 asok(0x364c000) register_command dump_mempools hook 0x3602548 -151> 2019-02-15 10:58:08.242 7f3bba04dec0 1 lockdep start -150> 2019-02-15 10:58:08.242 7f3bba04dec0 1 lockdep using id 0 -149> 2019-02-15 10:58:08.242 7f3bba04dec0 1 lockdep using id 1 -148> 2019-02-15 10:58:08.242 7f3bba04dec0 1 lockdep using id 2 -147> 2019-02-15 10:58:08.242 7f3bba04dec0 1 lockdep using id 3 -146> 2019-02-15 10:58:08.242 7f3bba04dec0 10 monclient: get_monmap_and_config -145> 2019-02-15 10:58:08.270 7f3bba04dec0 10 monclient: build_initial_monmap -144> 2019-02-15 10:58:08.270 7f3bba04dec0 10 monclient: monmap: epoch 0 fsid 24dd9774-9454-4dd7-a74d-3de8208aa5c6 last_changed 2019-02-15 10:58:08.271274 created 2019-02-15 10:58:08.271274 0: [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] mon.noname-a 1: [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] mon.noname-b 2: [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] mon.noname-c -143> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding auth protocol: cephx -142> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding auth protocol: cephx -141> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding auth protocol: cephx -140> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: crc -139> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: secure -138> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: crc -137> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: secure -136> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: crc -135> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: secure -134> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: crc -133> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: secure -132> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: crc -131> 2019-02-15 10:58:08.270 7f3bba04dec0 5 AuthRegistry(0x41e0a30) adding con mode: secure -130> 2019-02-15 10:58:08.271 7f3bba04dec0 2 auth: KeyRing::load: loaded key file /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/keyring -129> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 4 -128> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 5 -127> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 6 -126> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 7 -125> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 8 -124> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 9 -123> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 10 -122> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 11 -121> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 12 -120> 2019-02-15 10:58:08.271 7f3bba04dec0 1 lockdep using id 13 -119> 2019-02-15 10:58:08.272 7f3bba04dec0 1 lockdep using id 14 -118> 2019-02-15 10:58:08.272 7f3bba04dec0 1 Processor -- start -117> 2019-02-15 10:58:08.272 7f3bba04dec0 1 -- start start -116> 2019-02-15 10:58:08.272 7f3bba04dec0 10 monclient: init -115> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding auth protocol: cephx -114> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding auth protocol: cephx -113> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding auth protocol: cephx -112> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: crc -111> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: secure -110> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: crc -109> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: secure -108> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: crc -107> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: secure -106> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: crc -105> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: secure -104> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: crc -103> 2019-02-15 10:58:08.272 7f3bba04dec0 5 AuthRegistry(0x7fff22b345c8) adding con mode: secure -102> 2019-02-15 10:58:08.273 7f3bba04dec0 2 auth: KeyRing::load: loaded key file /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/keyring -101> 2019-02-15 10:58:08.273 7f3bba04dec0 2 auth: KeyRing::load: loaded key file /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/keyring -100> 2019-02-15 10:58:08.273 7f3bba04dec0 1 lockdep using id 15 -99> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient: _reopen_session rank -1 -98> 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=NONE pgs=0 cs=0 l=0).connect -97> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): picked mon.noname-c con 0x420c400 addr [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] -96> 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=NONE pgs=0 cs=0 l=0).connect -95> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): picked mon.noname-b con 0x3615c00 addr [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] -94> 2019-02-15 10:58:08.273 7f3bba04dec0 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=NONE pgs=0 cs=0 l=0).connect -93> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): picked mon.noname-a con 0x420cc00 addr [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] -92> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): start opening mon connection -91> 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] -- mon_getmap magic: 0 v1 -- 0x35da540 con 0x420cc00 -90> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): start opening mon connection -89> 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] -- mon_getmap magic: 0 v1 -- 0x35da700 con 0x3615c00 -88> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): start opening mon connection -87> 2019-02-15 10:58:08.273 7f3bba04dec0 1 -- --> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] -- mon_getmap magic: 0 v1 -- 0x35da8c0 con 0x420c400 -86> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): _renew_subs -85> 2019-02-15 10:58:08.273 7f3bba04dec0 10 monclient(hunting): authenticate will time out at 2019-02-15 11:03:08.274586 -84> 2019-02-15 10:58:08.274 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 -83> 2019-02-15 10:58:08.274 7f3ba552c700 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 -82> 2019-02-15 10:58:08.274 7f3ba652e700 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CONNECTING pgs=0 cs=0 l=0)._handle_peer_banner_payload supported=0 required=0 -81> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): get_auth_request con 0x420c400 auth_method 0 -80> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -79> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): _init_auth method 2 -78> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): get_auth_request con 0x420cc00 auth_method 0 -77> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -76> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): _init_auth method 2 -75> 2019-02-15 10:58:08.274 7f3ba552c700 10 monclient(hunting): get_auth_request con 0x3615c00 auth_method 0 -74> 2019-02-15 10:58:08.274 7f3ba552c700 10 monclient(hunting): get_auth_request method 2 preferred_modes [1,2] -73> 2019-02-15 10:58:08.274 7f3ba552c700 10 monclient(hunting): _init_auth method 2 -72> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): handle_auth_reply_more payload 9 -71> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): handle_auth_reply_more payload_len 9 -70> 2019-02-15 10:58:08.274 7f3ba5d2d700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes -69> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): handle_auth_reply_more payload 9 -68> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): handle_auth_reply_more payload_len 9 -67> 2019-02-15 10:58:08.274 7f3ba652e700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes -66> 2019-02-15 10:58:08.275 7f3ba552c700 10 monclient(hunting): handle_auth_reply_more payload 9 -65> 2019-02-15 10:58:08.275 7f3ba552c700 10 monclient(hunting): handle_auth_reply_more payload_len 9 -64> 2019-02-15 10:58:08.275 7f3ba552c700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes -63> 2019-02-15 10:58:08.275 7f3ba5d2d700 10 monclient(hunting): handle_auth_done global_id 4370 payload 386 -62> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 msgr2=0x41dd200 :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down -61> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CONNECTING pgs=0 cs=0 l=0).stop -60> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 msgr2=0x41dd680 :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down -59> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CONNECTING pgs=0 cs=0 l=0).stop -58> 2019-02-15 10:58:08.275 7f3ba5d2d700 10 monclient: _finish_hunting 0 -57> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 monclient: found mon.noname-c -56> 2019-02-15 10:58:08.275 7f3ba5d2d700 10 monclient: _send_mon_message to mon.noname-c at v2:127.0.0.1:40623/0 -55> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- --> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x35dc780 con 0x420c400 -54> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 --2- >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CONNECTING pgs=0 cs=0 l=0).send_client_ident getsockname reveals I am 127.0.0.1:38680 when talking to v2:127.0.0.1:40623/0 -53> 2019-02-15 10:58:08.275 7f3ba5d2d700 1 -- 127.0.0.1:0/351556750 learned_addr learned my addr 127.0.0.1:0/351556750 (peer_addr_for_me v2:127.0.0.1:0/0) -52> 2019-02-15 10:58:08.276 7f3ba5d2d700 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=READY pgs=34 cs=0 l=1).ready entity=mon.2 cookie=0 in_seq=0 out_seq=0 -51> 2019-02-15 10:58:08.277 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 1 ==== mon_map magic: 0 v1 ==== 376+0+0 (3611191555 0 0) 0x42e4200 con 0x420c400 -50> 2019-02-15 10:58:08.277 7f3ba4d2b700 10 monclient: handle_monmap mon_map magic: 0 v1 -49> 2019-02-15 10:58:08.277 7f3ba4d2b700 10 monclient: got monmap 1 from mon.noname-c (according to old e1) -48> 2019-02-15 10:58:08.277 7f3ba4d2b700 10 monclient: dump: epoch 1 fsid 24dd9774-9454-4dd7-a74d-3de8208aa5c6 last_changed 2019-02-15 10:56:50.021997 created 2019-02-15 10:56:50.021997 0: [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] mon.a 1: [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] mon.b 2: [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] mon.c -47> 2019-02-15 10:58:08.277 7f3ba4d2b700 10 monclient: _finish_auth 0 -46> 2019-02-15 10:58:08.277 7f3bba04dec0 5 monclient: authenticate success, global_id 4370 -45> 2019-02-15 10:58:08.277 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 2 ==== config(3 keys) v1 ==== 100+0+0 (40072250 0 0) 0x35dcb40 con 0x420c400 -44> 2019-02-15 10:58:08.278 7f3ba4d2b700 10 monclient: handle_config config(3 keys) v1 -43> 2019-02-15 10:58:08.278 7f3bba04dec0 10 monclient: get_monmap_and_config success -42> 2019-02-15 10:58:08.278 7f3bba04dec0 10 monclient: shutdown -41> 2019-02-15 10:58:08.278 7f3ba4d2b700 1 -- 127.0.0.1:0/351556750 <== mon.2 v2:127.0.0.1:40623/0 3 ==== mon_map magic: 0 v1 ==== 376+0+0 (3611191555 0 0) 0x42e4a00 con 0x420c400 -40> 2019-02-15 10:58:08.278 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 msgr2=0x41dcd80 :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down -39> 2019-02-15 10:58:08.278 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=READY pgs=34 cs=0 l=1).stop -38> 2019-02-15 10:58:08.278 7f3ba3528700 4 set_mon_vals no callback set -37> 2019-02-15 10:58:08.278 7f3ba4d2b700 10 monclient: discarding stray monitor message mon_map magic: 0 v1 -36> 2019-02-15 10:58:08.278 7f3ba3528700 10 set_mon_vals mon_pg_warn_min_per_osd = 3 -35> 2019-02-15 10:58:08.278 7f3ba3528700 10 set_mon_vals osd_pool_default_min_size = 1 -34> 2019-02-15 10:58:08.278 7f3ba3528700 10 set_mon_vals osd_pool_default_size = 3 -33> 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 shutdown_connections -32> 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40619/0,v1:127.0.0.1:40620/0] conn(0x420cc00 0x41dd680 :-1 s=CLOSED pgs=0 cs=0 l=0).stop -31> 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40621/0,v1:127.0.0.1:40622/0] conn(0x3615c00 0x41dd200 :-1 s=CLOSED pgs=0 cs=0 l=0).stop -30> 2019-02-15 10:58:08.288 7f3bba04dec0 1 --2- 127.0.0.1:0/351556750 >> [v2:127.0.0.1:40623/0,v1:127.0.0.1:40624/0] conn(0x420c400 0x41dcd80 :-1 s=CLOSED pgs=34 cs=0 l=1).stop -29> 2019-02-15 10:58:08.288 7f3bba04dec0 1 lockdep using id 16 -28> 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 shutdown_connections -27> 2019-02-15 10:58:08.288 7f3bba04dec0 1 lockdep reusing last freed id 16 -26> 2019-02-15 10:58:08.288 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 wait complete. -25> 2019-02-15 10:58:08.289 7f3bba04dec0 1 -- 127.0.0.1:0/351556750 >> 127.0.0.1:0/351556750 conn(0x3615400 msgr2=0x41fcb00 :-1 s=STATE_NONE l=0).mark_down -24> 2019-02-15 10:58:08.289 7f3bba04dec0 0 framework: civetweb -23> 2019-02-15 10:58:08.289 7f3bba04dec0 0 framework conf key: port, val: 8000 -22> 2019-02-15 10:58:08.290 7f3bba04dec0 1 lockdep reusing last freed id 1 -21> 2019-02-15 10:58:08.290 7f3bba04dec0 1 lockdep using id 2 -20> 2019-02-15 10:58:08.290 7f3bba04dec0 1 lockdep using id 3 -19> 2019-02-15 10:58:08.290 7f3bba04dec0 0 ceph version Development (no_version) nautilus (dev), process radosgw, pid 21605 -18> 2019-02-15 10:58:08.292 7f3bba04dec0 1 finished global_init_daemonize -17> 2019-02-15 10:58:08.292 7f3bba04dec0 1 lockdep reusing last freed id 13 -16> 2019-02-15 10:58:08.323 7f3bba04dec0 1 lockdep using id 4 -15> 2019-02-15 10:58:08.323 7f3bba04dec0 1 lockdep using id 5 -14> 2019-02-15 10:58:08.323 7f3bba04dec0 1 lockdep using id 6 -13> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) init /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/out/radosgw.8000.asok -12> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) bind_and_listen /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/out/radosgw.8000.asok -11> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) register_command 0 hook 0x35c5018 -10> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) register_command version hook 0x35c5018 -9> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) register_command git_version hook 0x35c5018 -8> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) register_command help hook 0x35c0090 -7> 2019-02-15 10:58:08.324 7f3bba04dec0 5 asok(0x364c000) register_command get_command_descriptions hook 0x35c0240 -6> 2019-02-15 10:58:08.324 7f3bba04dec0 1 lockdep using id 7 -5> 2019-02-15 10:58:08.324 7f3ba652e700 5 asok(0x364c000) entry start -4> 2019-02-15 10:58:08.324 7f3bba04dec0 1 lockdep using id 9 -3> 2019-02-15 10:58:08.324 7f3bba04dec0 0 ext_mime_map_init failed to open file=/etc/mime.types : (2) No such file or directory -2> 2019-02-15 10:58:08.324 7f3bba04dec0 -1 ERROR: unable to initialize rgw tools -1> 2019-02-15 10:58:08.330 7f3bba04dec0 -1 /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/common/Timer.cc: In function 'virtual SafeTimer::~SafeTimer()' thread 7f3bba04dec0 time 2019-02-15 10:58:08.325416 /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/src/common/Timer.cc: 50: FAILED ceph_assert(thread == __null) ceph version Development (no_version) nautilus (dev) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1aa) [0x7f3baf50483e] 2: (()+0x126eac0) [0x7f3baf504ac0] 3: (SafeTimer::~SafeTimer()+0x37) [0x7f3baf4b1f6d] 4: (main()+0x39e6) [0xf181d0] 5: (__libc_start_main()+0xf5) [0x7f3baacf23d5] 6: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0xf135a7] 0> 2019-02-15 10:58:08.336 7f3bba04dec0 -1 *** Caught signal (Aborted) ** in thread 7f3bba04dec0 thread_name:radosgw ceph version Development (no_version) nautilus (dev) 1: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0x108fccc] 2: (()+0xf5d0) [0x7f3bade855d0] 3: (gsignal()+0x37) [0x7f3baad06207] 4: (abort()+0x148) [0x7f3baad078f8] 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x379) [0x7f3baf504a0d] 6: (()+0x126eac0) [0x7f3baf504ac0] 7: (SafeTimer::~SafeTimer()+0x37) [0x7f3baf4b1f6d] 8: (main()+0x39e6) [0xf181d0] 9: (__libc_start_main()+0xf5) [0x7f3baacf23d5] 10: /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/bin/radosgw() [0xf135a7] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 1/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 journal 1/ 1 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 1 reserver 1/ 5 heartbeatmap 1/ 5 perfcounter 20/20 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /home/jenkins-build/build/workspace/ceph-dashboard-pull-requests/build/out/radosgw.8000.log --- end dump of recent events ---