inferring bluefs devices from bluestore path 2020-10-28T02:07:55.499+0000 7f0aa7f1e0c0 -1 bluefs _check_new_allocations invalid extent 1: 0x7ae18a0000~10000: duplicate reference, ino 26 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluefs mount failed to replay log: (14) Bad address 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _open_bluefs failed bluefs mount: (14) Bad address /root/project/src/ceph/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::expand_devices(std::ostream&)' thread 7f0aa7f1e0c0 time 2020-10-28T02:07:55.750217+0000 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: 6837: FAILED ceph_assert(r == 0) ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x151) [0x7f0a9e3ce731] 2: (()+0x26591b) [0x7f0a9e3ce91b] 3: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 4: (main()+0x214d) [0x55a4702739bd] 5: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 6: (_start()+0x2a) [0x55a470279c2a] *** Caught signal (Aborted) ** in thread 7f0aa7f1e0c0 thread_name:ceph-bluestore- 2020-10-28T02:07:55.751+0000 7f0aa7f1e0c0 -1 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::expand_devices(std::ostream&)' thread 7f0aa7f1e0c0 time 2020-10-28T02:07:55.750217+0000 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: 6837: FAILED ceph_assert(r == 0) ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x151) [0x7f0a9e3ce731] 2: (()+0x26591b) [0x7f0a9e3ce91b] 3: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 4: (main()+0x214d) [0x55a4702739bd] 5: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 6: (_start()+0x2a) [0x55a470279c2a] ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (()+0x128a0) [0x7f0a9da918a0] 2: (gsignal()+0xc7) [0x7f0a9c936f47] 3: (abort()+0x141) [0x7f0a9c9388b1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ac) [0x7f0a9e3ce78c] 5: (()+0x26591b) [0x7f0a9e3ce91b] 6: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 7: (main()+0x214d) [0x55a4702739bd] 8: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 9: (_start()+0x2a) [0x55a470279c2a] 2020-10-28T02:07:55.753+0000 7f0aa7f1e0c0 -1 *** Caught signal (Aborted) ** in thread 7f0aa7f1e0c0 thread_name:ceph-bluestore- ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (()+0x128a0) [0x7f0a9da918a0] 2: (gsignal()+0xc7) [0x7f0a9c936f47] 3: (abort()+0x141) [0x7f0a9c9388b1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ac) [0x7f0a9e3ce78c] 5: (()+0x26591b) [0x7f0a9e3ce91b] 6: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 7: (main()+0x214d) [0x55a4702739bd] 8: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 9: (_start()+0x2a) [0x55a470279c2a] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. -47> 2020-10-28T02:07:55.499+0000 7f0aa7f1e0c0 -1 bluefs _check_new_allocations invalid extent 1: 0x7ae18a0000~10000: duplicate reference, ino 26 -46> 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluefs mount failed to replay log: (14) Bad address -45> 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _open_bluefs failed bluefs mount: (14) Bad address -44> 2020-10-28T02:07:55.751+0000 7f0aa7f1e0c0 -1 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::expand_devices(std::ostream&)' thread 7f0aa7f1e0c0 time 2020-10-28T02:07:55.750217+0000 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: 6837: FAILED ceph_assert(r == 0) ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x151) [0x7f0a9e3ce731] 2: (()+0x26591b) [0x7f0a9e3ce91b] 3: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 4: (main()+0x214d) [0x55a4702739bd] 5: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 6: (_start()+0x2a) [0x55a470279c2a] -43> 2020-10-28T02:07:55.753+0000 7f0aa7f1e0c0 -1 *** Caught signal (Aborted) ** in thread 7f0aa7f1e0c0 thread_name:ceph-bluestore- ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (()+0x128a0) [0x7f0a9da918a0] 2: (gsignal()+0xc7) [0x7f0a9c936f47] 3: (abort()+0x141) [0x7f0a9c9388b1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ac) [0x7f0a9e3ce78c] 5: (()+0x26591b) [0x7f0a9e3ce91b] 6: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 7: (main()+0x214d) [0x55a4702739bd] 8: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 9: (_start()+0x2a) [0x55a470279c2a] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. -8> 2020-10-28T02:07:55.499+0000 7f0aa7f1e0c0 -1 bluefs _check_new_allocations invalid extent 1: 0x7ae18a0000~10000: duplicate reference, ino 26 -7> 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluefs mount failed to replay log: (14) Bad address -3> 2020-10-28T02:07:55.500+0000 7f0aa7f1e0c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _open_bluefs failed bluefs mount: (14) Bad address -1> 2020-10-28T02:07:55.751+0000 7f0aa7f1e0c0 -1 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::expand_devices(std::ostream&)' thread 7f0aa7f1e0c0 time 2020-10-28T02:07:55.750217+0000 /root/project/src/ceph/src/os/bluestore/BlueStore.cc: 6837: FAILED ceph_assert(r == 0) ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x151) [0x7f0a9e3ce731] 2: (()+0x26591b) [0x7f0a9e3ce91b] 3: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 4: (main()+0x214d) [0x55a4702739bd] 5: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 6: (_start()+0x2a) [0x55a470279c2a] 0> 2020-10-28T02:07:55.753+0000 7f0aa7f1e0c0 -1 *** Caught signal (Aborted) ** in thread 7f0aa7f1e0c0 thread_name:ceph-bluestore- ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable) 1: (()+0x128a0) [0x7f0a9da918a0] 2: (gsignal()+0xc7) [0x7f0a9c936f47] 3: (abort()+0x141) [0x7f0a9c9388b1] 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ac) [0x7f0a9e3ce78c] 5: (()+0x26591b) [0x7f0a9e3ce91b] 6: (BlueStore::expand_devices(std::ostream&)+0xb98) [0x55a47036aef8] 7: (main()+0x214d) [0x55a4702739bd] 8: (__libc_start_main()+0xe7) [0x7f0a9c919b97] 9: (_start()+0x2a) [0x55a470279c2a] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. reraise_fatal: default handler for signal 6 didn't terminate the process?