Project

General

Profile

Bug #48849 » crash_stacktraces.log

Christian Rohmann, 01/19/2021 10:11 AM

 
@timestamp,system.syslog.message
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7fdcbca25700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7fb42c32b700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7f3730c42700 -1 received signal: Hangup from (PID: 566508) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7f4f7af07700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7f3730c42700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7f332b9e0700 -1 received signal: Hangup from (PID: 566508) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7fb42c32b700 -1 received signal: Hangup from (PID: 566508) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7fbae8fde700 -1 Fail to open '/proc/566508/cmdline' error = (2) No such file or directory
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7fbae8fde700 -1 received signal: Hangup from <unknown> (PID: 566508) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7fbae8fde700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.839 7f332b9e0700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 566507) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7fdcbca25700 -1 received signal: Hangup from (PID: 566508) UID: 0
2021-01-05T06:25:05.000Z,2021-01-05 06:25:05.875 7f4f7af07700 -1 received signal: Hangup from (PID: 566508) UID: 0
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'._fastinfo' Value size = 186)
2021-01-05T08:35:16.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 5: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z, in thread 7fb4240b9700 thread_name:bstore_kv_sync
2021-01-05T08:35:16.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z, 5: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x5564aa454fba]"
2021-01-05T08:35:16.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, 6: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z, in thread 7fb4240b9700 thread_name:bstore_kv_sync
2021-01-05T08:35:16.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 8: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z, 9: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x5564aa45500b]"
2021-01-05T08:35:16.000Z, 8: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z, 9: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'.0001231346.00000000000462740887' Value size = 184)
2021-01-05T08:35:16.000Z, -1> 2021-01-05 08:35:16.041 7fb4240b9700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fb4240b9700 time 2021-01-05 08:35:16.020769
2021-01-05T08:35:16.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 2: (gsignal()+0xc7) [0x7fb4324b6fb7]
2021-01-05T08:35:16.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 8: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'._fastinfo' Value size = 186)
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z, -1> 2021-01-05 08:35:16.041 7fb4240b9700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fb4240b9700 time 2021-01-05 08:35:16.020769
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff6f00370000'x' Value size = 575)
2021-01-05T08:35:16.000Z, 6: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T08:35:16.000Z," -2> 2021-01-05 08:35:16.017 7fb4240b9700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 342500554, got 1411691565 in db/096069.sst offset 32090823 size 521815 code = 2 Rocksdb transaction:"
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff'o' Value size = 521)
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z,"2021-01-05 08:35:16.017 7fb4240b9700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 342500554, got 1411691565 in db/096069.sst offset 32090823 size 521815 code = 2 Rocksdb transaction:"
2021-01-05T08:35:16.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x5564aa454fba]"
2021-01-05T08:35:16.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z,2021-01-05 08:35:16.041 7fb4240b9700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fb4240b9700 time 2021-01-05 08:35:16.020769
2021-01-05T08:35:16.000Z, 1: (()+0x12980) [0x7fb433807980]
2021-01-05T08:35:16.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z, 1: (()+0x12980) [0x7fb433807980]
2021-01-05T08:35:16.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x5564aa45500b]"
2021-01-05T08:35:16.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z, 8: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z, in thread 7fb4240b9700 thread_name:bstore_kv_sync
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x5564aa454fba]"
2021-01-05T08:35:16.000Z, 6: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z, in thread 7fb4240b9700 thread_name:bstore_kv_sync
2021-01-05T08:35:16.000Z, 1: (()+0x12980) [0x7fb433807980]
2021-01-05T08:35:16.000Z, 3: (abort()+0x141) [0x7fb4324b8921]
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'.0001231346.00000000000462740887' Value size = 184)
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff6f00370000'x' Value size = 575)
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T08:35:16.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x5564aa454fba]"
2021-01-05T08:35:16.000Z, 0> 2021-01-05 08:35:16.049 7fb4240b9700 -1 *** Caught signal (Aborted) **
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z, 3: (abort()+0x141) [0x7fb4324b8921]
2021-01-05T08:35:16.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'._fastinfo' Value size = 186)
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff6f00370000'x' Value size = 575)
2021-01-05T08:35:16.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T08:35:16.000Z,Merge( Prefix = b key = 0x000000ca0dd80000 Value size = 16)
2021-01-05T08:35:16.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T08:35:16.000Z, 0> 2021-01-05 08:35:16.049 7fb4240b9700 -1 *** Caught signal (Aborted) **
2021-01-05T08:35:16.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x5564aa9ebd8d]
2021-01-05T08:35:16.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x5564aa45500b]"
2021-01-05T08:35:16.000Z, 5: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z, 9: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z, 2: (gsignal()+0xc7) [0x7fb4324b6fb7]
2021-01-05T08:35:16.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x5564aa455195]"
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z,2021-01-05 08:35:16.049 7fb4240b9700 -1 *** Caught signal (Aborted) **
2021-01-05T08:35:16.000Z, 1: (()+0x12980) [0x7fb433807980]
2021-01-05T08:35:16.000Z, 2: (gsignal()+0xc7) [0x7fb4324b6fb7]
2021-01-05T08:35:16.000Z, 3: (abort()+0x141) [0x7fb4324b8921]
2021-01-05T08:35:16.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x5564aa45500b]"
2021-01-05T08:35:16.000Z,Put( Prefix = P key = 0x00000000000009cf'.0001231346.00000000000462740887' Value size = 184)
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff'o' Value size = 521)
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fb4240b9700 time 2021-01-05 08:35:16.020769
2021-01-05T08:35:16.000Z, 5: (()+0x76db) [0x7fb4337fc6db]
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z, 6: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T08:35:16.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x5564aa9c8bb4]
2021-01-05T08:35:16.000Z,*** Caught signal (Aborted) **
2021-01-05T08:35:16.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T08:35:16.000Z," -2> 2021-01-05 08:35:16.017 7fb4240b9700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 342500554, got 1411691565 in db/096069.sst offset 32090823 size 521815 code = 2 Rocksdb transaction:"
2021-01-05T08:35:16.000Z, 3: (abort()+0x141) [0x7fb4324b8921]
2021-01-05T08:35:16.000Z,Put( Prefix = O key = 0x7f80000000000000023e41c9e4217262'd_data.da5731c8138641.000000000000db4a!='0xfffffffffffffffeffffffffffffffff'o' Value size = 521)
2021-01-05T08:35:16.000Z, 9: (clone()+0x3f) [0x7fb43259971f]
2021-01-05T08:35:16.000Z, 2: (gsignal()+0xc7) [0x7fb4324b6fb7]
2021-01-05T08:35:17.000Z,2021-01-05 08:35:17.841 7f6510a7cc00 -1 Falling back to public interface
2021-01-05T08:35:44.000Z,2021-01-05 08:35:44.553 7f6510a7cc00 -1 osd.56 1231346 log_to_monitors {default=true}
2021-01-05T13:14:19.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55c3c48c700b]"
2021-01-05T13:14:19.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 9: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z,"2021-01-05 13:14:19.343 7fbae0d6c700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2276580208, got 2324967102 in db/118574.sst offset 1491565 size 4016 code = 2 Rocksdb transaction:"
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'._fastinfo' Value size = 186)
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff6f000c0000'x' Value size = 870)
2021-01-05T13:14:19.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z,2021-01-05 13:14:19.367 7fbae0d6c700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fbae0d6c700 time 2021-01-05 13:14:19.348326
2021-01-05T13:14:19.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T13:14:19.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z,*** Caught signal (Aborted) **
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z, 3: (abort()+0x141) [0x7fbaef16b921]
2021-01-05T13:14:19.000Z,2021-01-05 13:14:19.375 7fbae0d6c700 -1 *** Caught signal (Aborted) **
2021-01-05T13:14:19.000Z, in thread 7fbae0d6c700 thread_name:bstore_kv_sync
2021-01-05T13:14:19.000Z, 5: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z, 6: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z, 8: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z, in thread 7fbae0d6c700 thread_name:bstore_kv_sync
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z, 8: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T13:14:19.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55c3c48c6fba]"
2021-01-05T13:14:19.000Z, 0> 2021-01-05 13:14:19.375 7fbae0d6c700 -1 *** Caught signal (Aborted) **
2021-01-05T13:14:19.000Z, in thread 7fbae0d6c700 thread_name:bstore_kv_sync
2021-01-05T13:14:19.000Z, 1: (()+0x12980) [0x7fbaf04ba980]
2021-01-05T13:14:19.000Z, 2: (gsignal()+0xc7) [0x7fbaef169fb7]
2021-01-05T13:14:19.000Z," -2> 2021-01-05 13:14:19.343 7fbae0d6c700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2276580208, got 2324967102 in db/118574.sst offset 1491565 size 4016 code = 2 Rocksdb transaction:"
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'.0001231357.00000000000483198465' Value size = 184)
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'._fastinfo' Value size = 186)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f97b880000 Value size = 16)
2021-01-05T13:14:19.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z, 1: (()+0x12980) [0x7fbaf04ba980]
2021-01-05T13:14:19.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55c3c48c700b]"
2021-01-05T13:14:19.000Z, 9: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 8: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'.0001231357.00000000000483198465' Value size = 184)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f97b880000 Value size = 16)
2021-01-05T13:14:19.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55c3c48c6fba]"
2021-01-05T13:14:19.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 5: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z, 6: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55c3c48c6fba]"
2021-01-05T13:14:19.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55c3c48c6fba]"
2021-01-05T13:14:19.000Z, 6: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z, 1: (()+0x12980) [0x7fbaf04ba980]
2021-01-05T13:14:19.000Z, 0> 2021-01-05 13:14:19.375 7fbae0d6c700 -1 *** Caught signal (Aborted) **
2021-01-05T13:14:19.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z, 3: (abort()+0x141) [0x7fbaef16b921]
2021-01-05T13:14:19.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T13:14:19.000Z, -1> 2021-01-05 13:14:19.367 7fbae0d6c700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fbae0d6c700 time 2021-01-05 13:14:19.348326
2021-01-05T13:14:19.000Z, in thread 7fbae0d6c700 thread_name:bstore_kv_sync
2021-01-05T13:14:19.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 2: (gsignal()+0xc7) [0x7fbaef169fb7]
2021-01-05T13:14:19.000Z, 2: (gsignal()+0xc7) [0x7fbaef169fb7]
2021-01-05T13:14:19.000Z, 2: (gsignal()+0xc7) [0x7fbaef169fb7]
2021-01-05T13:14:19.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 9: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'._fastinfo' Value size = 186)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f97b880000 Value size = 16)
2021-01-05T13:14:19.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 3: (abort()+0x141) [0x7fbaef16b921]
2021-01-05T13:14:19.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55c3c48c7195]"
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff'o' Value size = 520)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f943900000 Value size = 16)
2021-01-05T13:14:19.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fbae0d6c700 time 2021-01-05 13:14:19.348326
2021-01-05T13:14:19.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 5: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55c3c48c700b]"
2021-01-05T13:14:19.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 8: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z, 9: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z, 3: (abort()+0x141) [0x7fbaef16b921]
2021-01-05T13:14:19.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff6f000c0000'x' Value size = 870)
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff'o' Value size = 520)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f943900000 Value size = 16)
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 1: (()+0x12980) [0x7fbaf04ba980]
2021-01-05T13:14:19.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55c3c48c700b]"
2021-01-05T13:14:19.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55c3c4e3abb4]
2021-01-05T13:14:19.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T13:14:19.000Z," -2> 2021-01-05 13:14:19.343 7fbae0d6c700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2276580208, got 2324967102 in db/118574.sst offset 1491565 size 4016 code = 2 Rocksdb transaction:"
2021-01-05T13:14:19.000Z,Put( Prefix = P key = 0x0000000000000960'.0001231357.00000000000483198465' Value size = 184)
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff6f000c0000'x' Value size = 870)
2021-01-05T13:14:19.000Z,Put( Prefix = O key = 0x7f80000000000000021d36b296217262'd_data.da5731c8138641.000000000000d868!='0xfffffffffffffffeffffffffffffffff'o' Value size = 520)
2021-01-05T13:14:19.000Z,Merge( Prefix = b key = 0x000000f943900000 Value size = 16)
2021-01-05T13:14:19.000Z, -1> 2021-01-05 13:14:19.367 7fbae0d6c700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7fbae0d6c700 time 2021-01-05 13:14:19.348326
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:19.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55c3c4e5dd8d]
2021-01-05T13:14:19.000Z, 5: (()+0x76db) [0x7fbaf04af6db]
2021-01-05T13:14:19.000Z, 6: (clone()+0x3f) [0x7fbaef24c71f]
2021-01-05T13:14:19.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T13:14:21.000Z,2021-01-05 13:14:21.051 7f2b42379c00 -1 Falling back to public interface
2021-01-05T13:14:51.000Z,2021-01-05 13:14:51.898 7f2b42379c00 -1 osd.58 1231357 log_to_monitors {default=true}
2021-01-05T16:06:29.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 2: (gsignal()+0xc7) [0x7f3331b6bfb7]
2021-01-05T16:06:29.000Z, 3: (abort()+0x141) [0x7f3331b6d921]
2021-01-05T16:06:29.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z, 6: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z, 0> 2021-01-05 16:06:29.859 7f332376e700 -1 *** Caught signal (Aborted) **
2021-01-05T16:06:29.000Z, 1: (()+0x12980) [0x7f3332ebc980]
2021-01-05T16:06:29.000Z, 3: (abort()+0x141) [0x7f3331b6d921]
2021-01-05T16:06:29.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x558bbd44900b]"
2021-01-05T16:06:29.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 8: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z," -2> 2021-01-05 16:06:29.835 7f332376e700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2558176788, got 2324967102 in db/135611.sst offset 1771197 size 3885 code = 2 Rocksdb transaction:"
2021-01-05T16:06:29.000Z, -1> 2021-01-05 16:06:29.855 7f332376e700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f332376e700 time 2021-01-05 16:06:29.842283
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x558bbd448fba]"
2021-01-05T16:06:29.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 6: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 5: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, in thread 7f332376e700 thread_name:bstore_kv_sync
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z, 2: (gsignal()+0xc7) [0x7f3331b6bfb7]
2021-01-05T16:06:29.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 1: (()+0x12980) [0x7f3332ebc980]
2021-01-05T16:06:29.000Z, 8: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z," -2> 2021-01-05 16:06:29.835 7f332376e700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2558176788, got 2324967102 in db/135611.sst offset 1771197 size 3885 code = 2 Rocksdb transaction:"
2021-01-05T16:06:29.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e00000 Value size = 16)
2021-01-05T16:06:29.000Z, 2: (gsignal()+0xc7) [0x7f3331b6bfb7]
2021-01-05T16:06:29.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x558bbd44900b]"
2021-01-05T16:06:29.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'.0001231371.00000000000457091339' Value size = 184)
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'._fastinfo' Value size = 186)
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff'o' Value size = 446)
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e00000 Value size = 16)
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e80000 Value size = 16)
2021-01-05T16:06:29.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T16:06:29.000Z, in thread 7f332376e700 thread_name:bstore_kv_sync
2021-01-05T16:06:29.000Z, 2: (gsignal()+0xc7) [0x7f3331b6bfb7]
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'.0001231371.00000000000457091339' Value size = 184)
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'._fastinfo' Value size = 186)
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff'o' Value size = 446)
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e80000 Value size = 16)
2021-01-05T16:06:29.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f332376e700 time 2021-01-05 16:06:29.842283
2021-01-05T16:06:29.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T16:06:29.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x558bbd448fba]"
2021-01-05T16:06:29.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z,2021-01-05 16:06:29.855 7f332376e700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f332376e700 time 2021-01-05 16:06:29.842283
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x558bbd448fba]"
2021-01-05T16:06:29.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 5: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, 6: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 8: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, 9: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z,2021-01-05 16:06:29.859 7f332376e700 -1 *** Caught signal (Aborted) **
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 8: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, 9: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z, 9: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z, in thread 7f332376e700 thread_name:bstore_kv_sync
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff6f003c0000'x' Value size = 510)
2021-01-05T16:06:29.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x558bbd9bcbb4]
2021-01-05T16:06:29.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x558bbd448fba]"
2021-01-05T16:06:29.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, 5: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z, 9: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'.0001231371.00000000000457091339' Value size = 184)
2021-01-05T16:06:29.000Z,Put( Prefix = P key = 0x0000000000000a7c'._fastinfo' Value size = 186)
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff6f003c0000'x' Value size = 510)
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff'o' Value size = 446)
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e80000 Value size = 16)
2021-01-05T16:06:29.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T16:06:29.000Z, 5: (()+0x76db) [0x7f3332eb16db]
2021-01-05T16:06:29.000Z, 0> 2021-01-05 16:06:29.859 7f332376e700 -1 *** Caught signal (Aborted) **
2021-01-05T16:06:29.000Z, in thread 7f332376e700 thread_name:bstore_kv_sync
2021-01-05T16:06:29.000Z, 1: (()+0x12980) [0x7f3332ebc980]
2021-01-05T16:06:29.000Z, 3: (abort()+0x141) [0x7f3331b6d921]
2021-01-05T16:06:29.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x558bbd449195]"
2021-01-05T16:06:29.000Z,"2021-01-05 16:06:29.835 7f332376e700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2558176788, got 2324967102 in db/135611.sst offset 1771197 size 3885 code = 2 Rocksdb transaction:"
2021-01-05T16:06:29.000Z,Put( Prefix = O key = 0x7f8000000000000002e02cdc0c217262'd_data.756486b4486086.0000000000000128!='0xfffffffffffffffeffffffffffffffff6f003c0000'x' Value size = 510)
2021-01-05T16:06:29.000Z,Merge( Prefix = b key = 0x0000011048e00000 Value size = 16)
2021-01-05T16:06:29.000Z, 6: (clone()+0x3f) [0x7f3331c4e71f]
2021-01-05T16:06:29.000Z,*** Caught signal (Aborted) **
2021-01-05T16:06:29.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-05T16:06:29.000Z, 1: (()+0x12980) [0x7f3332ebc980]
2021-01-05T16:06:29.000Z, 3: (abort()+0x141) [0x7f3331b6d921]
2021-01-05T16:06:29.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x558bbd44900b]"
2021-01-05T16:06:29.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T16:06:29.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x558bbd44900b]"
2021-01-05T16:06:29.000Z, -1> 2021-01-05 16:06:29.855 7f332376e700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f332376e700 time 2021-01-05 16:06:29.842283
2021-01-05T16:06:29.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x558bbd9dfd8d]
2021-01-05T16:06:29.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-05T16:06:29.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-05T16:06:31.000Z,2021-01-05 16:06:31.499 7fdfcebf6c00 -1 Falling back to public interface
2021-01-05T16:07:03.000Z,2021-01-05 16:07:03.375 7fdfcebf6c00 -1 osd.54 1231371 log_to_monitors {default=true}
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7f3730c42700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.016 7f6507183700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7f2b38a80700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.012 7f4f7af07700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.016 7fdfc52fd700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.012 7fdcbca25700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.016 7f2b38a80700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7f4f7af07700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7f6507183700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.012 7f3730c42700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 933185) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7fdfc52fd700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T06:25:06.000Z,2021-01-06 06:25:06.052 7fdcbca25700 -1 received signal: Hangup from (PID: 933186) UID: 0
2021-01-06T21:58:00.000Z,Put( Prefix = X key = 0x000000000417b1b3 Value size = 84)
2021-01-06T21:58:00.000Z, 2: (gsignal()+0xc7) [0x7f4f81092fb7]
2021-01-06T21:58:00.000Z,Merge( Prefix = T key = 0x0000000000000002 Value size = 40)
2021-01-06T21:58:00.000Z, 9: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z, in thread 7f4f72c95700 thread_name:bstore_kv_sync
2021-01-06T21:58:00.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x561b43312fba]"
2021-01-06T21:58:00.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z, 0> 2021-01-06 21:58:00.498 7f4f72c95700 -1 *** Caught signal (Aborted) **
2021-01-06T21:58:00.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x561b4331300b]"
2021-01-06T21:58:00.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 8: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z,Merge( Prefix = b key = 0x0000012194500000 Value size = 16)
2021-01-06T21:58:00.000Z, -1> 2021-01-06 21:58:00.494 7f4f72c95700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f4f72c95700 time 2021-01-06 21:58:00.473650
2021-01-06T21:58:00.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 5: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z, 2: (gsignal()+0xc7) [0x7f4f81092fb7]
2021-01-06T21:58:00.000Z, 3: (abort()+0x141) [0x7f4f81094921]
2021-01-06T21:58:00.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'._fastinfo' Value size = 186)
2021-01-06T21:58:00.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f4f72c95700 time 2021-01-06 21:58:00.473650
2021-01-06T21:58:00.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 6: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z, in thread 7f4f72c95700 thread_name:bstore_kv_sync
2021-01-06T21:58:00.000Z,Merge( Prefix = b key = 0x0000012194500000 Value size = 16)
2021-01-06T21:58:00.000Z, -1> 2021-01-06 21:58:00.494 7f4f72c95700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f4f72c95700 time 2021-01-06 21:58:00.473650
2021-01-06T21:58:00.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-06T21:58:00.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x561b4331300b]"
2021-01-06T21:58:00.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 1: (()+0x12980) [0x7f4f823e3980]
2021-01-06T21:58:00.000Z, 8: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z,Put( Prefix = X key = 0x000000000417b1b3 Value size = 84)
2021-01-06T21:58:00.000Z,Merge( Prefix = T key = 0x0000000000000002 Value size = 40)
2021-01-06T21:58:00.000Z, 6: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z,2021-01-06 21:58:00.494 7f4f72c95700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f4f72c95700 time 2021-01-06 21:58:00.473650
2021-01-06T21:58:00.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-06T21:58:00.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x561b43312fba]"
2021-01-06T21:58:00.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff6f00120000'x' Value size = 968)
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z, 6: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-06T21:58:00.000Z," -2> 2021-01-06 21:58:00.470 7f4f72c95700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2259072026, got 13154490 in db/130399.sst offset 30407666 size 496736 code = 2 Rocksdb transaction:"
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'._fastinfo' Value size = 186)
2021-01-06T21:58:00.000Z, 3: (abort()+0x141) [0x7f4f81094921]
2021-01-06T21:58:00.000Z,Merge( Prefix = T key = 0x0000000000000002 Value size = 40)
2021-01-06T21:58:00.000Z, 8: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z,2021-01-06 21:58:00.498 7f4f72c95700 -1 *** Caught signal (Aborted) **
2021-01-06T21:58:00.000Z, 1: (()+0x12980) [0x7f4f823e3980]
2021-01-06T21:58:00.000Z, 2: (gsignal()+0xc7) [0x7f4f81092fb7]
2021-01-06T21:58:00.000Z, 3: (abort()+0x141) [0x7f4f81094921]
2021-01-06T21:58:00.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z, 8: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z, 9: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z,*** Caught signal (Aborted) **
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff'o' Value size = 525)
2021-01-06T21:58:00.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x561b4331300b]"
2021-01-06T21:58:00.000Z, 0> 2021-01-06 21:58:00.498 7f4f72c95700 -1 *** Caught signal (Aborted) **
2021-01-06T21:58:00.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x561b4331300b]"
2021-01-06T21:58:00.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 9: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-06T21:58:00.000Z,"2021-01-06 21:58:00.470 7f4f72c95700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2259072026, got 13154490 in db/130399.sst offset 30407666 size 496736 code = 2 Rocksdb transaction:"
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'.0001232543.00000000000445964007' Value size = 184)
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff6f00120000'x' Value size = 968)
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff'o' Value size = 525)
2021-01-06T21:58:00.000Z,Merge( Prefix = b key = 0x0000012194500000 Value size = 16)
2021-01-06T21:58:00.000Z," -2> 2021-01-06 21:58:00.470 7f4f72c95700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 2259072026, got 13154490 in db/130399.sst offset 30407666 size 496736 code = 2 Rocksdb transaction:"
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'.0001232543.00000000000445964007' Value size = 184)
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'._fastinfo' Value size = 186)
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff'o' Value size = 525)
2021-01-06T21:58:00.000Z, 5: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z, in thread 7f4f72c95700 thread_name:bstore_kv_sync
2021-01-06T21:58:00.000Z, 1: (()+0x12980) [0x7f4f823e3980]
2021-01-06T21:58:00.000Z, 2: (gsignal()+0xc7) [0x7f4f81092fb7]
2021-01-06T21:58:00.000Z, 5: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z, 3: (abort()+0x141) [0x7f4f81094921]
2021-01-06T21:58:00.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x561b43313195]"
2021-01-06T21:58:00.000Z, 9: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z,Put( Prefix = P key = 0x0000000000000954'.0001232543.00000000000445964007' Value size = 184)
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x561b43312fba]"
2021-01-06T21:58:00.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 5: (()+0x76db) [0x7f4f823d86db]
2021-01-06T21:58:00.000Z, 1: (()+0x12980) [0x7f4f823e3980]
2021-01-06T21:58:00.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z,Put( Prefix = O key = 0x7f8000000000000002fc30277f217262'd_data.e06f6e1e6e8b48.0000000000001825!='0xfffffffffffffffeffffffffffffffff6f00120000'x' Value size = 968)
2021-01-06T21:58:00.000Z,Put( Prefix = X key = 0x000000000417b1b3 Value size = 84)
2021-01-06T21:58:00.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-06T21:58:00.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-06T21:58:00.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x561b43312fba]"
2021-01-06T21:58:00.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x561b43886bb4]
2021-01-06T21:58:00.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x561b438a9d8d]
2021-01-06T21:58:00.000Z, 6: (clone()+0x3f) [0x7f4f8117571f]
2021-01-06T21:58:00.000Z, in thread 7f4f72c95700 thread_name:bstore_kv_sync
2021-01-06T21:58:02.000Z,2021-01-06 21:58:02.310 7f5c99b3bc00 -1 Falling back to public interface
2021-01-06T21:58:32.000Z,2021-01-06 21:58:32.866 7f5c99b3bc00 -1 osd.57 1232543 log_to_monitors {default=true}
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.013 7f3730c42700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7f5c90242700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7f6507183700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.013 7fdcbca25700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.017 7f5c90242700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.017 7fdfc52fd700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7fdfc52fd700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.017 7f2b38a80700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.017 7f6507183700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1290845) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7f3730c42700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7fdcbca25700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-07T06:25:06.000Z,2021-01-07 06:25:06.053 7f2b38a80700 -1 received signal: Hangup from (PID: 1290846) UID: 0
2021-01-08T06:13:54.000Z,Put( Prefix = P key = 0x0000000000000974'.0001234154.00000000000472581834' Value size = 184)
2021-01-08T06:13:54.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff'o' Value size = 541)
2021-01-08T06:13:54.000Z,Merge( Prefix = b key = 0x0000012a02680000 Value size = 16)
2021-01-08T06:13:54.000Z,Merge( Prefix = b key = 0x0000017981680000 Value size = 16)
2021-01-08T06:13:54.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-08T06:13:54.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55fd49604fba]"
2021-01-08T06:13:54.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:54.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:54.000Z,*** Caught signal (Aborted) **
2021-01-08T06:13:54.000Z, in thread 7f37289d0700 thread_name:bstore_kv_sync
2021-01-08T06:13:54.000Z, 1: (()+0x12980) [0x7f373811e980]
2021-01-08T06:13:54.000Z, 3: (abort()+0x141) [0x7f3736dcf921]
2021-01-08T06:13:54.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:54.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:54.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:54.000Z, 8: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:54.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f37289d0700 time 2021-01-08 06:13:54.954115
2021-01-08T06:13:54.000Z, 6: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:54.000Z,2021-01-08 06:13:54.974 7f37289d0700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f37289d0700 time 2021-01-08 06:13:54.954115
2021-01-08T06:13:54.000Z,Put( Prefix = P key = 0x0000000000000974'._fastinfo' Value size = 186)
2021-01-08T06:13:54.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:54.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:54.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:54.000Z, 5: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:54.000Z, 9: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:54.000Z, 2: (gsignal()+0xc7) [0x7f3736dcdfb7]
2021-01-08T06:13:54.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55fd4960500b]"
2021-01-08T06:13:54.000Z, 1: (()+0x12980) [0x7f373811e980]
2021-01-08T06:13:54.000Z, 2: (gsignal()+0xc7) [0x7f3736dcdfb7]
2021-01-08T06:13:54.000Z, 3: (abort()+0x141) [0x7f3736dcf921]
2021-01-08T06:13:54.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55fd4960500b]"
2021-01-08T06:13:54.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:54.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:54.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:54.000Z, 6: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:54.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:54.000Z, 9: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:54.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-08T06:13:54.000Z,"2021-01-08 06:13:54.946 7f37289d0700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 1858706858, got 291516022 in db/146430.sst offset 32863300 size 538979 code = 2 Rocksdb transaction:"
2021-01-08T06:13:54.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff6f00000000'x' Value size = 468)
2021-01-08T06:13:54.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:54.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-08T06:13:54.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:54.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55fd49604fba]"
2021-01-08T06:13:54.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:54.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:54.000Z, 5: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:54.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:54.000Z, 8: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:54.000Z,2021-01-08 06:13:54.978 7f37289d0700 -1 *** Caught signal (Aborted) **
2021-01-08T06:13:54.000Z, in thread 7f37289d0700 thread_name:bstore_kv_sync
2021-01-08T06:13:54.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:55.000Z, in thread 7f37289d0700 thread_name:bstore_kv_sync
2021-01-08T06:13:55.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55fd4960500b]"
2021-01-08T06:13:55.000Z, 9: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:55.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012a02680000 Value size = 16)
2021-01-08T06:13:55.000Z, -1> 2021-01-08 06:13:54.974 7f37289d0700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f37289d0700 time 2021-01-08 06:13:54.954115
2021-01-08T06:13:55.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:55.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:55.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:55.000Z, 6: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:55.000Z, in thread 7f37289d0700 thread_name:bstore_kv_sync
2021-01-08T06:13:55.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:55.000Z," NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this."
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012a02680000 Value size = 16)
2021-01-08T06:13:55.000Z, -1> 2021-01-08 06:13:54.974 7f37289d0700 -1 /build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f37289d0700 time 2021-01-08 06:13:54.954115
2021-01-08T06:13:55.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:55.000Z,Put( Prefix = P key = 0x0000000000000974'.0001234154.00000000000472581834' Value size = 184)
2021-01-08T06:13:55.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-08T06:13:55.000Z, 6: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:55.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:55.000Z, 1: (()+0x12980) [0x7f373811e980]
2021-01-08T06:13:55.000Z, 3: (abort()+0x141) [0x7f3736dcf921]
2021-01-08T06:13:55.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:55.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:55.000Z," -2> 2021-01-08 06:13:54.946 7f37289d0700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 1858706858, got 291516022 in db/146430.sst offset 32863300 size 538979 code = 2 Rocksdb transaction:"
2021-01-08T06:13:55.000Z,Put( Prefix = P key = 0x0000000000000974'.0001234154.00000000000472581834' Value size = 184)
2021-01-08T06:13:55.000Z,Put( Prefix = P key = 0x0000000000000974'._fastinfo' Value size = 186)
2021-01-08T06:13:55.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff6f00000000'x' Value size = 468)
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000017981680000 Value size = 16)
2021-01-08T06:13:55.000Z,/build/ceph-14.2.16/src/os/bluestore/BlueStore.cc: 11380: FAILED ceph_assert(r == 0)
2021-01-08T06:13:55.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:55.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55fd49604fba]"
2021-01-08T06:13:55.000Z, 5: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:55.000Z, 0> 2021-01-08 06:13:54.978 7f37289d0700 -1 *** Caught signal (Aborted) **
2021-01-08T06:13:55.000Z, 2: (gsignal()+0xc7) [0x7f3736dcdfb7]
2021-01-08T06:13:55.000Z, 6: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:55.000Z, 7: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:55.000Z," -2> 2021-01-08 06:13:54.946 7f37289d0700 -1 rocksdb: submit_common error: Corruption: block checksum mismatch: expected 1858706858, got 291516022 in db/146430.sst offset 32863300 size 538979 code = 2 Rocksdb transaction:"
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:55.000Z," 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55fd49604fba]"
2021-01-08T06:13:55.000Z," 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:55.000Z, 3: (BlueStore::_kv_sync_thread()+0x1144) [0x55fd49b78bb4]
2021-01-08T06:13:55.000Z, 5: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:55.000Z, ceph version 14.2.16 (762032d6f509d5e7ee7dc008d80fe9c87086603c) nautilus (stable)
2021-01-08T06:13:55.000Z, 3: (abort()+0x141) [0x7f3736dcf921]
2021-01-08T06:13:55.000Z," 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a3) [0x55fd4960500b]"
2021-01-08T06:13:55.000Z, 9: (clone()+0x3f) [0x7f3736eb071f]
2021-01-08T06:13:55.000Z,Put( Prefix = P key = 0x0000000000000974'._fastinfo' Value size = 186)
2021-01-08T06:13:55.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff6f00000000'x' Value size = 468)
2021-01-08T06:13:55.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff'o' Value size = 541)
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000017981680000 Value size = 16)
2021-01-08T06:13:55.000Z, 4: (BlueStore::KVSyncThread::entry()+0xd) [0x55fd49b9bd8d]
2021-01-08T06:13:55.000Z, 0> 2021-01-08 06:13:54.978 7f37289d0700 -1 *** Caught signal (Aborted) **
2021-01-08T06:13:55.000Z, 2: (gsignal()+0xc7) [0x7f3736dcdfb7]
2021-01-08T06:13:55.000Z," 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55fd49605195]"
2021-01-08T06:13:55.000Z, 8: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:55.000Z,Put( Prefix = O key = 0x7f800000000000000205e3b390217262'd_data.e46d9e7c3dbd3d.0000000000007c24!='0xfffffffffffffffeffffffffffffffff'o' Value size = 541)
2021-01-08T06:13:55.000Z,Merge( Prefix = b key = 0x0000012ca1980000 Value size = 16)
2021-01-08T06:13:55.000Z, 1: (()+0x12980) [0x7f373811e980]
2021-01-08T06:13:55.000Z, 8: (()+0x76db) [0x7f37381136db]
2021-01-08T06:13:56.000Z,2021-01-08 06:13:56.834 7f96097c3c00 -1 Falling back to public interface
2021-01-08T06:14:38.000Z,2021-01-08 06:14:38.534 7f96097c3c00 -1 osd.59 1234154 log_to_monitors {default=true}
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7f5c90242700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.102 7fdcbca25700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7fdcbca25700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7f6507183700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7fdfc52fd700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.102 7f6507183700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.102 7fdfc52fd700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.102 7f2b38a80700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.102 7f5c90242700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7f2b38a80700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.134 7f95ffeca700 -1 received signal: Hangup from (PID: 1651518) UID: 0
2021-01-08T06:25:05.000Z,2021-01-08 06:25:05.106 7f95ffeca700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw rbd-mirror (PID: 1651517) UID: 0
(4-4/6)