Project

General

Profile

Bug #24968 » fschk.txt

Bluestore check - Markus Stockhausen, 07/17/2018 07:15 PM

 
# ceph-bluestore-tool --log-level 30 --path /var/lib/ceph/osd/ceph-5 fsck
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f8150b36700 time 2018-07-17 21:14:54.401203
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: 8626: FAILED assert(r == 0)
2018-07-17 21:14:54.401137 7f8150b36700 -1 rocksdb: submit_transaction_sync error: Corruption: block checksum mismatch code = 2 Rocksdb transaction:
Put( Prefix = S key = 'nid_max' Value size = 8)
Put( Prefix = S key = 'blobid_max' Value size = 8)
SingleDelete(Prefix = L Key = 0x0000000000002a18)
SingleDelete(Prefix = L Key = 0x0000000000002a19)
SingleDelete(Prefix = L Key = 0x0000000000002a1a)
SingleDelete(Prefix = L Key = 0x0000000000002a1b)
SingleDelete(Prefix = L Key = 0x0000000000002a1c)
SingleDelete(Prefix = L Key = 0x0000000000002a1d)
SingleDelete(Prefix = L Key = 0x0000000000002a1e)
SingleDelete(Prefix = L Key = 0x0000000000002a1f)
SingleDelete(Prefix = L Key = 0x0000000000002a20)
SingleDelete(Prefix = L Key = 0x0000000000002a21)
SingleDelete(Prefix = L Key = 0x0000000000002a22)
SingleDelete(Prefix = L Key = 0x0000000000002a23)
SingleDelete(Prefix = L Key = 0x0000000000002a24)
SingleDelete(Prefix = L Key = 0x0000000000002a25)
SingleDelete(Prefix = L Key = 0x0000000000002a26)
SingleDelete(Prefix = L Key = 0x0000000000002a27)
SingleDelete(Prefix = L Key = 0x0000000000002a28)
SingleDelete(Prefix = L Key = 0x0000000000002a29)
ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f8157ef5d50]
2: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
3: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
4: (()+0x7e25) [0x7f81567bbe25]
5: (clone()+0x6d) [0x7f8155276bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2018-07-17 21:14:54.402719 7f8150b36700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f8150b36700 time 2018-07-17 21:14:54.401203
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: 8626: FAILED assert(r == 0)

ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f8157ef5d50]
2: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
3: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
4: (()+0x7e25) [0x7f81567bbe25]
5: (clone()+0x6d) [0x7f8155276bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

-1> 2018-07-17 21:14:54.401137 7f8150b36700 -1 rocksdb: submit_transaction_sync error: Corruption: block checksum mismatch code = 2 Rocksdb transaction:
Put( Prefix = S key = 'nid_max' Value size = 8)
Put( Prefix = S key = 'blobid_max' Value size = 8)
SingleDelete(Prefix = L Key = 0x0000000000002a18)
SingleDelete(Prefix = L Key = 0x0000000000002a19)
SingleDelete(Prefix = L Key = 0x0000000000002a1a)
SingleDelete(Prefix = L Key = 0x0000000000002a1b)
SingleDelete(Prefix = L Key = 0x0000000000002a1c)
SingleDelete(Prefix = L Key = 0x0000000000002a1d)
SingleDelete(Prefix = L Key = 0x0000000000002a1e)
SingleDelete(Prefix = L Key = 0x0000000000002a1f)
SingleDelete(Prefix = L Key = 0x0000000000002a20)
SingleDelete(Prefix = L Key = 0x0000000000002a21)
SingleDelete(Prefix = L Key = 0x0000000000002a22)
SingleDelete(Prefix = L Key = 0x0000000000002a23)
SingleDelete(Prefix = L Key = 0x0000000000002a24)
SingleDelete(Prefix = L Key = 0x0000000000002a25)
SingleDelete(Prefix = L Key = 0x0000000000002a26)
SingleDelete(Prefix = L Key = 0x0000000000002a27)
SingleDelete(Prefix = L Key = 0x0000000000002a28)
SingleDelete(Prefix = L Key = 0x0000000000002a29)
0> 2018-07-17 21:14:54.402719 7f8150b36700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_kv_sync_thread()' thread 7f8150b36700 time 2018-07-17 21:14:54.401203
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.5/rpm/el7/BUILD/ceph-12.2.5/src/os/bluestore/BlueStore.cc: 8626: FAILED assert(r == 0)

ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f8157ef5d50]
2: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
3: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
4: (()+0x7e25) [0x7f81567bbe25]
5: (clone()+0x6d) [0x7f8155276bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

*** Caught signal (Aborted) **
in thread 7f8150b36700 thread_name:bstore_kv_sync
ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (()+0x3eb8e1) [0x561ba006f8e1]
2: (()+0xf6d0) [0x7f81567c36d0]
3: (gsignal()+0x37) [0x7f81551ae277]
4: (abort()+0x148) [0x7f81551af968]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x7f8157ef5ec4]
6: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
8: (()+0x7e25) [0x7f81567bbe25]
9: (clone()+0x6d) [0x7f8155276bad]
2018-07-17 21:14:54.404476 7f8150b36700 -1 *** Caught signal (Aborted) **
in thread 7f8150b36700 thread_name:bstore_kv_sync

ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (()+0x3eb8e1) [0x561ba006f8e1]
2: (()+0xf6d0) [0x7f81567c36d0]
3: (gsignal()+0x37) [0x7f81551ae277]
4: (abort()+0x148) [0x7f81551af968]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x7f8157ef5ec4]
6: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
8: (()+0x7e25) [0x7f81567bbe25]
9: (clone()+0x6d) [0x7f8155276bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

0> 2018-07-17 21:14:54.404476 7f8150b36700 -1 *** Caught signal (Aborted) **
in thread 7f8150b36700 thread_name:bstore_kv_sync

ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
1: (()+0x3eb8e1) [0x561ba006f8e1]
2: (()+0xf6d0) [0x7f81567c36d0]
3: (gsignal()+0x37) [0x7f81551ae277]
4: (abort()+0x148) [0x7f81551af968]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x284) [0x7f8157ef5ec4]
6: (BlueStore::_kv_sync_thread()+0x34bf) [0x561b9fec62ff]
7: (BlueStore::KVSyncThread::entry()+0xd) [0x561b9ff144ad]
8: (()+0x7e25) [0x7f81567bbe25]
9: (clone()+0x6d) [0x7f8155276bad]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Aborted (core dumped)
(1-1/5)