Actions
Bug #34526
closedOSD crash in KernelDevice::direct_read_unaligned while scrubbing
Status:
Duplicate
Priority:
Normal
Assignee:
-
Target version:
-
% Done:
0%
Source:
Community (user)
Tags:
Luminous 12.2.7
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Ceph version: Luminous 12.2.7
After I migrate one pool date to another pool(by change pool crush rule). I found much osds down when they do pg scrub.
The stack as below:
2018-08-20 14:29:58.310716 7f473f0a0700 -1 ** Caught signal (Aborted) *
in thread 7f473f0a0700 thread_name:tp_osd_tp
ceph version 12.2.7 (3ec878d1e53e1aeb47a9f619c49d9e7c0aa384d5) luminous (stable)
1: (()+0xa84aa4) [0x5603e7af5aa4]
2: (()+0x11390) [0x7f475f23b390]
3: (pread64()+0x33) [0x7f475f23ad43]
4: (KernelDevice::direct_read_unaligned(unsigned long, unsigned long, char*)+0x81) [0x5603e7ad0241]
5: (KernelDevice::read_random(unsigned long, unsigned long, char*, bool)+0x563) [0x5603e7ad0d23]
6: (BlueFS::_read_random(BlueFS::FileReader*, unsigned long, unsigned long, char*)+0x4f2) [0x5603e7aa2312]
7: (BlueRocksRandomAccessFile::Read(unsigned long, unsigned long, rocksdb::Slice*, char*) const+0x20) [0x5603e7acc1f0]
8: (rocksdb::RandomAccessFileReader::Read(unsigned long, unsigned long, rocksdb::Slice*, char*) const+0xf8f) [0x5603e7f0bf1f]
9: (rocksdb::ReadBlockContents(rocksdb::RandomAccessFileReader*, rocksdb::Footer const&, rocksdb::ReadOptions const&, rocksdb::BlockHandle const&, rocksdb::BlockContents*, rocksdb::ImmutableCFOptions const&, bool, rocksdb::Slice const&, rocksdb::PersistentCacheOptions const&)+0x5f3) [0x5603e7edc8f3]
10: (()+0xe5b836) [0x5603e7ecc836]
11: (rocksdb::BlockBasedTable::MaybeLoadDataBlockToCache(rocksdb::BlockBasedTable::Rep*, rocksdb::ReadOptions const&, rocksdb::BlockHandle const&, rocksdb::Slice, rocksdb::BlockBasedTable::CachableEntry<rocksdb::Block>*, bool)+0x2f8) [0x5603e7ece998]
12: (rocksdb::BlockBasedTable::NewDataBlockIterator(rocksdb::BlockBasedTable::Rep*, rocksdb::ReadOptions const&, rocksdb::BlockHandle const&, rocksdb::BlockIter*, bool, rocksdb::Status)+0x2ac) [0x5603e7eced9c]
13: (rocksdb::BlockBasedTable::BlockEntryIteratorState::NewSecondaryIterator(rocksdb::Slice const&)+0x97) [0x5603e7ed7517]
14: (()+0xe91bbe) [0x5603e7f02bbe]
15: (()+0xe91c86) [0x5603e7f02c86]
16: (()+0xe91e01) [0x5603e7f02e01]
17: (rocksdb::MergingIterator::Next()+0x449) [0x5603e7ee6079]
18: (rocksdb::DBIter::FindNextUserEntryInternal(bool, bool)+0x182) [0x5603e7f832e2]
19: (rocksdb::DBIter::Next()+0x1eb) [0x5603e7f8409b]
20: (RocksDBStore::RocksDBWholeSpaceIteratorImpl::next()+0x9a) [0x5603e7a322ca]
21: (BlueStore::_collection_list(BlueStore::Collection*, ghobject_t const&, ghobject_t const&, int, std::vector<ghobject_t, std::allocator<ghobject_t> >, ghobject_t)+0x133e) [0x5603e798defe]
22: (BlueStore::collection_list(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ghobject_t const&, ghobject_t const&, int, std::vector<ghobject_t, std::allocator<ghobject_t> >, ghobject_t)+0x25a) [0x5603e798f1ca]
23: (PGBackend::objects_list_range(hobject_t const&, hobject_t const&, snapid_t, std::vector<hobject_t, std::allocator<hobject_t> >, std::vector<ghobject_t, std::allocator<ghobject_t> >)+0x192) [0x5603e774b6c2]
24: (PG::build_scrub_map_chunk(ScrubMap&, ScrubMapBuilder&, hobject_t, hobject_t, bool, ThreadPool::TPHandle&)+0x744) [0x5603e75e6794]
25: (PG::chunky_scrub(ThreadPool::TPHandle&)+0x484) [0x5603e7612c54]
26: (PG::scrub(unsigned int, ThreadPool::TPHandle&)+0x2cb) [0x5603e76152db]
27: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x106d) [0x5603e7556add]
28: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x884) [0x5603e7b3d7e4]
29: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x5603e7b40820]
30: (()+0x76ba) [0x7f475f2316ba]
31: (clone()+0x6d) [0x7f475e2a841d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.Files
Actions