Actions
Bug #18179
closedMDS crashes on missing metadata object
% Done:
0%
Source:
Community (user)
Tags:
jewel,cephfs,mds,metadata
Backport:
jewel, kraken
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Saw this crash happening on a Jewel 10.2.3 MDS when it was missing a object in the metadata pool:
2016-12-07 08:29:58.852595 7f3d74c96700 -1 log_channel(cluster) log [ERR] : dir 10011a4767b object missing on disk; some files may be lost 2016-12-07 08:29:58.855070 7f3d74c96700 -1 mds/MDCache.cc: In function 'virtual void C_MDC_OpenInoTraverseDir::finish(int)' thread 7f3d74c96700 time 2016-12-07 08:29:58.852637 mds/MDCache.cc: 8213: FAILED assert(r >= 0) ceph version 10.2.3 (ecc23778eb545d8dd55e2e4735b53cc93f92e65b) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x7f3d7fbe3a45] 2: (C_MDC_OpenInoTraverseDir::finish(int)+0x13e) [0x7f3d7f93d59e] 3: (MDSInternalContextBase::complete(int)+0x1eb) [0x7f3d7fa408ab] 4: (CDir::finish_waiting(unsigned long, int)+0x324) [0x7f3d7f9bb5a4] 5: (CDir::_omap_fetched(ceph::buffer::list&, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >&, std::string const&, int)+0x65a) [0x7f3d7f9d29ba] 6: (CDir::_tmap_fetched(ceph::buffer::list&, std::string const&, int)+0x368) [0x7f3d7f9d40f8] 7: (MDSIOContextBase::complete(int)+0xa4) [0x7f3d7fa40bd4] 8: (Finisher::finisher_thread_entry()+0x27d) [0x7f3d7fb0f17d] 9: (()+0x7dc5) [0x7f3d7e9ccdc5] 10: (clone()+0x6d) [0x7f3d7d497ced] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
As per discussion on the ceph-users list I'm opening this issue so that the MDS can be patched to record damaged metadata and continue it's operation.
Updated by Zheng Yan over 7 years ago
- Status changed from New to Fix Under Review
Updated by John Spray over 7 years ago
- Status changed from Fix Under Review to Pending Backport
- Backport set to jewel kraken
Updated by Nathan Cutler over 7 years ago
- Backport changed from jewel kraken to jewel, kraken
Updated by Nathan Cutler over 7 years ago
- Copied to Backport #18565: jewel: MDS crashes on missing metadata object added
Updated by Nathan Cutler over 7 years ago
- Copied to Backport #18566: kraken: MDS crashes on missing metadata object added
Updated by Nathan Cutler about 7 years ago
- Status changed from Pending Backport to Resolved
Actions