Actions
Bug #21252
closedmds: asok command error merged with partial Formatter output
Status:
Resolved
Priority:
Normal
Assignee:
Category:
Administration/Usability
Target version:
% Done:
0%
Source:
Q/A
Tags:
Backport:
luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2017-08-29T21:06:14.093 INFO:teuthology.orchestra.run.smithi141:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok dump tree /ceph-qa-suite/suites' 2017-08-29T21:06:14.711 INFO:teuthology.orchestra.run.smithi141.stdout:Failed to dump tree: (1) Operation not permitted[ 2017-08-29T21:06:14.746 INFO:teuthology.orchestra.run.smithi141.stdout: { 2017-08-29T21:06:14.833 INFO:teuthology.orchestra.run.smithi141.stdout: "ino": 1099511629559, 2017-08-29T21:06:14.846 INFO:teuthology.orchestra.run.smithi141.stdout: "rdev": 0, 2017-08-29T21:06:15.250 INFO:teuthology.orchestra.run.smithi141.stdout: "ctime": "2017-08-29 20:55:16.552499", 2017-08-29T21:06:15.364 INFO:teuthology.orchestra.run.smithi141.stdout: "btime": "2017-08-29 20:55:16.552499", 2017-08-29T21:06:15.405 INFO:teuthology.orchestra.run.smithi141.stdout: "mode": 33188, 2017-08-29T21:06:15.411 INFO:teuthology.orchestra.run.smithi141.stdout: "uid": 0, ...
From: /ceph/teuthology-archive/pdonnell-2017-08-29_20:29:11-fs-wip-pdonnell-testing-20170829-distro-basic-smithi/1578245/teuthology.log
Updated by Patrick Donnelly over 6 years ago
- Status changed from New to Fix Under Review
Updated by Patrick Donnelly over 6 years ago
I should note: the error itself is very concerning because the only way for dump_cache to fail is if it's operating on a file, which it isn't in this case.
This could be memory corruption on the stack but I don't immediately see the reason why. I've seen this appear in two jobs for the last test run. I'm seeing if it's still reproducible in the next run.
Updated by Zheng Yan over 6 years ago
Sorry, the bug was introduced by my commit:
commit 7b9eae62c8c654ff82684451c222257d2c93be64 (HEAD) Author: Yan, Zheng <zyan@redhat.com> Date: Wed Aug 2 17:26:56 2017 +0800 mds: track snap inodes through sorted map Current mds track both head inodes and snap inodes through unsorted map. The unsorted map makes finding snap inode that follows a given snapid difficult. Currnt MDCache::pick_inode_snap() use snap set to guess snap inode's last. The method isn't reliable because snap set may change after creating the snap inode. For example: MDS cows inode[2,head] with snap set[5,6], which results inode[2,6] and inode[7,head]. Later mds wants to find snap inode that follows snapid 2. But the snap set become [5], mds can't find snap inode [2,5]. Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
below incremental patch can fix it
diff --git a/src/mds/MDCache.cc b/src/mds/MDCache.cc index 2fbccccd6e..0bb1cda9b9 100644 --- a/src/mds/MDCache.cc +++ b/src/mds/MDCache.cc @@ -11979,6 +11979,7 @@ int MDCache::dump_cache(const char *fn, Formatter *f, if (r < 0) goto out; } + r = 0; out: if (f) {
https://github.com/ceph/ceph/pull/16778/commits/f519fca9dd958121a289676edf5175fb8be9894f
Updated by Patrick Donnelly over 6 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #21321: luminous: mds: asok command error merged with partial Formatter output added
Updated by Nathan Cutler over 6 years ago
- Status changed from Pending Backport to Resolved
Actions