Actions
Bug #11499
closedceph-fuse: don't try and remount during shutdown
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
Correctness/Safety
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, ceph-fuse
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2015-04-27T15:57:19.079 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@typica151.front.sepia.ceph.com...
2015-04-27T15:57:19.079 INFO:teuthology.orchestra.run.typica151:Running: 'sudo fusermount -u /home/ubuntu/cephtest/mnt.0'
2015-04-27T15:57:19.169 INFO:teuthology.orchestra.run.typica151.stderr:2015-04-27 15:57:19.166129 7f0adf49d700 1 -- 172.20.133.151:0/1019868 --> 172.20.133.164:6789/0 -- mon_command({"prefix": "mds dump", "format": "json"} v 0) v1 -- ?+0 0x7f0ad002a9c0 con 0x7f0ad0026aa0
2015-04-27T15:57:19.169 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:mount: /home/ubuntu/cephtest/mnt.0 not mounted or bad option
2015-04-27T15:57:19.170 INFO:teuthology.orchestra.run.typica151.stderr:2015-04-27 15:57:19.166911 7f0acffff700 1 -- 172.20.133.151:0/1019868 <== mon.1 172.20.133.164:6789/0 12 ==== mon_command_ack([{"prefix": "mds dump", "format": "json"}]=0 dumped mdsmap epoch 56 v56) v1 ==== 96+0+905 (3493472965 0 201683057) 0x7f0ac40009b0 con 0x7f0ad0026aa0
2015-04-27T15:57:19.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:2015-04-27 15:57:19.167342 7fd053fff700 -1 client.4136 tried to remount (to trim kernel dentries) and got error 32
2015-04-27T15:57:19.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:client/Client.cc: In function 'virtual void C_Client_Remount::finish(int)' thread 7fd053fff700 time 2015-04-27 15:57:19.167399
2015-04-27T15:57:19.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:client/Client.cc: 3457: FAILED assert(0 == "failed to remount for kernel dentry trimming")
2015-04-27T15:57:19.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: ceph version 0.94.1-8-g74c2dc1 (74c2dc1f3924fa05e2c40f4cceb2ab060493bdfb)
2015-04-27T15:57:19.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x70e81b]
2015-04-27T15:57:19.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 2: (C_Client_Remount::finish(int)+0xb5) [0x5ac2f5]
2015-04-27T15:57:19.174 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 3: (Context::complete(int)+0x9) [0x5aad59]
2015-04-27T15:57:19.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 4: (Finisher::finisher_thread_entry()+0x158) [0x633428]
2015-04-27T15:57:19.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 5: (()+0x8182) [0x7fd07a7d0182]
2015-04-27T15:57:19.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 6: (clone()+0x6d) [0x7fd07913a47d]
2015-04-27T15:57:19.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2015-04-27T15:57:19.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:2015-04-27 15:57:19.168551 7fd053fff700 -1 client/Client.cc: In function 'virtual void C_Client_Remount::finish(int)' thread 7fd053fff700 time 2015-04-27 15:57:19.167399
2015-04-27T15:57:19.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:client/Client.cc: 3457: FAILED assert(0 == "failed to remount for kernel dentry trimming")
2015-04-27T15:57:19.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:
2015-04-27T15:57:19.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: ceph version 0.94.1-8-g74c2dc1 (74c2dc1f3924fa05e2c40f4cceb2ab060493bdfb)
2015-04-27T15:57:19.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x70e81b]
2015-04-27T15:57:19.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 2: (C_Client_Remount::finish(int)+0xb5) [0x5ac2f5]
2015-04-27T15:57:19.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 3: (Context::complete(int)+0x9) [0x5aad59]
2015-04-27T15:57:19.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 4: (Finisher::finisher_thread_entry()+0x158) [0x633428]
2015-04-27T15:57:19.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 5: (()+0x8182) [0x7fd07a7d0182]
2015-04-27T15:57:19.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: 6: (clone()+0x6d) [0x7fd07913a47d]
2015-04-27T15:57:19.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2015-04-27T15:57:19.179 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.typica151.stderr:
I think maybe this we can fix this crash by checking whether we're unmounting during the remount callback? If we are then obviously we don't want to try and remount.
Updated by Zheng Yan almost 9 years ago
the test is already there
if (client->require_remount && !client->unmounting) {
assert(0 == "failed to remount for kernel dentry trimming");
}
Updated by Greg Farnum almost 9 years ago
I thought that solution seemed familiar. I guess there's still a race between when the unmount reaches the Client versus when the OS knows about it...blech. :/
Updated by Greg Farnum almost 8 years ago
- Category changed from 45 to Correctness/Safety
- Status changed from New to Can't reproduce
- Component(FS) Client, ceph-fuse added
We haven't seen this again.
Actions