Bug #11784
closedceph-fuse hang on unmount (stuck dentry refs)
0%
Description
The next step after the last "ceph-fuse not mounted, got fs type 'ext2/ext3'" log message would have been the self.fuse_daemon.wait in FuseMount.umount_wait, so this may be a case where the fuse mount is tearing down, but the ceph-fuse process is failing to terminate for some reason.
Need to watch for another case of this happening to get more information.
Updated by John Spray almost 9 years ago
- Project changed from Ceph to CephFS
- Category set to 45
Updated by Greg Farnum almost 9 years ago
John, is this likely to have been a dup of #11294? We can tell by checking out the ceph-fuse log if it's still available (although probably not on this one given the lag time).
Updated by Greg Farnum almost 9 years ago
- Subject changed from ceph-fuse hang on unmount after smbtorture workload to ceph-fuse hang on unmount (stuck dentry refs)
- Priority changed from Normal to High
We saw this again today, and it's definitely inode refs this time:
http://pulpito-rdu.front.sepia.ceph.com/gregf-2015-05-31_20:59:54-fs-greg-fs-testing---basic-typica/1145/
Unlike previously, that's not a samba run! :(
Updated by Greg Farnum almost 9 years ago
I copied the ceph-client log into that folder, although we're missing the server logs.
Updated by Zheng Yan almost 9 years ago
2015-06-01T06:50:15.928 INFO:teuthology.orchestra.run.typica012.stderr:fusermount: failed to unmount /home/ubuntu/cephtest/mnt.0: Device or resource busy 2015-06-01T06:50:15.930 INFO:tasks.cephfs.fuse_mount:Failed to unmount ceph-fuse on ubuntu@typica012.front.sepia.ceph.com, aborting...
except run.CommandFailedError: log.info('Failed to unmount ceph-fuse on {name}, aborting...'.format(name=self.client_remote.name)) # abort the fuse mount, killing all hung processes if self._fuse_conn: self.run_python(dedent(""" import os path = "/sys/fs/fuse/connections/{0}/abort" if os.path.exists(path): open(path, "w").write("1") """).format(self._fuse_conn)) self._fuse_conn = None stderr = StringIO()
aborting fuse can explian the ll_ref leaking
Updated by Greg Farnum almost 9 years ago
Hmm, there shouldn't have been any activity on the mount by this point. Maybe we've got some other kind of bug, though.
Updated by Sage Weil over 8 years ago
- Status changed from Need More Info to Can't reproduce