Project

General

Profile

Bug #40093

qa: client mount cannot be forcibly unmounted when all MDS are down

Added by Patrick Donnelly 5 months ago. Updated 4 months ago.

Status:
New
Priority:
Urgent
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
nautilus,mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
qa-suite
Labels (FS):
qa
Pull request ID:
Crash signature:

Description

2019-03-31T14:56:10.763 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@smithi118.front.sepia.ceph.com...
2019-03-31T14:56:39.121 INFO:teuthology.orchestra.run.smithi118:Running:
2019-03-31T14:56:39.122 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:56:39.126 INFO:teuthology.orchestra.run.smithi142:Running:
2019-03-31T14:56:39.127 INFO:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:57:09.260 INFO:teuthology.orchestra.run.smithi118:Running:
2019-03-31T14:57:09.261 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:57:09.266 INFO:teuthology.orchestra.run.smithi142:Running:
2019-03-31T14:57:09.266 INFO:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:57:10.777 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: <Greenlet at 0x7f7514fe47d0: copy_file_to(<paramiko.ChannelFile from <paramiko.Channel 631 (, <logging.Logger object at 0x7f751564c390>, None)>
2019-03-31T14:57:39.415 INFO:teuthology.orchestra.run.smithi118:Running:
2019-03-31T14:57:39.415 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:57:39.420 INFO:teuthology.orchestra.run.smithi142:Running:
2019-03-31T14:57:39.420 INFO:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:58:09.548 INFO:teuthology.orchestra.run.smithi118:Running:
2019-03-31T14:58:09.549 INFO:teuthology.orchestra.run.smithi118:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:58:09.553 INFO:teuthology.orchestra.run.smithi142:Running:
2019-03-31T14:58:09.553 INFO:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2019-03-31T14:58:10.778 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: <Greenlet at 0x7f7514fe4c30: copy_file_to(<paramiko.ChannelFile from <paramiko.Channel 631 (, <logging.Logger object at 0x7f751564c0d0>, None)>
2019-03-31T14:58:10.779 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection
2019-03-31T14:58:10.779 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi118.front.sepia.ceph.com...
2019-03-31T14:58:10.779 INFO:teuthology.orchestra.run:Running command with timeout 1800
2019-03-31T14:58:10.779 INFO:teuthology.orchestra.run.smithi118:Running:
2019-03-31T14:58:10.780 INFO:teuthology.orchestra.run.smithi118:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0
2019-03-31T14:58:10.882 DEBUG:teuthology.orchestra.run:got remote process result: 1
2019-03-31T14:58:10.901 INFO:teuthology.orchestra.run.smithi118.stderr:fusermount: failed to unmount /home/ubuntu/cephtest/mnt.0: Device or resource busy
2019-03-31T14:58:10.901 INFO:tasks.cephfs.fuse_mount:Failed to unmount ceph-fuse on ubuntu@smithi118.front.sepia.ceph.com, aborting...

From: /ceph/teuthology-archive/teuthology-2019-03-29_05:10:02-fs-mimic-distro-basic-smithi/3783145/teuthology.log.gz

Also: http://pulpito.ceph.com/yuriw-2019-05-16_23:40:20-fs-mimic_v13.2.6_QE-distro-basic-smithi/3960510/
Also: http://pulpito.ceph.com/yuriw-2019-05-18_16:55:40-fs-mimic_v13.2.6_QE-distro-basic-smithi/3966546/

^ Those two QE runs indicate it's easy to reproduce.

History

#1 Updated by Patrick Donnelly 4 months ago

  • Subject changed from mimic: qa: client mount cannot be forcibly unmounted when all MDS are down to qa: client mount cannot be forcibly unmounted when all MDS are down
  • Assignee set to Patrick Donnelly
  • Priority changed from High to Urgent
  • Target version set to v15.0.0
  • Start date deleted (05/30/2019)
  • Backport set to nautilus,mimic

/ceph/teuthology-archive/pdonnell-2019-06-11_01:05:56-fs-wip-pdonnell-testing-20190610.220401-distro-basic-smithi/4020224/teuthology.log

in master too.

Also available in: Atom PDF