Project

General

Profile

Bug #49845

qa: failed umount in test_volumes

Added by Patrick Donnelly 9 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
pacific,octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
kceph, mgr/volumes
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2021-03-15T23:52:36.715 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo dd if=/dev/urandom of=rogue.bin bs=1M conv=fdatasync count=1 seek=0)
2021-03-15T23:52:38.101 DEBUG:teuthology.orchestra.run:got remote process result: 137
2021-03-15T23:52:38.102 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: 164880 Killed                  ( cd /home/ubuntu/cephtest/mnt.0 && exec sudo dd if=/dev/urandom of=rogue.bin bs=1M conv=fdatasync count=1 seek=0 )
2021-03-15T23:52:38.102 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.guest...
2021-03-15T23:52:38.103 INFO:teuthology.orchestra.run:Running command with timeout 900
2021-03-15T23:52:38.103 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0
2021-03-15T23:52:38.142 INFO:teuthology.orchestra.run.smithi094.stderr:umount: /home/ubuntu/cephtest/mnt.0: target is busy.
2021-03-15T23:52:38.144 DEBUG:teuthology.orchestra.run:got remote process result: 32
...
2021-03-15T23:53:23.812 INFO:tasks.cephfs_test_runner:test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes) ... ERROR
2021-03-15T23:53:23.813 INFO:tasks.cephfs_test_runner:ERROR
2021-03-15T23:53:23.813 INFO:tasks.cephfs_test_runner:
2021-03-15T23:53:23.814 INFO:tasks.cephfs_test_runner:======================================================================
2021-03-15T23:53:23.814 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_evict_client (tasks.cephfs.test_volumes.TestSubvolumes)
2021-03-15T23:53:23.814 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2021-03-15T23:53:23.815 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2021-03-15T23:53:23.815 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b47b2982a9ebb626e52184161131f523dae60107/qa/tasks/cephfs/test_volumes.py", line 1908, in test_subvolume_evict_client
2021-03-15T23:53:23.815 INFO:tasks.cephfs_test_runner:    guest_mounts[0].umount_wait()
2021-03-15T23:53:23.816 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b47b2982a9ebb626e52184161131f523dae60107/qa/tasks/cephfs/kernel_mount.py", line 126, in umount_wait
2021-03-15T23:53:23.816 INFO:tasks.cephfs_test_runner:    self.umount(force)
2021-03-15T23:53:23.816 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b47b2982a9ebb626e52184161131f523dae60107/qa/tasks/cephfs/kernel_mount.py", line 112, in umount
2021-03-15T23:53:23.816 INFO:tasks.cephfs_test_runner:    raise e
2021-03-15T23:53:23.817 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_b47b2982a9ebb626e52184161131f523dae60107/qa/tasks/cephfs/kernel_mount.py", line 106, in umount
2021-03-15T23:53:23.817 INFO:tasks.cephfs_test_runner:    self.client_remote.run(args=cmd, timeout=(15*60), omit_sudo=False)
2021-03-15T23:53:23.817 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/remote.py", line 215, in run
2021-03-15T23:53:23.817 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2021-03-15T23:53:23.818 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 455, in run
2021-03-15T23:53:23.818 INFO:tasks.cephfs_test_runner:    r.wait()
2021-03-15T23:53:23.818 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 161, in wait
2021-03-15T23:53:23.819 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2021-03-15T23:53:23.819 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b96569170f15eae4604f361990ea65737b28dff1/teuthology/orchestra/run.py", line 183, in _raise_for_status
2021-03-15T23:53:23.819 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2021-03-15T23:53:23.820 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 32: 'sudo umount /home/ubuntu/cephtest/mnt.0'

From: /ceph/teuthology-archive/pdonnell-2021-03-15_22:16:56-fs-wip-pdonnell-testing-20210315.182203-distro-basic-smithi/5968037/teuthology.log

busy mount, possibly related to recent changes in testing kernel.

History

#1 Updated by Jeff Layton 8 months ago

  • Status changed from New to Resolved

The client kernel in this test had a bad patch in it that has since been fixed. See:

https://tracker.ceph.com/issues/49843#note-2

Let's chalk this up to that bug for now, and we can reopen this if it happens again.

#2 Updated by Patrick Donnelly 6 months ago

Back: /ceph/teuthology-archive/pdonnell-2021-05-20_14:09:54-fs-wip-pdonnell-testing-20210518.214114-distro-basic-smithi/6124921/teuthology.log

Also available in: Atom PDF