Actions
Bug #43129
openqa: `fs dump` fails during snaptests
Status:
New
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
nautilus,mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
qa-suite
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
019-11-28T06:43:16.907 INFO:tasks.workunit.client.0.smithi156.stdout:coreutils-8.5/m4/wcrtomb.m4 2019-11-28T06:43:16.988 INFO:tasks.workunit.client.0.smithi156.stdout:coreutils-8.5/m4/putenv.m4 2019-11-28T06:43:16.995 INFO:tasks.workunit.client.0.smithi156.stdout:coreutils-8.5/m4/lstat.m4 2019-11-28T06:43:20.541 DEBUG:teuthology.orchestra.run:got remote process result: 124 2019-11-28T06:43:20.541 ERROR:tasks.mds_thrash.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20191127.175901/qa/tasks/mds_thrash.py", line 124, in _run self.do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20191127.175901/qa/tasks/mds_thrash.py", line 327, in do_thrash status = self.fs.status() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20191127.175901/qa/tasks/cephfs/filesystem.py", line 306, in status return FSStatus(self.mon_manager) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20191127.175901/qa/tasks/cephfs/filesystem.py", line 42, in __init__ self.map = json.loads(self.mon.raw_cluster_cmd("fs", "dump", "--format=json")) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20191127.175901/qa/tasks/ceph_manager.py", line 1272, in raw_cluster_cmd stdout=StringIO(), File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 433, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi156 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json' ... 2019-11-28T06:43:23.614 INFO:tasks.workunit.client.0.smithi156.stdout:coreutils-8.5/man/stat.x 2019-11-28T06:43:26.946 INFO:tasks.daemonwatchdog.daemon_watchdog:thrasher.fs.[cephfs] failed 2019-11-28T06:43:26.946 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-11-28T06:43:26.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2019-11-28T06:43:26.947 INFO:teuthology.orchestra.run.smithi156:> stat --file-system '--printf=%T 2019-11-28T06:43:26.947 INFO:teuthology.orchestra.run.smithi156:> ' -- /home/ubuntu/cephtest/mnt.0 2019-11-28T06:43:26.960 INFO:teuthology.orchestra.run.smithi156.stdout:fuseblk 2019-11-28T06:43:26.961 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2019-11-28T06:43:26.961 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi156.front.sepia.ceph.com... 2019-11-28T06:43:26.961 INFO:teuthology.orchestra.run:Running command with timeout 1800 2019-11-28T06:43:26.961 INFO:teuthology.orchestra.run.smithi156:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2019-11-28T06:43:27.056 INFO:tasks.workunit.client.0.smithi156.stderr:tar: coreutils-8.5/man/stat.x: Cannot open: Software caused connection abort 2019-11-28T06:43:27.057 INFO:tasks.workunit.client.0.smithi156.stderr:tar: coreutils-8.5/man/sleep.x: Cannot open: Transport endpoint is not connected 2019-11-28T06:43:27.057 INFO:tasks.workunit.client.0.smithi156.stderr:tar: coreutils-8.5/man/printenv.1: Cannot open: Transport endpoint is not connected 2019-11-28T06:43:27.057 INFO:tasks.workunit.client.0.smithi156.stderr:tar: coreutils-8.5/man/tr.x: Cannot open: Transport endpoint is not connected ...
From: /ceph/teuthology-archive/pdonnell-2019-11-27_23:31:21-fs-wip-pdonnell-testing-20191127.175901-distro-basic-smithi/4548047/teuthology.log
See also job 4548113
Actions