Actions
Bug #48640
closedqa: snapshot mismatch during mds thrashing
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
2020-12-17T00:01:48.575 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2020-12-17T00:01:48.576 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-12-17T00:01:48.576 INFO:teuthology.orchestra.run:waiting for 300 2020-12-17T00:01:48.594 INFO:tasks.ceph.mds.b:Stopped ... 2020-12-17T00:02:23.217 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=5220 2020-12-17T00:02:23.217 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.b 2020-12-17T00:02:23.325 INFO:tasks.workunit.client.0.smithi183.stdout:HEAD is now at a4c752df71 v0.6 2020-12-17T00:02:23.346 INFO:tasks.workunit.client.0.smithi183.stderr:+ mkdir .snap/v0.6 2020-12-17T00:02:23.366 INFO:tasks.workunit.client.0.smithi183.stderr:+ for v in $versions 2020-12-17T00:02:23.366 INFO:tasks.workunit.client.0.smithi183.stderr:+ ver=v0.7 2020-12-17T00:02:23.366 INFO:tasks.workunit.client.0.smithi183.stderr:+ echo v0.7 2020-12-17T00:02:23.367 INFO:tasks.workunit.client.0.smithi183.stderr:+ git reset --hard v0.7 ... 2020-12-17T00:02:58.017 INFO:tasks.workunit.client.0.smithi183.stdout:HEAD is now at 090436f56a v0.21 2020-12-17T00:02:58.063 INFO:tasks.workunit.client.0.smithi183.stderr:+ mkdir .snap/v0.21 2020-12-17T00:02:58.169 INFO:tasks.workunit.client.0.smithi183.stderr:+ for v in $versions 2020-12-17T00:02:58.169 INFO:tasks.workunit.client.0.smithi183.stderr:+ ver=v0.1 2020-12-17T00:02:58.170 INFO:tasks.workunit.client.0.smithi183.stderr:+ echo checking v0.1 2020-12-17T00:02:58.170 INFO:tasks.workunit.client.0.smithi183.stderr:+ cd .snap/v0.1 2020-12-17T00:02:58.170 INFO:tasks.workunit.client.0.smithi183.stdout:checking v0.1 2020-12-17T00:02:58.171 INFO:tasks.workunit.client.0.smithi183.stderr:+ git diff --exit-code 2020-12-17T00:03:04.945 DEBUG:teuthology.orchestra.run.smithi001:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-17T00:03:04.949 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-12-17T00:03:14.037 INFO:tasks.workunit.client.0.smithi183.stdout:diff --git a/src/ebofs/Cnode.h b/src/ebofs/Cnode.h 2020-12-17T00:03:14.037 INFO:tasks.workunit.client.0.smithi183.stdout:index 810124532c..e69de29bb2 100644 2020-12-17T00:03:14.038 INFO:tasks.workunit.client.0.smithi183.stdout:--- a/src/ebofs/Cnode.h 2020-12-17T00:03:14.038 INFO:tasks.workunit.client.0.smithi183.stdout:+++ b/src/ebofs/Cnode.h ... 2020-12-17T00:03:14.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-12-17T00:03:14.062 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2020-12-17T00:03:14.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2020-12-17T00:03:15.067 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 91, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 70, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201214.211711/qa/tasks/workunit.py", line 147, in task cleanup=cleanup) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201214.211711/qa/tasks/workunit.py", line 297, in _spawn_on_all_clients timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201214.211711/qa/tasks/workunit.py", line 425, in _run_tests label="workunit test {workunit}".format(workunit=workunit) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 215, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 183, in _raise_for_status node=self.hostname, label=self.label teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi183 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81d0278a02295c5a0d481303def7f69477d4ff55 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2020-12-17T00:03:15.157 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5e1183f827214bc9b23ceabe848260a9
From: /ceph/teuthology-archive/pdonnell-2020-12-16_23:01:45-fs-wip-pdonnell-testing-20201214.211711-distro-basic-smithi/5714203/teuthology.log
Actions