Actions
Bug #16275
closed"failed to recover before timeout expired" in powercycle-jewel-testing-basic-smithi
Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
powercycle
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Verified by two runs:
http://pulpito.ceph.com/teuthology-2016-06-13_09:48:57-powercycle-jewel-testing-basic-mira/
job: 256858
http://pulpito.ceph.com/teuthology-2016-06-13_13:58:29-powercycle-jewel-testing-basic-smithi/
job: 256948
2016-06-13T16:32:43.894 INFO:teuthology.orchestra.run.smithi056:Running: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' 2016-06-13T16:32:43.997 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2016-06-13T16:32:43.997 INFO:teuthology.orchestra.run.smithi056:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/workunit.client.0 /home/ubuntu/cephtest/clone' 2016-06-13T16:32:44.318 DEBUG:teuthology.parallel:result is None 2016-06-13T16:32:44.318 INFO:teuthology.orchestra.run.smithi056:Running: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0' 2016-06-13T16:32:44.347 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2016-06-13T16:32:44.347 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2016-06-13T16:32:46.073 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2016-06-13T16:32:46.073 INFO:teuthology.orchestra.run.smithi056:Running: "stat --file-system '--printf=%T\n' -- /home/ubuntu/cephtest/mnt.0" 2016-06-13T16:32:46.086 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2016-06-13T16:32:46.087 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi056.front.sepia.ceph.com... 2016-06-13T16:32:46.087 INFO:teuthology.orchestra.run.smithi056:Running: 'sudo fusermount -u /home/ubuntu/cephtest/mnt.0' 2016-06-13T16:32:46.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi056.stderr:ceph-fuse[14972]: fuse finished with error 0 and tester_r 0 2016-06-13T16:32:46.211 INFO:teuthology.orchestra.run.smithi056:Running: "stat --file-system '--printf=%T\n' -- /home/ubuntu/cephtest/mnt.0" 2016-06-13T16:32:46.222 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2016-06-13T16:32:46.222 INFO:teuthology.orchestra.run:waiting for 900 2016-06-13T16:32:52.224 INFO:teuthology.orchestra.run.smithi056:Running: 'rmdir -- /home/ubuntu/cephtest/mnt.0' 2016-06-13T16:32:52.235 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2016-06-13T16:32:52.273 INFO:tasks.thrashosds:joining thrashosds 2016-06-13T16:32:52.273 ERROR:teuthology.run_tasks:Manager failed: thrashosds Traceback (most recent call last): File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 139, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ self.gen.next() File "/var/lib/teuthworker/src/ceph-qa-suite_jewel/tasks/thrashosds.py", line 184, in task thrash_proc.do_join() File "/var/lib/teuthworker/src/ceph-qa-suite_jewel/tasks/ceph_manager.py", line 431, in do_join self.thread.get() File "/home/teuthworker/src/teuthology_master/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 454, in get self._raise_exception() File "/home/teuthworker/src/teuthology_master/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 171, in _raise_exception reraise(*self.exc_info) File "/home/teuthworker/src/teuthology_master/virtualenv/local/lib/python2.7/site-packages/gevent/greenlet.py", line 534, in run result = self._run(*self.args, **self.kwargs) File "/var/lib/teuthworker/src/ceph-qa-suite_jewel/tasks/ceph_manager.py", line 646, in wrapper return func(self) File "/var/lib/teuthworker/src/ceph-qa-suite_jewel/tasks/ceph_manager.py", line 693, in do_thrash timeout=self.config.get('timeout') File "/var/lib/teuthworker/src/ceph-qa-suite_jewel/tasks/ceph_manager.py", line 1673, in wait_for_recovery 'failed to recover before timeout expired' AssertionError: failed to recover before timeout expired
Updated by Yuri Weinstein almost 8 years ago
- Is duplicate of Bug #16277: ReplicatedBackend doesn't increment stats on pull, only push added
Actions