Actions
Bug #19540
closed.../rados/test.sh: line 9: 29593 Terminated (times out)
Status:
Duplicate
Priority:
Immediate
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2017-04-06T21:06:51.961 INFO:tasks.workunit.client.0.smithi104.stderr:++ cleanup 2017-04-06T21:06:51.962 INFO:tasks.workunit.client.0.smithi104.stderr:++ pkill -P 29591 2017-04-06T21:06:51.970 INFO:tasks.workunit.client.0.smithi104.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh: line 9: 29593 Terminated bash -o pipefail -exc "ceph_test_rados_$f $color 2>&1 | tee ceph_test_rados_$f.log | sed \"s/^/$r: /\"" 2017-04-06T21:06:51.970 INFO:tasks.workunit.client.0.smithi104.stderr:++ true 2017-04-06T21:06:51.971 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.0... 2017-04-06T21:06:51.971 INFO:teuthology.orchestra.run.smithi104:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0' 2017-04-06T21:06:52.098 ERROR:teuthology.parallel:Exception in parallel execution Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 83, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 101, in next resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 19, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-mgr-init/qa/tasks/workunit.py", line 450, in _run_tests label="workunit test {workunit}".format(workunit=workunit) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 414, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 149, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 171, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed (workunit test rados/test.sh) on smithi104 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=880858be35b5b586cab1a9ca5605828c0c6de840 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh'
/a/sage-2017-04-06_17:19:30-rados:thrash-wip-mgr-init---basic-smithi/993669
I keep seeing these on my various runs and I can't find an actual failure in the test script.
Updated by Sage Weil about 7 years ago
/a/sage-2017-04-06_19:43:41-rados-wip-sage-testing---basic-smithi/994042
Updated by Sage Weil about 7 years ago
- Subject changed from .../rados/test.sh: line 9: 29593 Terminated to .../rados/test.sh: line 9: 29593 Terminated (times out)
- Status changed from New to 12
/a/sage-2017-04-07_04:46:46-rados:thrash-wip-bluestore-osr-drain-hang---basic-smithi/995352
it's api_aio just being slow
2017-04-07T06:28:22.210 INFO:tasks.workunit.client.0.smithi099.stderr:+ wait 16559 ...3 hours... 2017-04-07T09:13:34.429 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: Running main() from gmock_main.cc ... 2017-04-07T09:13:34.435 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync ...2 minutes... 2017-04-07T09:15:11.877 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3587 ms) ... 2017-04-07T09:15:11.885 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: ...3 minutes... 2017-04-07T09:18:57.090 INFO:tasks.workunit.client.0.smithi099.stderr:++ cleanup 2017-04-07T09:18:57.090 INFO:tasks.workunit.client.0.smithi099.stderr:++ pkill -P 16557 2017-04-07T09:18:57.099 INFO:tasks.workunit.client.0.smithi099.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh: line 9: 16559 Terminated bash -o pipefail -exc "ceph_test_rados_$f $color 2>&1 | tee ceph_test_rados_$f.log | sed \"s/^/$r: /\"" 2017-04-07T09:18:57.100 INFO:tasks.workunit.client.0.smithi099.stderr:++ true
and it times out.
Updated by Sage Weil about 7 years ago
- Status changed from 12 to Duplicate
2017-04-07T09:13:34.430 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: [ RUN ] LibRadosAio.PoolQuotaPP 2017-04-07T09:13:34.431 INFO:tasks.workunit.client.0.smithi099.stdout: api_aio: [ OK ] LibRadosAio.PoolQuotaPP (10377302 ms)
this is a side-effect of #19430.
Updated by Sage Weil about 7 years ago
- Is duplicate of Bug #19430: objecter: full_try behavior not consistent with osd added
Actions