Bug #11548
closed"[ FAILED ] LibRadosTwoPoolsPP.Evict" in upgrade:giant-x-next-distro-basic-multi run
0%
Description
Run: http://pulpito-rdu.front.sepia.ceph.com/teuthology-2015-05-06_11:04:14-ceph-deploy-next-distro-basic-typica/
Jobs: ['11196', '11194', '11193', '11195']
Logs for one: http://typica002.front.sepia.ceph.com/teuthology-2015-05-06_11:04:14-ceph-deploy-next-distro-basic-typica/11194/
015-05-06T12:06:12.851 INFO:tasks.workunit.client.0.typica158.stdout:test/librados/tier.cc:792: Failure 2015-05-06T12:06:12.852 INFO:tasks.workunit.client.0.typica158.stdout:Value of: completion->get_return_value() 2015-05-06T12:06:12.852 INFO:tasks.workunit.client.0.typica158.stdout: Actual: 0 2015-05-06T12:06:12.853 INFO:tasks.workunit.client.0.typica158.stdout:Expected: -2 2015-05-06T12:06:19.045 INFO:tasks.workunit.client.0.typica158.stdout:[ FAILED ] LibRadosTwoPoolsPP.Evict (13252 ms)
Updated by Yuri Weinstein almost 9 years ago
Run: http://pulpito-rdu.front.sepia.ceph.com/teuthology-2015-05-07_16:00:02-ceph-deploy-next-distro-basic-typica/
Jobs: ['11463', '11464', '11466', '11465']
Updated by Zhiqiang Wang almost 9 years ago
In commit 1fa343f2360c6ef725bf6ec63337b379f4f28bfb, we made a change to return success for the evict op when the obs doesn't exist in the cache tier. And updated the corresponding testcases in test/librados/tier.cc in commit 41b132fc9ab1eeb39f919c56a284bb1bbf4212e3. Looks like this is running with the old testcases, that's why we see the failure.
Updated by Samuel Just almost 9 years ago
Yuri: this specific test case needs to be whitelisted in the upgrade tests.
Updated by Samuel Just almost 9 years ago
Or we could just remove it in firefly, giant, and hammer?
Updated by Yuri Weinstein almost 9 years ago
I replaced work units wit a new ceph-deploy/ceph-deploy_hello_world workunit (https://github.com/ceph/ceph/commit/13abae186357f4e9bb40990a7a212f93ec2e1e79)
and modified the ceph-deploy qa suite re:https://github.com/ceph/ceph-qa-suite/pull/436
So this can be resolved now.
Updated by Yuri Weinstein almost 9 years ago
- Priority changed from Urgent to Low
Updated by Yuri Weinstein almost 9 years ago
- Status changed from New to Won't Fix
Updated by Yuri Weinstein almost 9 years ago
- Subject changed from "[ FAILED ] LibRadosTwoPoolsPP.Evict" in ceph-deploy-next-distro-basic-typica run to "[ FAILED ] LibRadosTwoPoolsPP.Evict" in upgrade:giant-x-next-distro-basic-multi run
- Status changed from Won't Fix to New
- ceph-qa-suite upgrade/giant added
Reopened as same errors in a diff suite
Run: http://pulpito.ceph.com/teuthology-2015-05-10_17:05:03-upgrade:giant-x-next-distro-basic-multi/
Jobs: ['884221', '884222', '884223', '884224', '884225', '884226', '884227', '884228', '884230', '884232', '884233', '884235', '884237', '884241', '884243', '884245', '884247', '884249', '884251', '884253', '884259', '884261', '884263', '884264', '884270', '884272', '884274', '884276']
Logs for one: http://qa-proxy.ceph.com/teuthology/teuthology-2015-05-10_17:05:03-upgrade:giant-x-next-distro-basic-multi/884259/
2015-05-10T19:39:07.688 INFO:tasks.workunit.client.1.plana12.stdout:[ FAILED ] 2 tests, listed below:
2015-05-10T19:39:07.688 INFO:tasks.workunit.client.1.plana12.stdout:[ FAILED ] LibRadosTwoPoolsPP.Evict
2015-05-10T19:39:07.688 INFO:tasks.workunit.client.1.plana12.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict
2015-05-10T19:39:07.688 INFO:tasks.workunit.client.1.plana12.stdout:
2015-05-10T19:39:07.688 INFO:tasks.workunit.client.1.plana12.stdout: 2 FAILED TESTS
2015-05-10T19:39:07.689 INFO:tasks.workunit:Stopping ['rados/test.sh'] on client.1...
2015-05-10T19:39:07.690 INFO:teuthology.orchestra.run.plana12:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/workunit.client.1'
2015-05-10T19:39:07.698 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
for result in self:
File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
resurrect_traceback(result)
File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
return func(*args, **kwargs)
File "/var/lib/teuthworker/src/ceph-qa-suite_next/tasks/workunit.py", line 361, in _run_tests
label="workunit test {workunit}".format(workunit=workunit)
File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 156, in run
r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
r.wait()
File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
label=self.label)
CommandFailedError: Command failed (workunit test rados/test.sh) on plana12 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b29cc4c18e53515302dd3b21717f973e8f98487b TESTDIR="/home/ubuntu/cephtest" CEPH_ID="1" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.1/rados/test.sh'
Updated by Yuri Weinstein almost 9 years ago
Looks like it, but a different suite.
Updated by Zhiqiang Wang almost 9 years ago
Looks like still running with the old testcases
2015-05-10T19:32:02.755 INFO:tasks.workunit.client.1.plana12.stdout:test/librados/tier.cc:718: Failure
2015-05-10T19:32:02.756 INFO:tasks.workunit.client.1.plana12.stdout:Value of: completion->get_return_value()
2015-05-10T19:32:02.756 INFO:tasks.workunit.client.1.plana12.stdout: Actual: 0
2015-05-10T19:32:02.756 INFO:tasks.workunit.client.1.plana12.stdout:Expected: -2
Updated by Yuri Weinstein almost 9 years ago
On glance I don't see issues with tests (maybe bad cherry pick somewhere), but here are it is - https://github.com/ceph/ceph-qa-suite/tree/next/suites/upgrade/giant-x/parallel
@Zhiqiang - do you see anything wrong here? ^
Updated by Zhiqiang Wang almost 9 years ago
SOrry, Yuri, I'm not quite familiar with teuthology, so I'm not able to see things wrong...
Updated by Joao Eduardo Luis almost 9 years ago
FWIW, triggered this while running an upgrade suite on wip-11545. All 5 failures in the suite are related to this failed test.
teuthology:/a/joao-2015-05-14_11:32:34-upgrade:hammer-x-wip-11545---basic-multi/{892284,892285,892289,892301,892302}
Updated by Yuri Weinstein almost 9 years ago
- Priority changed from Low to High
Updated by Yuri Weinstein almost 9 years ago
- Priority changed from High to Urgent
Run: http://pulpito.ceph.com/teuthology-2015-05-21_17:05:01-upgrade:giant-x-next-distro-basic-vps/
Jobs: ~100
Logs for one: http://qa-proxy.ceph.com/teuthology/teuthology-2015-05-21_17:05:01-upgrade:giant-x-next-distro-basic-vps/903918/
2015-05-21T20:23:17.771 INFO:tasks.workunit.client.1.vpm156.stdout:[ FAILED ] 2 tests, listed below: 2015-05-21T20:23:17.771 INFO:tasks.workunit.client.1.vpm156.stdout:[ FAILED ] LibRadosTwoPoolsPP.Evict 2015-05-21T20:23:17.772 INFO:tasks.workunit.client.1.vpm156.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict
Updated by Yuri Weinstein almost 9 years ago
Updated by Sage Weil almost 9 years ago
- Assignee set to Yuri Weinstein
same issue as before.. this test needs to be disabled.
Updated by Yuri Weinstein almost 9 years ago
- Assignee changed from Yuri Weinstein to Josh Durgin
- Release set to next
- ceph-qa-suite upgrade/firefly-x, upgrade/giant-x added
- ceph-qa-suite deleted (
ceph-deploy, upgrade/giant)
Updated by Josh Durgin almost 9 years ago
- Status changed from New to Resolved
c0e3f8f79c6d985a13776898f43664495920d9b0 in ceph-qa-suite and 348a3d3c9880e7d022e71a2faafe51c8f771406e + backports thereof in ceph.git.