Actions
Bug #11993
closedLibRadosTwoPoolsPP * tests failed in upgrade:giant-x-next-distro-basic-vps run
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
upgrade/dumpling-firefly-x, upgrade/firefly-x, upgrade/giant-x
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
NOTE: Could be a duplicate if #11548
Run: http://pulpito.ceph.com/teuthology-2015-06-11_17:05:04-upgrade:giant-x-next-distro-basic-vps/
Jobs: 40 jobs
Logs for one: http://qa-proxy.ceph.com/teuthology/teuthology-2015-06-11_17:05:04-upgrade:giant-x-next-distro-basic-vps/929546/
2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2667: Failure 2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:11.964 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:12.212 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (1286 ms) 2015-06-11T18:20:12.212 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout 2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2708: Failure 2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:12.221 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:12.269 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Whiteout (57 ms) 2015-06-11T18:20:12.270 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict 2015-06-11T18:20:12.293 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2771: Failure 2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:12.337 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict (68 ms) 2015-06-11T18:20:12.337 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2015-06-11T18:20:13.186 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2015-06-11T18:20:13.187 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2015-06-11T18:20:13.187 INFO:tasks.ceph.mon.b:Restarting daemon 2015-06-11T18:20:13.187 INFO:teuthology.orchestra.run.vpm015:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f -i b' 2015-06-11T18:20:13.191 INFO:tasks.ceph.mon.b:Started 2015-06-11T18:20:13.191 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2015-06-11T18:20:13.191 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph quorum_status' 2015-06-11T18:20:13.294 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2914: Failure 2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.340 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.EvictSnap (1003 ms) 2015-06-11T18:20:13.340 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush 2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3087: Failure 2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.381 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlush (41 ms) 2015-06-11T18:20:13.381 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush 2015-06-11T18:20:13.383 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3195: Failure 2015-06-11T18:20:13.383 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.384 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.384 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.415 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Flush (34 ms) 2015-06-11T18:20:13.415 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2015-06-11T18:20:13.416 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:13.417252 7f441b305700 -1 asok(0x7f4414000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14114.asok': (13) Permission denied 2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3355: Failure 2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.418 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.454 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushSnap (39 ms) 2015-06-11T18:20:13.455 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2015-06-11T18:20:13.458 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3665: Failure 2015-06-11T18:20:13.458 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.459 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.459 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.499 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (45 ms) 2015-06-11T18:20:13.499 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2015-06-11T18:20:13.501 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3831: Failure 2015-06-11T18:20:13.501 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.502 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.502 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.564 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlushReadRace (65 ms) 2015-06-11T18:20:13.564 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3913: Failure 2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.607 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetRead (43 ms) 2015-06-11T18:20:13.607 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:4054: Failure 2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.611 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.628 INFO:tasks.ceph.mon.b.vpm015.stdout:starting mon.b rank 0 at 10.214.130.15:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 20c38c67-62bd-4624-ada0-d42475a70b2e 2015-06-11T18:20:13.639 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":98,"quorum":[1,2],"quorum_names":["a","c"],"quorum_leader_name":"a","monmap":{"epoch":1,"fsid":"20c38c67-62bd-4624-ada0-d42475a70b2e","modified":"2015-06-11 20:29:44.513983","created":"2015-06-11 20:29:44.513983","mons":[{"rank":0,"name":"b","addr":"10.214.130.15:6789\/0"},{"rank":1,"name":"a","addr":"10.214.130.68:6789\/0"},{"rank":2,"name":"c","addr":"10.214.130.15:6790\/0"}]}} 2015-06-11T18:20:13.657 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetTrim (50 ms) 2015-06-11T18:20:13.658 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:4133: Failure 2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null) 2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39 2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0 2015-06-11T18:20:13.722 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (65 ms) 2015-06-11T18:20:16.638 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph quorum_status' 2015-06-11T18:20:16.819 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:16.821660 7f9e6f382700 -1 asok(0x7f9e68000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14201.asok': (13) Permission denied 2015-06-11T18:20:17.101 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":100,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_leader_name":"b","monmap":{"epoch":1,"fsid":"20c38c67-62bd-4624-ada0-d42475a70b2e","modified":"2015-06-11 20:29:44.513983","created":"2015-06-11 20:29:44.513983","mons":[{"rank":0,"name":"b","addr":"10.214.130.15:6789\/0"},{"rank":1,"name":"a","addr":"10.214.130.68:6789\/0"},{"rank":2,"name":"c","addr":"10.214.130.15:6790\/0"}]}} 2015-06-11T18:20:17.102 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2015-06-11T18:20:17.102 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.68:6789 mon_status' 2015-06-11T18:20:17.281 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:17.283914 7fbbb8db8700 -1 asok(0x7fbbb4000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14253.asok': (13) Permission denied 2015-06-11T18:20:17.458 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.15:6789 mon_status' 2015-06-11T18:20:17.649 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:17.651313 7f2d21f95700 -1 asok(0x7f2d1c000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14307.asok': (13) Permission denied 2015-06-11T18:20:17.894 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.15:6790 mon_status' 2015-06-11T18:20:17.924 INFO:tasks.workunit.client.1.vpm128.stdout:[----------] 15 tests from LibRadosTwoPoolsECPP (74622 ms total) 2015-06-11T18:20:17.924 INFO:tasks.workunit.client.1.vpm128.stdout: 2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[----------] Global test environment tear-down 2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[==========] 39 tests from 4 test cases ran. (225153 ms total) 2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[ PASSED ] 12 tests. 2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] 27 tests, listed below: 2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteSnapScrub 2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Whiteout 2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Evict 2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.EvictSnap 2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.EvictSnap2 2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.TryFlush 2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Flush 2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.FlushSnap 2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.TryFlushReadRace 2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetRead 2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetWrite 2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetTrim 2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Whiteout 2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict 2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.EvictSnap 2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlush 2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Flush 2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushSnap 2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlushReadRace 2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetRead 2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetTrim 2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout: 2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:27 FAILED TESTS
Actions