Bug #11993
Updated by Yuri Weinstein almost 9 years ago
NOTE: Could be a duplicate if #11548
Run: http://pulpito.ceph.com/teuthology-2015-06-11_17:05:04-upgrade:giant-x-next-distro-basic-vps/
Jobs: 40 jobs
Logs for one: http://qa-proxy.ceph.com/teuthology/teuthology-2015-06-11_17:05:04-upgrade:giant-x-next-distro-basic-vps/929546/
<pre>
2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2667: Failure
2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:11.963 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:11.964 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:12.212 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (1286 ms)
2015-06-11T18:20:12.212 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Whiteout
2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2708: Failure
2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:12.220 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:12.221 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:12.269 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Whiteout (57 ms)
2015-06-11T18:20:12.270 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Evict
2015-06-11T18:20:12.293 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2771: Failure
2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:12.294 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:12.337 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict (68 ms)
2015-06-11T18:20:12.337 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.EvictSnap
2015-06-11T18:20:13.186 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b
2015-06-11T18:20:13.187 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b
2015-06-11T18:20:13.187 INFO:tasks.ceph.mon.b:Restarting daemon
2015-06-11T18:20:13.187 INFO:teuthology.orchestra.run.vpm015:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f -i b'
2015-06-11T18:20:13.191 INFO:tasks.ceph.mon.b:Started
2015-06-11T18:20:13.191 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3
2015-06-11T18:20:13.191 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph quorum_status'
2015-06-11T18:20:13.294 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:2914: Failure
2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.295 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.340 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.EvictSnap (1003 ms)
2015-06-11T18:20:13.340 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlush
2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3087: Failure
2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.343 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.381 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlush (41 ms)
2015-06-11T18:20:13.381 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.Flush
2015-06-11T18:20:13.383 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3195: Failure
2015-06-11T18:20:13.383 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.384 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.384 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.415 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Flush (34 ms)
2015-06-11T18:20:13.415 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushSnap
2015-06-11T18:20:13.416 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:13.417252 7f441b305700 -1 asok(0x7f4414000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14114.asok': (13) Permission denied
2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3355: Failure
2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.417 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.418 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.454 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushSnap (39 ms)
2015-06-11T18:20:13.455 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces
2015-06-11T18:20:13.458 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3665: Failure
2015-06-11T18:20:13.458 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.459 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.459 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.499 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (45 ms)
2015-06-11T18:20:13.499 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace
2015-06-11T18:20:13.501 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3831: Failure
2015-06-11T18:20:13.501 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.502 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.502 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.564 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlushReadRace (65 ms)
2015-06-11T18:20:13.564 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetRead
2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:3913: Failure
2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.567 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.607 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetRead (43 ms)
2015-06-11T18:20:13.607 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.HitSetTrim
2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:4054: Failure
2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.610 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.611 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.628 INFO:tasks.ceph.mon.b.vpm015.stdout:starting mon.b rank 0 at 10.214.130.15:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 20c38c67-62bd-4624-ada0-d42475a70b2e
2015-06-11T18:20:13.639 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":98,"quorum":[1,2],"quorum_names":["a","c"],"quorum_leader_name":"a","monmap":{"epoch":1,"fsid":"20c38c67-62bd-4624-ada0-d42475a70b2e","modified":"2015-06-11 20:29:44.513983","created":"2015-06-11 20:29:44.513983","mons":[{"rank":0,"name":"b","addr":"10.214.130.15:6789\/0"},{"rank":1,"name":"a","addr":"10.214.130.68:6789\/0"},{"rank":2,"name":"c","addr":"10.214.130.15:6790\/0"}]}}
2015-06-11T18:20:13.657 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetTrim (50 ms)
2015-06-11T18:20:13.658 INFO:tasks.workunit.client.1.vpm128.stdout:[ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead
2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:test/librados/tier.cc:4133: Failure
2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:Value of: cluster.mon_command( "{\"prefix\": \"osd tier add\", \"pool\": \"" + pool_name + "\", \"tierpool\": \"" + cache_pool_name + "\", \"force_nonempty\": \"--force-nonempty\" }", inbl, __null, __null)
2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout: Actual: -39
2015-06-11T18:20:13.673 INFO:tasks.workunit.client.1.vpm128.stdout:Expected: 0
2015-06-11T18:20:13.722 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (65 ms)
2015-06-11T18:20:16.638 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph quorum_status'
2015-06-11T18:20:16.819 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:16.821660 7f9e6f382700 -1 asok(0x7f9e68000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14201.asok': (13) Permission denied
2015-06-11T18:20:17.101 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":100,"quorum":[0,1,2],"quorum_names":["b","a","c"],"quorum_leader_name":"b","monmap":{"epoch":1,"fsid":"20c38c67-62bd-4624-ada0-d42475a70b2e","modified":"2015-06-11 20:29:44.513983","created":"2015-06-11 20:29:44.513983","mons":[{"rank":0,"name":"b","addr":"10.214.130.15:6789\/0"},{"rank":1,"name":"a","addr":"10.214.130.68:6789\/0"},{"rank":2,"name":"c","addr":"10.214.130.15:6790\/0"}]}}
2015-06-11T18:20:17.102 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3
2015-06-11T18:20:17.102 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.68:6789 mon_status'
2015-06-11T18:20:17.281 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:17.283914 7fbbb8db8700 -1 asok(0x7fbbb4000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14253.asok': (13) Permission denied
2015-06-11T18:20:17.458 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.15:6789 mon_status'
2015-06-11T18:20:17.649 INFO:teuthology.orchestra.run.vpm068.stderr:2015-06-11 21:20:17.651313 7f2d21f95700 -1 asok(0x7f2d1c000fc0) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-client.admin.14307.asok': (13) Permission denied
2015-06-11T18:20:17.894 INFO:teuthology.orchestra.run.vpm068:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.214.130.15:6790 mon_status'
2015-06-11T18:20:17.924 INFO:tasks.workunit.client.1.vpm128.stdout:[----------] 15 tests from LibRadosTwoPoolsECPP (74622 ms total)
2015-06-11T18:20:17.924 INFO:tasks.workunit.client.1.vpm128.stdout:
2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[----------] Global test environment tear-down
2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[==========] 39 tests from 4 test cases ran. (225153 ms total)
2015-06-11T18:20:17.925 INFO:tasks.workunit.client.1.vpm128.stdout:[ PASSED ] 12 tests.
2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] 27 tests, listed below:
2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteSnapScrub
2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteSnapTrimRace
2015-06-11T18:20:17.926 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Whiteout
2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Evict
2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.EvictSnap
2015-06-11T18:20:17.927 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.EvictSnap2
2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.TryFlush
2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.Flush
2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.FlushSnap
2015-06-11T18:20:17.928 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.FlushTryFlushRaces
2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.TryFlushReadRace
2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetRead
2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetWrite
2015-06-11T18:20:17.929 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.HitSetTrim
2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsPP.PromoteOn2ndRead
2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace
2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Whiteout
2015-06-11T18:20:17.930 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Evict
2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.EvictSnap
2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlush
2015-06-11T18:20:17.931 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.Flush
2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushSnap
2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.FlushTryFlushRaces
2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.TryFlushReadRace
2015-06-11T18:20:17.932 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetRead
2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.HitSetTrim
2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:[ FAILED ] LibRadosTwoPoolsECPP.PromoteOn2ndRead
2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:
2015-06-11T18:20:17.933 INFO:tasks.workunit.client.1.vpm128.stdout:27 FAILED TESTS
</pre>