Bug #15491
Updated by Yuri Weinstein about 8 years ago
We have several runs failed like this one: Ru: http://pulpito.ovh.sepia.ceph.com:8081/teuthology-2016-04-13_02:00:02-rados-infernalis-distro-basic-openstack Logs: http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-04-13_02:00:02-rados-infernalis-distro-basic-openstack/34527/teuthology.log <pre> 2016-04-13T02:37:52.848 INFO:teuthology.packaging:Looking for package version: http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/eff3ff4cbe9bfef7c6429b183f7dc0a16359c395/version 2016-04-13T02:37:53.024 INFO:teuthology.packaging:Package is not found at: http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/eff3ff4cbe9bfef7c6429b183f7dc0a16359c395/version (got HTTP code 404)... </pre> Troubleshooting logic: 1. problematic sha1 @eff3ff4cbe9bfef7c6429b183f7dc0a16359c395@ 2. it does not exist by checking - http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/ 3. on actual box @ubuntu@gitbuilder-ceph-deb-trusty-amd64-basic@ <pre> ubuntu@gitbuilder-ceph-deb-trusty-amd64-basic:/srv/autobuild-ceph/gitbuilder.git/out/pass$ ls -al eff3ff4cbe9bfef7c6429b183f7dc0a16359c395 -rw-rw-r-- 1 autobuild-ceph autobuild-ceph 671446 Mar 19 16:26 eff3ff4cbe9bfef7c6429b183f7dc0a16359c395 </pre> notice the date @Mar 19@ 4. however @infernalis@ branch was built successfully on April 3 http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-trusty-amd64-basic/log.cgi?log=6b72dac33f3e04707f24879adbb2d9328e39d43e 5. but teuthology checks for success status status via centos builds and it failed http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-rpm-centos7-amd64-basic/log.cgi?log=6b72dac33f3e04707f24879adbb2d9328e39d43e 6. @infernalis@ started failing after it was removed from gitbulder as old (sha1 get removed by this scripts after a week https://github.com/ceph/autobuild-ceph/blob/master/run-real#L81) That caused the scheduled runs test old @inferanils@ sha1 not the latest tip of the branch Zack, Dan Zack is looking at how to enhance the logic to compensate for this use case.