Bug #15491
Updated by Yuri Weinstein about 8 years ago
We have several runs failed like this one:
Ru: http://pulpito.ovh.sepia.ceph.com:8081/teuthology-2016-04-13_02:00:02-rados-infernalis-distro-basic-openstack
Logs: http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-04-13_02:00:02-rados-infernalis-distro-basic-openstack/34527/teuthology.log
<pre>
2016-04-13T02:37:52.848 INFO:teuthology.packaging:Looking for package version: http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/eff3ff4cbe9bfef7c6429b183f7dc0a16359c395/version
2016-04-13T02:37:53.024 INFO:teuthology.packaging:Package is not found at: http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/eff3ff4cbe9bfef7c6429b183f7dc0a16359c395/version (got HTTP code 404)...
</pre>
Troubleshooting logic:
1. problematic sha1 @eff3ff4cbe9bfef7c6429b183f7dc0a16359c395@
2. it does not exist by checking - http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/sha1/
3. on actual box @ubuntu@gitbuilder-ceph-deb-trusty-amd64-basic@
<pre>
ubuntu@gitbuilder-ceph-deb-trusty-amd64-basic:/srv/autobuild-ceph/gitbuilder.git/out/pass$ ls -al eff3ff4cbe9bfef7c6429b183f7dc0a16359c395
-rw-rw-r-- 1 autobuild-ceph autobuild-ceph 671446 Mar 19 16:26 eff3ff4cbe9bfef7c6429b183f7dc0a16359c395
</pre>
notice the date @Mar 19@
4. however @infernalis@ branch was built successfully on April 3
http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-trusty-amd64-basic/log.cgi?log=6b72dac33f3e04707f24879adbb2d9328e39d43e
5. but teuthology checks for success status status via centos builds and it failed
http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-rpm-centos7-amd64-basic/log.cgi?log=6b72dac33f3e04707f24879adbb2d9328e39d43e
6. @infernalis@ started failing after it was removed from gitbulder as old
(sha1 get removed by this scripts after a week https://github.com/ceph/autobuild-ceph/blob/master/run-real#L81)
That caused the scheduled runs test old @inferanils@ sha1 not the latest tip of the branch
Zack is looking at how to enhance the logic to compensate for this use case.