Ceph » devops

Jenkins builds were failing with on the Centos7 machine with:

Enabling updates-source repository
http://apt-mirror.front.sepia.ceph.com/centosvault/centos/7/os/Source/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
failure: repodata/repomd.xml from base-source: [Errno 256] No more mirrors to try.
http://apt-mirror.front.sepia.ceph.com/centosvault/centos/7/os/Source/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Build step 'Execute shell' marked build as failure
Archiving artifacts
Finished: FAILURE

Jenkins link: http://jenkins.ceph.com/job/ceph-build/Arch=x86_64,Distro=centos7/311/console

I tried logging into apt-mirror to run the mirror scripts and there is no /home/apt-mirror. After spelunking more I found out that
there is another server that is getting proxied.

After looking at the script decided to check the repo urls for the RPM node that was failing, and those were incorrect (the archive doesn't have
any centos 7 urls).

The repos had 'centosvault' in them and I modified them to use the right urls from the centos mirror (in /etc/yum.repos.d/):

$ grep -Hirn "64" *
CentOS-Sources.repo:15:baseurl=http://apt-mirror.front.sepia.ceph.com/centos/$releasever/os/x86_64/
CentOS-Sources.repo:23:baseurl=http://apt-mirror.front.sepia.ceph.com/centos/$releasever/updates/x86_64/
CentOS-Sources.repo:31:baseurl=http://apt-mirror.front.sepia.ceph.com/centos/$releasever/extras/x86_64/
CentOS-Sources.repo:39:baseurl=http://apt-mirror.front.sepia.ceph.com/centos/$releasever/centosplus/x86_64/

I opened this issue because I am not sure if we are automating this and something else needs to change, this all feels
very hacky although it now works.