Bug #24079
centos7 kernel jobs are hitting JENKINS-37229
0%
Description
[EnvInject] - Loading node environment variables. Building remotely on 172.21.1.40+slave-centos04 (sepia x86_64 huge centos7 amd64) in workspace /home/jenkins-build/build/workspace/kernel/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge Wiping out workspace first. Cloning the remote Git repository Using shallow clone Avoid fetching tags Cloning repository https://github.com/ceph/ceph-client.git > git init /home/jenkins-build/build/workspace/kernel/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge # timeout=10 Fetching upstream changes from https://github.com/ceph/ceph-client.git > git --version # timeout=10 > git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1 # timeout=20 > git config remote.origin.url https://github.com/ceph/ceph-client.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/ceph/ceph-client.git # timeout=10 Fetching upstream changes from https://github.com/ceph/ceph-client.git > git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1 # timeout=20 ERROR: Error fetching remote repo 'origin' hudson.plugins.git.GitException: Failed to fetch from https://github.com/ceph/ceph-client.git at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:862) at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160) at hudson.scm.SCM.checkout(SCM.java:504) at hudson.model.AbstractProject.checkout(AbstractProject.java:1208) at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574) at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499) at hudson.model.Run.execute(Run.java:1794) at hudson.matrix.MatrixRun.run(MatrixRun.java:153) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1" returned status code 128: stdout: stderr: fatal: git fetch-pack: expected shallow list
"git --version" on 172.21.1.40+slave-centos04 and a couple of others I have checked is 1.8.3.1. Looks like https://issues.jenkins-ci.org/browse/JENKINS-37229 to me.
History
#1 Updated by Ilya Dryomov almost 6 years ago
- Priority changed from Normal to High
#2 Updated by Ilya Dryomov almost 6 years ago
- Priority changed from High to Normal
https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/431/ seems to have worked fine on 172.21.1.40+slave-centos04, even though git version is still 1.8.3.1. Not sure what has changed.
#3 Updated by Ilya Dryomov almost 6 years ago
Here is another failed job, after about a couple of weeks without failures: https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/447/
#4 Updated by David Galloway almost 6 years ago
- Assignee set to David Galloway
Is the shallow clone required? The first alternative I can think of for this is going to be to install a newer version of a git RPM from an untrusted source which I'd like to avoid.
#5 Updated by Ilya Dryomov almost 6 years ago
Shallow clone was switched on to reduce fetch processing time because at some point it started to take too long -- see d3ec6e485cfa ("kernel: set git clone/fetch timeout to 20 minutes") and 35beb188cbda ("kernel: use shallow clones") in ceph-build repo.
Re: git version, I'm not actually sure it is to blame here. As noted in https://tracker.ceph.com/issues/24079#note-2, I saw jobs both pass and fail with that trace on the same 172.21.1.40+slave-centos04 worker with the same git version, which makes me question if it is indeed a git CLI parsing bug. I might be misinterpreting jenkins logs and missing something about how we provision workers though. Perhaps go though the failed jobs and see if something catches your eye?
#6 Updated by Ilya Dryomov over 5 years ago
It's back: https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/475/, https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/476/, https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/477/.