Project

General

Profile

Bug #24079

centos7 kernel jobs are hitting JENKINS-37229

Added by Ilya Dryomov almost 6 years ago. Updated about 5 years ago.

Status:
New
Priority:
Normal
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/427/console

[EnvInject] - Loading node environment variables.
Building remotely on 172.21.1.40+slave-centos04 (sepia x86_64 huge centos7 amd64) in workspace /home/jenkins-build/build/workspace/kernel/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge
Wiping out workspace first.
Cloning the remote Git repository
Using shallow clone
Avoid fetching tags
Cloning repository https://github.com/ceph/ceph-client.git
 > git init /home/jenkins-build/build/workspace/kernel/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge # timeout=10
Fetching upstream changes from https://github.com/ceph/ceph-client.git
 > git --version # timeout=10
 > git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1 # timeout=20
 > git config remote.origin.url https://github.com/ceph/ceph-client.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/ceph/ceph-client.git # timeout=10
Fetching upstream changes from https://github.com/ceph/ceph-client.git
 > git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1 # timeout=20
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from https://github.com/ceph/ceph-client.git
    at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:862)
    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1129)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1160)
    at hudson.scm.SCM.checkout(SCM.java:504)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1208)
    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
    at hudson.model.Run.execute(Run.java:1794)
    at hudson.matrix.MatrixRun.run(MatrixRun.java:153)
    at hudson.model.ResourceController.execute(ResourceController.java:97)
    at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.plugins.git.GitException: Command "git fetch --no-tags --progress https://github.com/ceph/ceph-client.git +refs/heads/*:refs/remotes/origin/* --depth=1" returned status code 128:
stdout: 
stderr: fatal: git fetch-pack: expected shallow list

"git --version" on 172.21.1.40+slave-centos04 and a couple of others I have checked is 1.8.3.1. Looks like https://issues.jenkins-ci.org/browse/JENKINS-37229 to me.

History

#1 Updated by Ilya Dryomov almost 6 years ago

  • Priority changed from Normal to High

#2 Updated by Ilya Dryomov almost 6 years ago

  • Priority changed from High to Normal

https://jenkins.ceph.com/job/kernel/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/431/ seems to have worked fine on 172.21.1.40+slave-centos04, even though git version is still 1.8.3.1. Not sure what has changed.

#4 Updated by David Galloway almost 6 years ago

  • Assignee set to David Galloway

Is the shallow clone required? The first alternative I can think of for this is going to be to install a newer version of a git RPM from an untrusted source which I'd like to avoid.

#5 Updated by Ilya Dryomov almost 6 years ago

Shallow clone was switched on to reduce fetch processing time because at some point it started to take too long -- see d3ec6e485cfa ("kernel: set git clone/fetch timeout to 20 minutes") and 35beb188cbda ("kernel: use shallow clones") in ceph-build repo.

Re: git version, I'm not actually sure it is to blame here. As noted in https://tracker.ceph.com/issues/24079#note-2, I saw jobs both pass and fail with that trace on the same 172.21.1.40+slave-centos04 worker with the same git version, which makes me question if it is indeed a git CLI parsing bug. I might be misinterpreting jenkins logs and missing something about how we provision workers though. Perhaps go though the failed jobs and see if something catches your eye?

Also available in: Atom PDF