Project

General

Profile

Actions

Support #62204

open

jenkins jobs are taking a very long time to clone the repo

Added by Guillaume Abrioux 9 months ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Assignee:
-
% Done:

0%

Tags:
Reviewed:
Affected Versions:
Pull request ID:

Description

I noticed that when a job is triggered in Jenkins, it takes a very long time to clone ceph/ceph

~20min to run the following command:

[jenkins-build@irvingi01 workspace]$ git init /home/jenkins-build/build/workspace/test-guits/
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:     git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:     git branch -m <name>
Initialized empty Git repository in /home/jenkins-build/build/workspace/test-guits/.git/
[jenkins-build@irvingi01 workspace]$ cd test-guits/
[jenkins-build@irvingi01 test-guits]$ time git fetch --tags --force --progress -- https://github.com/ceph/ceph +refs/heads/*:refs/remotes/origin/*
remote: Enumerating objects: 1233821, done.
remote: Counting objects: 100% (192/192), done.
remote: Compressing objects: 100% (157/157), done.
remote: Total 1233821 (delta 102), reused 69 (delta 35), pack-reused 1233629
Receiving objects: 100% (1233821/1233821), 707.16 MiB | 643.00 KiB/s, done.
Resolving deltas: 100% (984261/984261), done.
From https://github.com/ceph/ceph

... omitted output ...

real    19m5.232s
user    2m11.748s
sys    0m14.542s
[jenkins-build@irvingi01 test-guits]$

for most of the jobs run in the CI, it is a very significant amount of time, just for cloning the repo.
For instance, I can see the average for `make check` job is ~1h, it means the cloning step represents ~33% of the total time.
Another example, the pytest/flake8 commands in 'ceph-volume tox' take less than 3min but this job takes more than 20min in the CI just for something that actually runs in less than 3min it means more than 80% of the total time is actually the cloning step.

FWIW, adding --depth=1 reduces the time from ~20min to ~6min:

[jenkins-build@irvingi01 test-guits]$ time git fetch --depth=1 --tags --force --progress -- https://github.com/ceph/ceph +refs/heads/*:refs/remotes/origin/*
remote: Enumerating objects: 159153, done.
remote: Counting objects: 100% (159153/159153), done.
remote: Compressing objects: 100% (84910/84910), done.
remote: Total 159153 (delta 117857), reused 106075 (delta 71013), pack-reused 0
Receiving objects: 100% (159153/159153), 162.68 MiB | 9.68 MiB/s, done.
Resolving deltas: 100% (117857/117857), done.
From https://github.com/ceph/ceph

... omitted output ...

real    5m37.648s
user    1m50.410s
sys    0m17.528s
[jenkins-build@irvingi01 test-guits]$

That being said:
- I'm unsure if we can customize the parameters used by Jenkins to clone the repo,
- I'm unsure whether we don't need the full commits history. (eg: could it be an issue if there's an automatic rebase onto the last commit from the target branch?)

I'm just raising this if you think this is worthwhile to investigate.
(I couldn't find if this concern was already raised and tracked somewhere else, feel free to close in that case.)

Actions #1

Updated by Radoslaw Zarzynski 9 months ago

- I'm unsure whether we don't need the full commits history. (eg: could it be an issue if there's an automatic rebase onto the last commit from the target branch?)

I'm afraid this might be a valid worry as the make check bot really rebases the tested PR on top of main.
Anyway, this is definitely worth bring to a CDM.

Actions #2

Updated by Ronen Friedman 9 months ago

(apart from supporting the idea of --depth...)
would adding '--jobs=..' help?

Actions

Also available in: Atom PDF