Project

General

Profile

Bug #19976

ceph-docs job can fail to build with "fatal: ambiguous argument"

Added by Alfredo Deza almost 7 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Output from failed build:

Started by GitHub push by cbodley
[EnvInject] - Loading node environment variables.
Building remotely on docs.ceph.com (docs) in workspace /home/jenkins-build/build/workspace/ceph-docs
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/ceph/ceph
 > git init /home/jenkins-build/build/workspace/ceph-docs # timeout=10
Fetching upstream changes from https://github.com/ceph/ceph
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/ceph/ceph +refs/heads/*:refs/remotes/origin/* # timeout=20
 > git config remote.origin.url https://github.com/ceph/ceph # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/ceph/ceph # timeout=10
Fetching upstream changes from https://github.com/ceph/ceph
 > git fetch --tags --progress https://github.com/ceph/ceph +refs/heads/*:refs/remotes/origin/* # timeout=20
Seen branch in repository origin/argonaut
Seen branch in repository origin/bobtail
Seen branch in repository origin/cuttlefish
Seen branch in repository origin/dnm-dang-reshard
Seen branch in repository origin/dumpling
Seen branch in repository origin/emperor
Seen branch in repository origin/evelu-ceph-disk
Seen branch in repository origin/firefly
Seen branch in repository origin/giant
Seen branch in repository origin/hammer
Seen branch in repository origin/hammer-next
Seen branch in repository origin/infernalis
Seen branch in repository origin/jewel
Seen branch in repository origin/jewel-next
Seen branch in repository origin/kraken
Seen branch in repository origin/luminous
Seen branch in repository origin/master
Seen branch in repository origin/osd-mark-inactive
Seen branch in repository origin/rh-jewel
Seen branch in repository origin/wip-15752-master_ceph
Seen branch in repository origin/wip-19280
Seen branch in repository origin/wip-19805-jewel
Seen branch in repository origin/wip-bluestore-prefer-wal-size
Seen branch in repository origin/wip-bp-kraken-systemd
Seen branch in repository origin/wip-buildfix-python-out-of-tree
Seen branch in repository origin/wip-cd-dev-option
Seen branch in repository origin/wip-cd-hammer
Seen branch in repository origin/wip-ceph-disk-fix-jewel
Seen branch in repository origin/wip-cherrypick-4tamil
Seen branch in repository origin/wip-da-SCA-20170510
Seen branch in repository origin/wip-daemon-failure
Seen branch in repository origin/wip-doc-rgw-remove-agent
Seen branch in repository origin/wip-json-smarter-escapes
Seen branch in repository origin/wip-master-rgw-0lenfix1
Seen branch in repository origin/wip-message-factory
Seen branch in repository origin/wip-nfs-ganesha-teuthology
Seen branch in repository origin/wip-orit-resharding-hammer
Seen branch in repository origin/wip-orit-testing
Seen branch in repository origin/wip-osd-ioinject
Seen branch in repository origin/wip-qemu-redir
Seen branch in repository origin/wip-rest-flask
Seen branch in repository origin/wip-rest-flask-kraken
Seen branch in repository origin/wip-rgw-config-docs
Seen branch in repository origin/wip-rgw-multisite-teuthology
Seen branch in repository origin/wip-rgw-polymorphic-errors
Seen branch in repository origin/wip-rgw-ssl-teuthology
Seen branch in repository origin/wip-s3a-hadoop
Seen branch in repository origin/wip-sage-testing
Seen branch in repository origin/wip-selinux-optimize
Seen branch in repository origin/wip-snap-trim-sleep
Seen branch in repository origin/wip-turn-off-fcgi-tserlin
Seen branch in repository origin/wip-use-only-dev
Seen branch in repository origin/wip-v10.2.7
Seen branch in repository origin/wip-xml-string-wrap
Seen branch in repository origin/wip_dmclock_clients
Seen 55 remote branches
JENKINS-19022: warning: possible memory leak due to Git plugin usage; see: https://wiki.jenkins-ci.org/display/JENKINS/Remove+Git+Plugin+BuildsByBranch+BuildData
Checking out Revision ba249854e586b056947a51f7d23bc5ec821bd976 (origin/wip-doc-rgw-remove-agent)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ba249854e586b056947a51f7d23bc5ec821bd976 # timeout=20
 > git rev-list 0f9935cc97d259e045993762eced89388e3bb712 # timeout=10
First time build. Skipping changelog.
[ceph-docs] $ /bin/bash /tmp/jenkins838689737648461580.sh
++ git diff --name-only 'ba249854e586b056947a51f7d23bc5ec821bd976^1...ba249854e586b056947a51f7d23bc5ec821bd976^2'
fatal: ambiguous argument 'ba249854e586b056947a51f7d23bc5ec821bd976^1...ba249854e586b056947a51f7d23bc5ec821bd976^2': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
+ files=
Build step 'Execute shell' marked build as failure
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 0s
Finished: FAILURE

Offending line in the job config seems to be:

files="$(git diff --name-only ${GIT_COMMIT}^1...${GIT_COMMIT}^2)" 

History

#1 Updated by Alfredo Deza almost 7 years ago

  • Description updated (diff)
  • Assignee set to Dan Mick

#2 Updated by Dan Mick almost 7 years ago

Yes. <sha1>^1 and <sha1>^2 only have meaning for merge commits. The problem is I'm unclear how to use the env vars passed in by the GitSCM plugin to properly identify the commit range in the case of a non-merge commit.

https://wiki.jenkins-ci.org/display/JENKINS/Git+plugin#GitPlugin-AdvancedFeatures "Environment Variables" documents them:

  • GIT_COMMIT - SHA of the current
  • GIT_BRANCH - Name of the remote repository (defaults to origin), followed by name of the branch currently being used, e.g. "origin/master" or "origin/foo"
  • GIT_LOCAL_BRANCH - Name of the branch on Jenkins. When the "checkout to specific local branch" behavior is configured, the variable is published. If the behavior is configured as null or **, the property will contain the resulting local branch name sans the remote name.
  • GIT_PREVIOUS_COMMIT - SHA of the previous built commit from the same branch (the current SHA on first build in branch)
  • GIT_PREVIOUS_SUCCESSFUL_COMMIT - SHA of the previous successfully built commit from the same branch.
  • GIT_URL - Repository remote URL
  • GIT_URL_N - Repository remote URLs when there are more than 1 remotes, e.g. GIT_URL_1, * GIT_URL_2
  • GIT_AUTHOR_NAME and GIT_COMMITTER_NAME - The name entered if the "Custom user name/e-mail address" behaviour is enabled; falls back to the value entered in the Jenkins system config under "Global Config user.name Value" (if any)
  • GIT_AUTHOR_EMAIL and GIT_COMMITTER_EMAIL - The email entered if the "Custom user name/e-mail address" behaviour is enabled; falls back to the value entered in the Jenkins system config under "Global Config user.email Value" (if any)

So, maybe the right strategy is to 1) determine if GIT_COMMIT is a merge commit, and if so do what we do now; otherwise, maybe use GIT_COMMIT and...GIT_PREVIOUS_SUCCESSFUL_COMMIT?...seems like there ought to be a way to get a 'parent' commit that doesn't depend on what Jenkins has or hasn't built, though

#3 Updated by Dan Mick almost 7 years ago

Close. For a new branch, the environment doesn't contain anything useful but GIT_COMMIT:

BUILD_CAUSE=SCMTRIGGER
BUILD_CAUSE_SCMTRIGGER=true
BUILD_DISPLAY_NAME=#10426
BUILD_ID=10426
BUILD_NUMBER=10426
BUILD_TAG=jenkins-ceph-docs-10426
BUILD_URL=https://jenkins.ceph.com/job/ceph-docs/10426/
CHACRACTL_URL=https://chacra.ceph.com/
CHACRACTL_USER=admin
EXECUTOR_NUMBER=1
GIT_BRANCH=origin/wip-dmick-doc2
GIT_COMMIT=f2213331b233f24863e0e3c5ea474d29f601a244
GIT_URL=https://github.com/ceph/ceph
HOME=/home/jenkins-build
HUDSON_COOKIE=****
HUDSON_HOME=/var/lib/jenkins
HUDSON_SERVER_COOKIE=****
HUDSON_URL=https://jenkins.ceph.com/
JENKINS_HOME=/var/lib/jenkins
JENKINS_SERVER_COOKIE=****
JENKINS_URL=https://jenkins.ceph.com/
JOB_BASE_NAME=ceph-docs
JOB_BUILDER_USER=ceph-jenkins
JOB_DISPLAY_URL=https://jenkins.ceph.com/job/ceph-docs/display/redirect
JOB_NAME=ceph-docs
JOB_URL=https://jenkins.ceph.com/job/ceph-docs/
LANG=en_US.UTF-8
LOGNAME=jenkins-build
NODE_LABELS=docs docs.ceph.com
NODE_NAME=docs.ceph.com
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/home/jenkins-build/build/workspace/ceph-docs
ROOT_BUILD_CAUSE=SCMTRIGGER
ROOT_BUILD_CAUSE_SCMTRIGGER=true
RUN_CHANGES_DISPLAY_URL=https://jenkins.ceph.com/job/ceph-docs/10426/display/redirect?page=changes
RUN_DISPLAY_URL=https://jenkins.ceph.com/job/ceph-docs/10426/display/redirect
SHAMAN_API_USER=admin
SHLVL=1
USER=jenkins-build

yet, if you look at RUN_CHANGES_DISPLAY_URL=https://jenkins.ceph.com/job/ceph-docs/10426/display/redirect?page=changes, clearly Jenkins knows that only the one
commit is contained.

I'm puzzled.

#4 Updated by Dan Mick almost 7 years ago

Thinking, and bouncing ideas off/absorbing ideas from joshd, I think a workable solution would be to loop across the well-known branches, looking for common ancestors, and counting the commits from the common ancestor to the pushed tip; the shortest sequence is the minimal set of 'new' commits that should be evaluated. I'll hack something up based on this.

#5 Updated by Dan Mick almost 7 years ago

So I hacked this up, which shows some promise, but it may not be reliable:

#!/bin/bash

WKBLIST="hammer infernalis jewel kraken luminous master" 

for b in "$@"; do
    echo -en "$b: " 
    minwkb="" 
    mincount=100000
    branchcommit=$(git rev-parse $b)
    for wkb in $WKBLIST; do
        mergebase=$(git merge-base origin/$wkb $b)
        if [ -z $mergebase ] ; then continue; fi
        if [ $mergebase == $branchcommit ] ; then
            echo "already merged into $wkb" 
            exit 1
        fi
        count=$(git rev-list --count $mergebase..$b)
        if [ $count -lt $mincount ] ; then
            mincount=$count
            minwkb=$wkb
            minmb=$mergebase
        fi
    done
    echo "is $mincount past ${minmb:0:8} on $minwkb" 
done
exit 0

#6 Updated by David Galloway over 2 years ago

  • Status changed from New to Resolved

Also available in: Atom PDF