Project

General

Profile

Before a backported commit can be merged into the release branch, it must be tested in an integration branch. It serves two purposes:

  • Detect trivial problems that would break the upgrade tests run on the release branch (they are run at the tip of the branch, not at the latest point release)
  • Do not pollute the release branch with reverted commits (commits would have to be reverted because the stable branch must never be rebased)

Let say the release to be tested is $release (i.e. hammer for instance) and your local clone has two remotes: origin which is your read/write fork of the official ceph repository and ceph which is the read only official ceph repository.

ceph    https://github.com/ceph/ceph.git (fetch)
ceph    https://github.com/ceph/ceph.git (push)
origin    git@github.com:dachary/ceph.git (fetch)
origin    git@github.com:dachary/ceph.git (push)

  • Fetch the latest $release : git fetch ceph
  • Verify each pull request has a successful make check bot report. DNM is prefixed to the title of the ones that do not pass the make check bot, so that they can be ignored (Note that this convention is used by the collect_pr function in the shell snippet at the end of this page).
  • Create a $release-backports branch locally if it does not already exists: git checkout -b $release-backports ceph/$release
  • Reset it to ceph/$release: git reset --hard ceph/$release
  • Fetch all pull requests from github. Each pull request XXX has its own branch in the official ceph repository, named pull/XXX
    • For finding out the pull requests in github against a milestone; Github's api can be queried (see the collect_pr shell function below).
    • Retrieve the commits of the pull request XXX: git fetch --force ceph +refs/pull/XXX/head:refs/remotes/ceph/pull/XXX/head
    • Get the title of the pull request: eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/XXX | jq .title)
    • Merge the pull request in the local integration branch. The --no-ff ensures that even if the merge commit could be skipped, it will not be. git merge --no-ff -m "$(echo -e "Merge XXX: $title\n\nReviewed-by: Myself <>")" ceph/pull/XXX/head
  • Modify the pull requests individually to fix merge conflicts (i.e. they must merge cleanly, which may involve aggregating two pull requests into a single one designed to properly resolve the conflict).
  • Push the integration branch to github: git push --force origin $release-backports
  • Verify the integration branch compiles and passes make check
    • On the local machine: ./run-make-check.sh
    • Via the make check bot by creating a pull request with the $release-backports integration branch
  • Push the integration branch to the official repository: git push --force ceph $release-backports (requires write permission to the ceph repository)
  • Add a new comment to the issue tracking the progress of the release (see http://tracker.ceph.com/issues/14692#note-1 for instance), with the list of commits that are in the integration branch. Using the line starting with git --no-pager log --format='%H %s'... will produce such a list in a way that can be conveniently copy/pasted in redmine. This list has links to the individual pull requests for future reference.

The following snippet summarizes all the actions above:

github_token=XXXXXXXXX # dachary
release=hammer
reviewer='Loic Dachary <ldachary@redhat.com>'

function collect_prs() {
    local pages=$(curl -Is "https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100&access_token=$github_token" | grep ^Link | sed  's/.*?page=\([0-9]\+\).*/\1/')
    local page
    for page in $(seq 1 $pages)
    do
        curl --silent "https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100&access_token=$github_token" | jq --arg release "$release" '.[] | select(.milestone.title == $release) | select(.title | contains("DNM") | not) | [.number, .title] | @sh' | while read line ; do 
            eval eval set $line
            echo $1
            echo "https://github.com/ceph/ceph/pull/$1 $2" >&2
        done
    done
}
PRS=$(collect_prs)

git fetch ceph
git checkout -b $release-backports ceph/$release
git reset --hard ceph/$release
git fetch --force ceph $(for ref in $PRS ; do echo +refs/pull/$ref/head:refs/remotes/ceph/pull/$ref/head ; echo +refs/pull/$ref/merge:refs/remotes/ceph/pull/$ref/merge ; done)
for pr in $PRS ; do eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/$pr?access_token=$github_token | jq .title) ; echo "PR $pr $title" ; git --no-pager log --oneline ceph/pull/$pr/merge^1..ceph/pull/$pr/merge^2 ; git --no-pager merge --no-ff -m "$(echo -e "Merge pull request #$pr: $title\n\nReviewed-by: $reviewer")" ceph/pull/$pr/head ; done
git push --force ceph $release-backports
git --no-pager log --format='%H %s' --graph ceph/hammer..hammer-backports | perl -p -e 's/"/ /g; if (/\w+\s+Merge pull request #(\d+)/) { s|\w+\s+Merge pull request #(\d+).*|"Pull request $1":https://github.com/ceph/ceph/pull/$1|; } else { s|(\w+)\s+(.*)|"$2":https://github.com/ceph/ceph/commit/$1|; } s/\*/+/; s/^/* /;'