Project

General

Profile

HOWTO populate the integration branch » History » Version 28

« Previous - Version 28/31 (diff) - Next » - Current version
Loïc Dachary, 08/10/2015 02:37 PM


Before a backported commit can be merged into the release branch, it must be tested in an integration branch. It serves two purposes:

  • Detect trivial problems that would break the upgrade tests run on the release branch (they are run at the tip of the branch, not at the latest point release)
  • Do not pollute the release branch with reverted commits (commits would have to be reverted because the stable branch must never be rebased)

Let say the release to be tested is $release (i.e. hammer for instance) and your local clone has two remotes: origin which is your read/write fork of the official ceph repository and ceph which is the read only official ceph repository.

ceph    https://github.com/ceph/ceph.git (fetch)
ceph    https://github.com/ceph/ceph.git (push)
origin    git@github.com:dachary/ceph.git (fetch)
origin    git@github.com:dachary/ceph.git (push)

  • Fetch the latest $release : git fetch ceph
  • Verify each pull request have a successful make check bot report. Prefix with DNM the title of the one that did not pass the make check bot so they are ignored (Note that this convention is used by the collect_pr function in the shell snippet at the end of this page).
  • Create a $release-backports branch locally if it does not already exists: git checkout -b $release-backports ceph/$release
  • Reset it to ceph/$release: git reset --hard ceph/$release
  • Fetch all pull requests from github. Each pull request XXX has its own branch in the official ceph repository, named pull/XXX
    • For finding out the pull requests in github against a milestone; Github's api can be queried (see the collect_pr shell function below).
    • Retrieve the commits of the pull request XXX: git fetch --force ceph +refs/pull/XXX/head:refs/remotes/ceph/pull/XXX/head
    • Get the title of the pull request: eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/XXX | jq .title)
    • Merge the pull request in the local integration branch. The --no-ff ensures that even if the merge commit could be skipped, it will not be. git merge --no-ff -m "$(echo -e "Merge XXX: $title\n\nReviewed-by: Myself <>")" ceph/pull/XXX/head
  • Modify the pull requests individually to fix merge conflicts (i.e. they must merge cleanly, which may involve aggregating two pull requests into a single one designed to properly resolve the conflict).
  • Push the integration branch to github: git push --force origin $release-backports
  • Verify the integration branch compiles and passes make check
    • On the local machine: ./run-make-check.sh
    • Via the make check bot by creating a pull request with the $release-backports integration branch
  • Push the integration branch to the official repository: git push --force ceph $release-backports (requires write permission to the ceph repository)

The following snippet summarizes all the actions above:

github_token=55844b338f33e616d6fec2ee # dachary
release=hammer
reviewer='Loic Dachary <ldachary@redhat.com>'

function collect_prs() {
    local pages=$(curl -Is "https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100&access_token=$github_token" | grep ^Link | sed  's/.*?page=\([0-9]\+\).*/\1/')
    local page
    for page in $(seq 1 $pages)
    do
        curl --silent "https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100&access_token=$github_token" | jq --arg release "$release" '.[] | select(.milestone.title == $release) | select(.title | contains("DNM") | not) | [.number, .title] | @sh' | while read line ; do 
            eval eval set $line
            echo $1
            echo "https://github.com/ceph/ceph/pull/$1 $2" >&2
        done
    done
}
PRS=$(collect_prs)

git fetch ceph
git checkout -b $release-backports ceph/$release
git reset --hard ceph/$release
git fetch --force ceph $(for ref in $PRS ; do echo +refs/pull/$ref/head:refs/remotes/ceph/pull/$ref/head ; echo +refs/pull/$ref/merge:refs/remotes/ceph/pull/$ref/merge ; done)
for pr in $PRS ; do eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/$pr?access_token=$github_token | jq .title) ; echo "$pr $title" ; git --no-pager log --oneline ceph/pull/$pr/merge^1..ceph/pull/$pr/merge^2 ; git --no-pager merge --no-ff -m "$(echo -e "Merge $pr: $title\n\nReviewed-by: $reviewer")" ceph/pull/$pr/head ; done
git push --force ceph $release-backports

# hack to figure out what conflicts with what, lot of room for improvement
# PRS_B are the PRS found to conflict and set aside
PRS_A="5171 5170 5129 5062 5056 5051 5044 5043 5039 5037 4867 4788 4771 4769 4765 4762 4642 4641 4639 4633 4632 4631 4630 4584 4583 4582 4535" PR_B="4635 4636 4597" 
for pr_a in $PR_A ; do git reset --hard ceph/$release ; for pr_b in $PR_B ; do git --no-pager merge --no-ff -m "$(echo -e "Merge $pr_a" ceph/pull/$pr/head ; git --no-pager merge --no-ff -m "Merge $pr_b" ceph/pull/$pr/head || echo "CONFLICT $pr_a $pr_b" ; done ; done