Project

General

Profile

HOWTO populate the integration branch » History » Version 31

Loïc Dachary, 02/26/2016 06:13 AM
Add helper to show the commits of the integration branch

1 3 Loïc Dachary
Before a backported commit can be merged into the release branch, it must be tested in an integration branch. It serves two purposes:
2 3 Loïc Dachary
3 3 Loïc Dachary
* Detect trivial problems that would break the upgrade tests run on the release branch (they are run at the tip of the branch, not at the latest point release)
4 13 Loïc Dachary
* Do not pollute the release branch with reverted commits (commits would have to be reverted because the stable branch must never be rebased)
5 1 Loïc Dachary
6 24 Loïc Dachary
Let say the release to be tested is *$release* (i.e. hammer for instance) and your local clone has two remotes: *origin* which is your read/write fork of the official ceph repository and *ceph* which is the read only official ceph repository.
7 11 Loïc Dachary
<pre>
8 11 Loïc Dachary
ceph	https://github.com/ceph/ceph.git (fetch)
9 11 Loïc Dachary
ceph	https://github.com/ceph/ceph.git (push)
10 11 Loïc Dachary
origin	git@github.com:dachary/ceph.git (fetch)
11 11 Loïc Dachary
origin	git@github.com:dachary/ceph.git (push)
12 11 Loïc Dachary
</pre>
13 1 Loïc Dachary
14 22 Loïc Dachary
* Fetch the latest $release : git fetch ceph
15 29 Abhishek Varshney
* Verify each pull request has a successful make check bot report. *DNM* is prefixed to the title of the ones that do not pass the make check bot, so that they can be ignored (Note that this convention is used by the *collect_pr* function in the shell snippet at the end of this page).
16 18 Loïc Dachary
* Create a $release-backports branch locally if it does not already exists: *git checkout -b $release-backports ceph/$release*
17 11 Loïc Dachary
* Reset it to ceph/$release: *git reset --hard ceph/$release*
18 11 Loïc Dachary
* Fetch all pull requests from github. Each pull request XXX has its own branch in the official ceph repository, named *pull/XXX*
19 21 Loïc Dachary
** For finding out the pull requests in github against a milestone; Github's api can be queried (see the *collect_pr* shell function below).
20 11 Loïc Dachary
** Retrieve the commits of the pull request XXX: *git fetch --force ceph +refs/pull/XXX/head:refs/remotes/ceph/pull/XXX/head*
21 12 Loïc Dachary
** Get the title of the pull request: *eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/XXX | jq .title)*
22 11 Loïc Dachary
** Merge the pull request in the local integration branch. The *--no-ff* ensures that even if the merge commit could be skipped, it will not be. *git merge --no-ff -m "$(echo -e "Merge XXX: $title\n\nReviewed-by: Myself <me@me.com>")" ceph/pull/XXX/head*
23 26 Loïc Dachary
* Modify the pull requests individually to fix merge conflicts (i.e. they must merge cleanly, which may involve aggregating two pull requests into a single one designed to properly resolve the conflict). 
24 11 Loïc Dachary
* Push the integration branch to github: *git push --force origin $release-backports*
25 11 Loïc Dachary
* Verify the integration branch compiles and passes make check
26 11 Loïc Dachary
** On the local machine: *./run-make-check.sh*
27 27 Loïc Dachary
** Via the make check bot by creating a pull request with the *$release-backports* integration branch
28 11 Loïc Dachary
* Push the integration branch to the official repository: *git push --force ceph $release-backports* (requires write permission to the ceph repository)
29 31 Loïc Dachary
* Add a new comment to the issue tracking the progress of the release (see http://tracker.ceph.com/issues/14692#note-1 for instance), with the list of commits that are in the integration branch. Using the line starting with *git --no-pager log --format='%H %s'...* will produce such a list in a way that can be conveniently copy/pasted in redmine. This list has links to the individual pull requests for future reference.
30 1 Loïc Dachary
31 12 Loïc Dachary
The following snippet summarizes all the actions above:
32 9 Loïc Dachary
<pre>
33 30 Loïc Dachary
github_token=XXXXXXXXX # dachary
34 1 Loïc Dachary
release=hammer
35 20 Loïc Dachary
reviewer='Loic Dachary <ldachary@redhat.com>'
36 1 Loïc Dachary
37 20 Loïc Dachary
function collect_prs() {
38 20 Loïc Dachary
    local pages=$(curl -Is "https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100&access_token=$github_token" | grep ^Link | sed  's/.*?page=\([0-9]\+\).*/\1/')
39 20 Loïc Dachary
    local page
40 20 Loïc Dachary
    for page in $(seq 1 $pages)
41 20 Loïc Dachary
    do
42 20 Loïc Dachary
        curl --silent "https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100&access_token=$github_token" | jq --arg release "$release" '.[] | select(.milestone.title == $release) | select(.title | contains("DNM") | not) | [.number, .title] | @sh' | while read line ; do 
43 20 Loïc Dachary
            eval eval set $line
44 20 Loïc Dachary
            echo $1
45 20 Loïc Dachary
            echo "https://github.com/ceph/ceph/pull/$1 $2" >&2
46 20 Loïc Dachary
        done
47 20 Loïc Dachary
    done
48 20 Loïc Dachary
}
49 20 Loïc Dachary
PRS=$(collect_prs)
50 1 Loïc Dachary
51 22 Loïc Dachary
git fetch ceph
52 17 Abhishek Lekshmanan
git checkout -b $release-backports ceph/$release
53 17 Abhishek Lekshmanan
git reset --hard ceph/$release
54 17 Abhishek Lekshmanan
git fetch --force ceph $(for ref in $PRS ; do echo +refs/pull/$ref/head:refs/remotes/ceph/pull/$ref/head ; echo +refs/pull/$ref/merge:refs/remotes/ceph/pull/$ref/merge ; done)
55 30 Loïc Dachary
for pr in $PRS ; do eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/$pr?access_token=$github_token | jq .title) ; echo "PR $pr $title" ; git --no-pager log --oneline ceph/pull/$pr/merge^1..ceph/pull/$pr/merge^2 ; git --no-pager merge --no-ff -m "$(echo -e "Merge pull request #$pr: $title\n\nReviewed-by: $reviewer")" ceph/pull/$pr/head ; done
56 20 Loïc Dachary
git push --force ceph $release-backports
57 31 Loïc Dachary
git --no-pager log --format='%H %s' --graph ceph/hammer..hammer-backports | perl -p -e 's/"/ /g; if (/\w+\s+Merge pull request #(\d+)/) { s|\w+\s+Merge pull request #(\d+).*|"Pull request $1":https://github.com/ceph/ceph/pull/$1|; } else { s|(\w+)\s+(.*)|"$2":https://github.com/ceph/ceph/commit/$1|; } s/\*/+/; s/^/* /;'
58 1 Loïc Dachary
</pre>