HOWTO populate the integration branch » History » Version 20
Loïc Dachary, 07/08/2015 05:28 PM
1 | 3 | Loïc Dachary | Before a backported commit can be merged into the release branch, it must be tested in an integration branch. It serves two purposes: |
---|---|---|---|
2 | |||
3 | * Detect trivial problems that would break the upgrade tests run on the release branch (they are run at the tip of the branch, not at the latest point release) |
||
4 | 13 | Loïc Dachary | * Do not pollute the release branch with reverted commits (commits would have to be reverted because the stable branch must never be rebased) |
5 | 1 | Loïc Dachary | |
6 | 11 | Loïc Dachary | Let say the release to be tested is *$release* (i.e. giant for instance) and your local clone has two remotes: *origin* which is your read/write fork of the official ceph repository and *ceph* which is the read only official ceph repository. |
7 | <pre> |
||
8 | ceph https://github.com/ceph/ceph.git (fetch) |
||
9 | ceph https://github.com/ceph/ceph.git (push) |
||
10 | origin git@github.com:dachary/ceph.git (fetch) |
||
11 | origin git@github.com:dachary/ceph.git (push) |
||
12 | </pre> |
||
13 | 1 | Loïc Dachary | |
14 | 18 | Loïc Dachary | * Create a $release-backports branch locally if it does not already exists: *git checkout -b $release-backports ceph/$release* |
15 | 11 | Loïc Dachary | * Reset it to ceph/$release: *git reset --hard ceph/$release* |
16 | * Fetch all pull requests from github. Each pull request XXX has its own branch in the official ceph repository, named *pull/XXX* |
||
17 | 15 | Abhishek Lekshmanan | ** For finding out the pull requests in github against a milestone; Github's api can be queried. Since github paginates results by default, this has to be run against all the pages. |
18 | <pre> |
||
19 | pages=$(curl -Is 'https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100' | grep ^Link | sed 's/.*?page=\([0-9]\+\).*/\1/') |
||
20 | PRS="" |
||
21 | for i in $(seq 1 $pages) |
||
22 | do |
||
23 | 16 | Abhishek Lekshmanan | PRS+=$(curl --silent 'https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100' | jq --arg release "$release" '.[] | select(.milestone.title == $release) | .number') |
24 | PRS+=' ' # otherwise we'll have a PR number that is appending the last PR from this loop |
||
25 | 15 | Abhishek Lekshmanan | done |
26 | </pre> |
||
27 | 11 | Loïc Dachary | ** Retrieve the commits of the pull request XXX: *git fetch --force ceph +refs/pull/XXX/head:refs/remotes/ceph/pull/XXX/head* |
28 | 12 | Loïc Dachary | ** Get the title of the pull request: *eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/XXX | jq .title)* |
29 | 11 | Loïc Dachary | ** Merge the pull request in the local integration branch. The *--no-ff* ensures that even if the merge commit could be skipped, it will not be. *git merge --no-ff -m "$(echo -e "Merge XXX: $title\n\nReviewed-by: Myself <me@me.com>")" ceph/pull/XXX/head* |
30 | * Modify the pull requests individually to fix merge conflicts (i.e. they must merge cleanly, which may involving aggregating two pull requests into a single one designed to properly resolve the conflict). |
||
31 | * Push the integration branch to github: *git push --force origin $release-backports* |
||
32 | * Verify the integration branch compiles and passes make check |
||
33 | ** On the local machine: *./run-make-check.sh* |
||
34 | ** Via the make check bot by creating a pull request with the $release-backports* integration branch |
||
35 | * Push the integration branch to the official repository: *git push --force ceph $release-backports* (requires write permission to the ceph repository) |
||
36 | 1 | Loïc Dachary | |
37 | 12 | Loïc Dachary | The following snippet summarizes all the actions above: |
38 | 9 | Loïc Dachary | <pre> |
39 | 20 | Loïc Dachary | github_token=55844b338f33e616d6fec2ee # dachary |
40 | 1 | Loïc Dachary | release=hammer |
41 | 20 | Loïc Dachary | reviewer='Loic Dachary <ldachary@redhat.com>' |
42 | 1 | Loïc Dachary | |
43 | 20 | Loïc Dachary | function collect_prs() { |
44 | local pages=$(curl -Is "https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100&access_token=$github_token" | grep ^Link | sed 's/.*?page=\([0-9]\+\).*/\1/') |
||
45 | local page |
||
46 | for page in $(seq 1 $pages) |
||
47 | do |
||
48 | curl --silent "https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100&access_token=$github_token" | jq --arg release "$release" '.[] | select(.milestone.title == $release) | select(.title | contains("DNM") | not) | [.number, .title] | @sh' | while read line ; do |
||
49 | eval eval set $line |
||
50 | echo $1 |
||
51 | echo "https://github.com/ceph/ceph/pull/$1 $2" >&2 |
||
52 | done |
||
53 | done |
||
54 | } |
||
55 | PRS=$(collect_prs) |
||
56 | 1 | Loïc Dachary | |
57 | 17 | Abhishek Lekshmanan | git checkout -b $release-backports ceph/$release |
58 | git reset --hard ceph/$release |
||
59 | git fetch --force ceph $(for ref in $PRS ; do echo +refs/pull/$ref/head:refs/remotes/ceph/pull/$ref/head ; echo +refs/pull/$ref/merge:refs/remotes/ceph/pull/$ref/merge ; done) |
||
60 | 20 | Loïc Dachary | for pr in $PRS ; do eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/$pr?access_token=$github_token | jq .title) ; echo "$pr $title" ; git --no-pager log --oneline ceph/pull/$pr/merge^1..ceph/pull/$pr/merge^2 ; git --no-pager merge --no-ff -m "$(echo -e "Merge $pr: $title\n\nReviewed-by: $reviewer")" ceph/pull/$pr/head ; done |
61 | git push --force ceph $release-backports |
||
62 | |||
63 | # hack to figure out what conflicts with what, lot of room for improvement |
||
64 | # PRS_B are the PRS found to conflict and set aside |
||
65 | PRS_A="5171 5170 5129 5062 5056 5051 5044 5043 5039 5037 4867 4788 4771 4769 4765 4762 4642 4641 4639 4633 4632 4631 4630 4584 4583 4582 4535" PR_B="4635 4636 4597" |
||
66 | for pr_a in $PR_A ; do git reset --hard ceph/$release ; for pr_b in $PR_B ; do git --no-pager merge --no-ff -m "$(echo -e "Merge $pr_a" ceph/pull/$pr/head ; git --no-pager merge --no-ff -m "Merge $pr_b" ceph/pull/$pr/head || echo "CONFLICT $pr_a $pr_b" ; done ; done |
||
67 | |||
68 | 1 | Loïc Dachary | </pre> |