Ceph » Stable releases

HOWTO »

HOWTO populate the integration branch » History » Revision 29

Revision 28 (Loïc Dachary, 08/10/2015 02:37 PM) → Revision 29/31 (Abhishek Varshney, 08/10/2015 02:49 PM)

Before a backported commit can be merged into the release branch, it must be tested in an integration branch. It serves two purposes: 

 * Detect trivial problems that would break the upgrade tests run on the release branch (they are run at the tip of the branch, not at the latest point release) 
 * Do not pollute the release branch with reverted commits (commits would have to be reverted because the stable branch must never be rebased) 

 Let say the release to be tested is *$release* (i.e. hammer for instance) and your local clone has two remotes: *origin* which is your read/write fork of the official ceph repository and *ceph* which is the read only official ceph repository. 
 <pre> 
 ceph 	 https://github.com/ceph/ceph.git (fetch) 
 ceph 	 https://github.com/ceph/ceph.git (push) 
 origin 	 git@github.com:dachary/ceph.git (fetch) 
 origin 	 git@github.com:dachary/ceph.git (push) 
 </pre> 

 * Fetch the latest $release : git fetch ceph 
 * Verify each pull request has have a successful make check bot report. Prefix with *DNM* is prefixed to the title of the ones one that do did not pass the make check bot, bot so that they can be are ignored (Note that this convention is used by the *collect_pr* function in the shell snippet at the end of this page). 
 * Create a $release-backports branch locally if it does not already exists: *git checkout -b $release-backports ceph/$release* 
 * Reset it to ceph/$release: *git reset --hard ceph/$release* 
 * Fetch all pull requests from github. Each pull request XXX has its own branch in the official ceph repository, named *pull/XXX* 
 ** For finding out the pull requests in github against a milestone; Github's api can be queried (see the *collect_pr* shell function below). 
 ** Retrieve the commits of the pull request XXX: *git fetch --force ceph +refs/pull/XXX/head:refs/remotes/ceph/pull/XXX/head* 
 ** Get the title of the pull request: *eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/XXX | jq .title)* 
 ** Merge the pull request in the local integration branch. The *--no-ff* ensures that even if the merge commit could be skipped, it will not be. *git merge --no-ff -m "$(echo -e "Merge XXX: $title\n\nReviewed-by: Myself <me@me.com>")" ceph/pull/XXX/head* 
 * Modify the pull requests individually to fix merge conflicts (i.e. they must merge cleanly, which may involve aggregating two pull requests into a single one designed to properly resolve the conflict).  
 * Push the integration branch to github: *git push --force origin $release-backports* 
 * Verify the integration branch compiles and passes make check 
 ** On the local machine: *./run-make-check.sh* 
 ** Via the make check bot by creating a pull request with the *$release-backports* integration branch 
 * Push the integration branch to the official repository: *git push --force ceph $release-backports* (requires write permission to the ceph repository) 

 The following snippet summarizes all the actions above: 
 <pre> 
 github_token=55844b338f33e616d6fec2ee # dachary 
 release=hammer 
 reviewer='Loic Dachary <ldachary@redhat.com>' 

 function collect_prs() { 
     local pages=$(curl -Is "https://api.github.com/repos/ceph/ceph/issues?page=1&per_page=100&access_token=$github_token" | grep ^Link | sed    's/.*?page=\([0-9]\+\).*/\1/') 
     local page 
     for page in $(seq 1 $pages) 
     do 
         curl --silent "https://api.github.com/repos/ceph/ceph/issues?page=$page&per_page=100&access_token=$github_token" | jq --arg release "$release" '.[] | select(.milestone.title == $release) | select(.title | contains("DNM") | not) | [.number, .title] | @sh' | while read line ; do  
             eval eval set $line 
             echo $1 
             echo "https://github.com/ceph/ceph/pull/$1 $2" >&2 
         done 
     done 
 } 
 PRS=$(collect_prs) 

 git fetch ceph 
 git checkout -b $release-backports ceph/$release 
 git reset --hard ceph/$release 
 git fetch --force ceph $(for ref in $PRS ; do echo +refs/pull/$ref/head:refs/remotes/ceph/pull/$ref/head ; echo +refs/pull/$ref/merge:refs/remotes/ceph/pull/$ref/merge ; done) 
 for pr in $PRS ; do eval title=$(curl --silent https://api.github.com/repos/ceph/ceph/pulls/$pr?access_token=$github_token | jq .title) ; echo "$pr $title" ; git --no-pager log --oneline ceph/pull/$pr/merge^1..ceph/pull/$pr/merge^2 ; git --no-pager merge --no-ff -m "$(echo -e "Merge $pr: $title\n\nReviewed-by: $reviewer")" ceph/pull/$pr/head ; done 
 git push --force ceph $release-backports 

 # hack to figure out what conflicts with what, lot of room for improvement 
 # PRS_B are the PRS found to conflict and set aside 
 PRS_A="5171 5170 5129 5062 5056 5051 5044 5043 5039 5037 4867 4788 4771 4769 4765 4762 4642 4641 4639 4633 4632 4631 4630 4584 4583 4582 4535" PR_B="4635 4636 4597" 
 for pr_a in $PR_A ; do git reset --hard ceph/$release ; for pr_b in $PR_B ; do git --no-pager merge --no-ff -m "$(echo -e "Merge $pr_a" ceph/pull/$pr/head ; git --no-pager merge --no-ff -m "Merge $pr_b" ceph/pull/$pr/head || echo "CONFLICT $pr_a $pr_b" ; done ; done 

 </pre>