Actions
QA result review and history.¶
Release QA has a new protocol. There are two roles which are rotated monthly:
- Release Backport Wrangler
- Release Backport Reviewer
On an every two week (avoiding the ambiguity of biweekly) basis, we cycle between maintained releases (e.g. Pacific, Quincy, and Reef). Usually there are only two releases maintained but after a new release there can be three for a short time.
Release Backport Wrangler¶
The job of the Wrangler is to:
- Make sure backports in "New" are done. This mostly involves bothering the Assignee to actually do the backport. If there are any backports which require discussion or have issues, bring it up to the larger team.
- Create a release branch:
- Label PRs to test with your own personal testing label (e.g. wip-$USER-testing)
- Use ptl-tool.py to build the branch (or manually): https://github.com/ceph/ceph/blob/main/src/script/ptl-tool.py
- Push the branch to ceph-ci.git. It must have the release name in the branch, e.g. "wip-pdonnell-20230915-pacific". https://github.com/ceph/ceph-ci.git
- When it's done building, run the "fs" suite with appropriate options (subset/priority/etc.). Generally, a priority of 125+ is used for release QA runs.
- When the run is complete, hand it off to the Release Backport Reviewer.
Release Backport Reviewer¶
- The Release Backport Wrangler or QA tester (probably Yuri) will ping you with QA results. Current process is that there will be a link to a Trello card like this one.
- Review each backport in the Trello description to avoid analyzing failures that may be caused by incorrect backports:
- Double check backport tracker, original tracker, and original PR for any notes about followup issues.
- Confirm # of commits with original PR, look for discrepancies.
- Confirm correctness of any conflict resolution in backport commits. Commit message should note this normally.
- (Try to) confirm that the backport is not missing changes that would not be necessary in the original PR. For example, Octopus and older releases still have kcephfs and multimds suites which may require QA changes not necessary in main/Pacific.
- It is also a good idea to confirm (via git) the following for the integration branch created by Yuri:
- The base branch is correct.
- The integration branch has the name of the release in it. For example wip-yuri... -pacific. This is necessary for ceph-ci to build it correctly.
- The PRs in the integration branch are the same as what's in the Trello.
- Now, collect results for the QA run. It's recommended to use the scrape tool.
- Confirm that the sha1 for the ceph version and the qa suite matches the integration branch.
- Make a record of the results:
- Add a new section to the release branch wiki with the date/link of the tests. It's recommended to copy/paste the wiki contents to a local text file for easier search and editing. You don't want to accidentally change the wrong thing in the browser or lose your work; we all know browsers are not a very usable interface to edit text. Instead work locally and then copy the new section to the wiki when done.
- For each group of failures:
- Check if the failure is tracked previously on the wiki already or via Google. If it's an existing failure, copy/paste the tracker ticket link/title to a new section on the wiki for the integration branch.
- Otherwise, if there is a new failure and you suspect it's not caused by a backport PR being tested, create a new tracker ticket like this one and add it to the list.
- Approve backport PRs you think are good. If it's your own PR, leave a "Reviewed-by..." comment and ask Yuri to approve.
- For backports that need fixes, leave a review requesting changes in the PR.
- Finally, summarize the result on Trello for Yuri, like this.
You may use this checklist to make sure all the tasks are covered while reviewing - https://tracker.ceph.com/projects/cephfs/wiki/Checklist_Backport_QA_Review.
Branches:¶
Maintained Releases¶
EOL Releases¶
Updated by Venky Shankar about 2 months ago · 18 revisions