Ceph » Stable releases

Workflow¶

• Preparing the release
• Cutting the release
  • Abhishek asks Sage if a point release should be published YES
  • Abhishek gets approval from all leads
    • Yehuda, rgw: (Yehuda backported two more commits, inferring an implicit approval, see the previous approval thread at OK)
    • Gregory, CephFS: OK
    • Josh, RBD: OK
    • Sam, rados: OK
  • Sage writes and commits the release notes DONE (draft release notes)
  • Abhishek informs Yuri that the branch is ready for testing
  • Yuri runs additional integration tests DONE
  • If Yuri discovers new bugs with severity Critical, the release goes back to being prepared, it was not ready after all
  • Yuri informs Alfredo that the branch is ready for release DONE
  • Alfredo creates the packages and sets the release tag DONE

Release information¶

• branch to build from: hammer, commit:d967cecf0a5d7fbf992a0195341cbd893a358264
• version: v0.94.2
• type of release: point release
• where to publish the release: debian/rpm-$release

• + Pull request 4773
• |\
• | + debian: set rest-bench-dbg ceph-test-dbg dependencies
• |/
• + Pull request 4696
• |\
• | + rgw: merge manifests correctly when there's prefix override
• | + rgw: restore buffer of multipart upload after EEXIST
• + | Pull request 4649
• |\ \
• | + | packaging: include ceph_perf_objectstore
• + | | Pull request 4568
• |\ \ \
• | |/ /
• |/| |
• | + | rgw: improve metadata handling on copy operation of Swift API.
• | + | rgw: improve code formatting ONLY.
• | + | rgw: send X-Copied-From-Last-Modified header of Swift API.
• | + | rgw: dump object metadata in response for COPY request of Swift API.
• | + | rgw: refactor dumping metadata of Swift objects.
• | + | rgw: add support for X-Copied-From{-Account} headers of Swift API.
• | + | rgw: send ETag, Last-Modified in response for copying Swift cobject.
• | + | rgw: Swift API. Allows setting attributes with COPY object operation.
• + | | Pull request 4629
• |\ \ \
• | + | | client: fix error handling in check_pool_perm
• | + | | client: use SaferCond in check_pool_perm
• | + | | client: check OSD caps before read/write
• |/ / /
• + | | Pull request 4567
• |\ \ \
• | + | | rgw: improve format of X-Timestamp on Swift objects.
• | + | | rgw: add support for X-Timestamp on Swift containers.
• | |/ /
• + | | Pull request 4570
• |\ \ \
• | + | | civetweb: update max num of threads
• | |/ /
• + | | Pull request 4573
• |\ \ \
• | + | | rgw: update keystone cache with token info
• | |/ /
• + | | Pull request 4574
• |\ \ \
• | + | | rgw: don't use rgw_socket_path if frontend is configured
• | |/ /
• + | | Pull request 4575
• |\ \ \
• | + | | rgw: send Last-Modified header in response for PUT on Swift object.
• | |/ /
• + | | Pull request 4576
• |\ \ \
• | + | | rgw: enforce Content-Length in response for POST on Swift cont/obj.
• | |/ /
• + | | Pull request 4579
• |\ \ \
• | + | | rgw: send Content-Length in response for HEAD on Swift account.
• | + | | rgw: send Content-Length in response for DELETE on Swift container.
• | + | | rgw: send Content-Length in response for PUT on Swift container.
• | + | | rgw: send Content-Length in response for GET on Swift container.
• | + | | rgw: enable end_header() to handle proposal of Content-Length.
• | |/ /
• + | | Pull request 4580
• |\ \ \
• | + | | rgw: fix handling empty metadata items on Swift container.
• | |/ /
• + | | Pull request 4581
• |\ \ \
• | + | | rgw: quota not respected in POST object
• | |/ /
• + | | Pull request 4566
• |\ \ \
• | + | | tests: AioCompletion incorrectly freed
• | + | | librbd: update ref count when queueing AioCompletion
• | |/ /
• + | | Pull request 4564
• |\ \ \
• | + | | librbd: failure to update the object map should always return success
• | + | | tests: librados_test_stub reads should deep-copy
• | |/ /
• + | | Pull request 4585
• |\ \ \
• | + | | librbd: flatten should return -EROFS if image is read-only
• | + | | librbd: allow snapshots to be created when snapshot is active
• | + | | librbd: better handling for duplicate flatten requests
• | + | | librbd: use generic helper for issuing async requests
• + | | | Pull request 4618
• |\ \ \ \
• | + | | | tests: ceph-helpers kill_daemons fails when kill fails
• |/ / / /
• + | | | Added a ceph hello world for a simple check for ceph-deploy qa suite
• + | | | Pull request 4502
• |\ \ \ \
• | + | | | osdc: add epoch_t last_force_resend in Op/LingerOp.
• + | | | | Pull request 4562
• |\ \ \ \ \
• | + | | | | packaging: mv ceph-objectstore-tool to main ceph pkg
• | | |_|/ /
• | |/| | |
• + | | | | Pull request 4561
• |\ \ \ \ \
• | + | | | | Fix disk zap sgdisk invocation
• | |/ / / /
• + | | | | Pull request 4560
• |\ \ \ \ \
• | + | | | | osd: refuse to write a new erasure coded object with an offset > 0
• | |/ / / /
• + | | | | Pull request 4559
• |\ \ \ \ \
• | + | | | | OSD: handle the case where we resurrected an old, deleted pg
• | |/ / / /
• + | | | | Pull request 4563
• |\ \ \ \ \
• | + | | | | mon: Total size of OSDs is a maginitude less than it is supposed to be.
• | | |_|/ /
• | |/| | |
• + | | | | Pull request 4577
• |\ \ \ \ \
• | |/ / / /
• |/| | | |
• | + | | | rgw: Swift API. Complement the response to show container details
• | |/ / /
• + | | | Pull request 4481
• |\ \ \ \
• | |/ / /
• |/| | |
• | + | | mds: remove caps from revoking list when caps are voluntarily released
• | |/ /
• + | | ceph_json: add decode / encoder for multimap
• + | | cls_rgw: use multimap to keep pending operations in bucket index
• + | | rgw: generate new tag for object when setting object attrs
• + | | java: libcephfs_jni.so is in /usr/lib64 on rhel
• + | | Pull request 4498
• |\ \ \
• | + | | librbd: moved snap_create header update notification to initiator
• | |/ /
• + | | Pull request 4497
• |\ \ \
• | + | | librbd: updated cache max objects calculation
• | |/ /
• + | | Pull request 4499
• |\ \ \
• | + | | librbd: ImageWatcher should cancel in-flight ops on watch error
• | |/ /
• + | | Pull request 4496
• |\ \ \
• | + | | librbd: acquire cache_lock before refreshing parent
• | + | | librados_test_stub: AIO operation callbacks should be via Finisher
• | |/ /
• + | | Pull request 4505
• |\ \ \
• | + | | rgw : Issue AIO for next chunk first before flush the (cached) data.
• | |/ /
• + | | Pull request 4501
• |\ \ \
• | + | | rgw: do not pre-fetch data for HEAD requests
• | |/ /
• + | | Pull request 4500
• |\ \ \
• | + | | rgw - make starting quota/gc threads configurable
• | |/ /
• + | | Pull request 4504
• |\ \ \
• | + | | Move ceph-dencoder build to client
• | + | | Rework mds/Makefile.am to support a dencoder client build
• | + | | rgw/Makefile.am: Populate DENCODER_SOURCES properly
• | + | | Dencoder should never be built with tcmalloc
• | |/ /
• + | | Pull request 4503
• |\ \ \
• | + | | ceph-disk: more robust parted output parser
• | |/ /
• + | | Pull request 4507
• |\ \ \
• | + | | Add support for PPC architecture, provide fallback
• | |/ /
• + | | Pull request 4515
• |\ \ \
• | + | | rgw-admin: a tool to fix object locator issue
• | + | | rgw: set a special object locator if object starts with underscore
• | + | | rgw: use correct oid when creating gc chains
• | + | | rgw: civetweb should use unique request id
• | |/ /
• + | | Pull request 4462
• |\ \ \
• | + | | osd/ReplicatedPG: don't check order in finish_proxy_read
• + | | | Pull request 4506
• |\ \ \ \
• | |_|/ /
• |/| | |
• | + | | Fix clear_pipe after reaping progress
• |/ / /
• + | | Pull request 4463
• |\ \ \
• | |/ /
• |/| |
• | + | init-radosgw: run RGW as root
• |/ /
• + | Fix ceph_test_async_driver failed
• |/
• + rgw: remove meta file after deleting bucket The meta file is deleted only if the bucket meta data is not synced
• + Pull request 4332
• |\
• | + bug fix: test case for lfn index
• + Pull request 4357
• + release-notes: backport Hammer release notes

Teuthology run commit commit:823da44e24ce95c0afa9dd1be8d1df4f33396e2d (hammer-backports branch May 2015)¶

• + Merge 4696: Wip 11622 hammer
• + rgw: merge manifests correctly when there's prefix override
• + rgw: restore buffer of multipart upload after EEXIST

rgw¶

Note: the wip-11622-hammer branch contains the same commits as hammer-backports

filter='rgw/multifs/{overrides.yaml clusters/fixed-2.yaml frontend/civetweb.yaml fs/xfs.yaml rgw_pool_type/ec-cache.yaml tasks/rgw_readwrite.yaml}'
./virtualenv/bin/teuthology-suite --priority 101 --suite rgw --filter="$filter" --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --ceph wip-11622-hammer

• green http://pulpito.ceph.com/loic-2015-05-18_11:02:32-rgw-wip-11622-hammer---basic-multi/

filter='rgw/multifs/{overrides.yaml clusters/fixed-2.yaml frontend/civetweb.yaml fs/xfs.yaml rgw_pool_type/ec-cache.yaml tasks/rgw_readwrite.yaml}'
./virtualenv/bin/teuthology-suite --priority 101 --suite rgw --filter="$filter" --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --ceph wip-11622-hammer

• red http://pulpito.ceph.com/loic-2015-05-18_00:59:08-rgw-wip-11622-hammer---basic-multi/
  • known bug (unrelated to rgw) osd/ReplicatedPG.cc: 2141: FAILED assert

• red http://pulpito.ceph.com/loic-2015-05-16_01:48:25-rgw-wip-11622-hammer---basic-multi/
  • environmental noise rgw/multifs/{overrides.yaml clusters/fixed-2.yaml frontend/civetweb.yaml fs/xfs.yaml rgw_pool_type/ec-cache.yaml tasks/rgw_readwrite.yaml} (the machine apparently died, the log end abruptly)

Teuthology run commit commit:a79146fc3cae28bf4c07478fb4566b06942da60d (hammer-backports branch May 2015)¶

rados¶

Note: rados suite must run on branch wip-11429-hammer because it is required by OSD::load_pgs: we need to handle the case where an upgrade from earlier versions which ignored non-existent pgs resurrects a pg with a prehistoric osdmap

• green http://pulpito.ceph.com/loic-2015-05-07_09:34:21-rados-hammer-backports---basic-multi/

fs¶

Note: the following run contains https://github.com/ceph/ceph/pull/4629 in addition to the commits above. It was scheduled after they were merged, except for one pull request related to rgw and with no consequence on fs testing (https://github.com/ceph/ceph/pull/4568).

• green http://pulpito.ceph.com/loic-2015-05-09_11:11:23-fs-hammer-uclient-checking-testing-basic-multi/

• red http://pulpito.ceph.com/loic-2015-05-07_09:44:39-fs-hammer-backports-testing-basic-multi/
  • environmental noise sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes ... or hung
    • fs/basic/{clusters/fixed-3-cephfs.yaml debug/mds_client.yaml fs/btrfs.yaml inline/yes.yaml overrides/whitelist_wrongly_marked_down.yaml tasks/mds_creation_retry.yaml}
    • fs/basic/{clusters/fixed-3-cephfs.yaml debug/mds_client.yaml fs/btrfs.yaml inline/no.yaml overrides/whitelist_wrongly_marked_down.yaml tasks/cfuse_workunit_suites_ffsb.yaml}

rgw¶

• green http://pulpito.ceph.com/loic-2015-05-07_09:21:24-rgw-hammer-backports---basic-multi/

rbd¶

• red http://pulpito.ceph.com/loic-2015-05-07_09:46:27-rbd-hammer-backports---basic-multi
  • can be ignored librbd: crash when two clients try to write to an exclusive locked image rare enough, pre-existing)
    • rbd/thrash/{base/install.yaml clusters/fixed-2.yaml fs/xfs.yaml msgr-failures/few.yaml thrashers/default.yaml workloads/rbd_fsx_cache_writeback.yaml}

Teuthology run commit d7828f7a8dcff7ec5a1a8458e6a610c0a07867ca (hammer-backports branch May 2015)¶

• + Merge 4515: rgw: critical fixes for hammer
• + Merge 4496: librbd: deadlock in image refresh
• |\
• | + librbd: acquire cache_lock before refreshing parent
• | + librados_test_stub: AIO operation callbacks should be via Finisher
• + Merge 4497: Objectcacher setting max object counts too low
• |\
• | + librbd: updated cache max objects calculation
• + Merge 4498: librbd notification race condition on snap_create
• |\
• | + librbd: moved snap_create header update notification to initiator
• + Merge 4499: ImageWatcher should cancel in-flight ops on watch error
• |\
• | + librbd: ImageWatcher should cancel in-flight ops on watch error
• + Merge 4500: rgw : make quota/gc thread configurable for starting
• |\
• | + rgw - make starting quota/gc threads configurable
• + Merge 4501: Improve rgw HEAD request by avoiding read the body of the first chunk
• |\
• | + rgw: do not pre-fetch data for HEAD requests
• + Merge 4502: objecter: can get stuck in redirect loop if osdmap epoch == last_force_op_resend
• |\
• | + osdc: add epoch_t last_force_resend in Op/LingerOp.
• + Merge 4503: use a new disk as journal disk,ceph-disk prepare fail
• |\
• | + ceph-disk: more robust parted output parser
• + Merge 4504: ceph-dencoder links to libtcmalloc, and shouldn't
• |\
• | + Move ceph-dencoder build to client
• | + Rework mds/Makefile.am to support a dencoder client build
• | + rgw/Makefile.am: Populate DENCODER_SOURCES properly
• | + Dencoder should never be built with tcmalloc
• + Merge 4505: rgw - improve performance for large object (multiple chunks) GET
• |\
• | + rgw : Issue AIO for next chunk first before flush the (cached) data.
• + Merge 4507: compilation error: No high-precision counter available
• + Add support for PPC architecture, provide fallback

rgw¶

./virtualenv/bin/teuthology-suite --priority 101 --suite rgw --filter-out btrfs,ext4,apache --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email loic@dachary.org --owner loic@dachary.org --ceph hammer-backports

• green http://pulpito.ceph.com/loic-2015-05-03_14:50:25-rgw-hammer-backports---basic-multi/

fs¶

run=loic-2015-05-04_11:27:13-fs-hammer-backports---basic-multi
filter='fs/recovery/{clusters/2-remote-clients.yaml debug/mds_client.yaml mounts/ceph-fuse.yaml tasks/mds-flush.yaml}'
./virtualenv/bin/teuthology-suite --filter="$filter" -k testing --priority 101 --suite fs --suite-branch wip-10712-hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --owner loic@dachary.org --ceph hammer-backports

• green http://pulpito.ceph.com/loic-2015-05-05_11:54:53-fs-hammer-backports-testing-basic-multi/

run=loic-2015-05-03_14:58:27-fs-hammer-backports---basic-multi
eval filter=$(curl --silent http://paddles.front.sepia.ceph.com/runs/$run/jobs/?status=fail | jq '.[].description' | while read description ; do echo -n $description, ; done | sed -e 's/,$//')
./virtualenv/bin/teuthology-suite --filter="$filter" --priority 101 --suite fs --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --owner loic@dachary.org --ceph hammer-backports

• red http://pulpito.ceph.com/loic-2015-05-04_11:27:13-fs-hammer-backports---basic-multi/
  • TestFlush intermittent failure on scatter_writebehind event Test failure: test_flush (tasks.mds_flush.TestFlush)
    • fs/recovery/{clusters/2-remote-clients.yaml debug/mds_client.yaml mounts/ceph-fuse.yaml tasks/mds-flush.yaml}

./virtualenv/bin/teuthology-suite --priority 101 --suite fs --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --owner loic@dachary.org --ceph hammer-backports

• red http://pulpito.ceph.com/loic-2015-05-03_14:58:27-fs-hammer-backports---basic-multi/
  • Test failure: test_flush (tasks.mds_flush.TestFlush)
    • fs/recovery/{clusters/2-remote-clients.yaml debug/mds_client.yaml mounts/ceph-fuse.yaml tasks/mds-flush.yaml}
  • Test failure: test_client_pin (tasks.mds_client_limits.TestClientLimits)
    • fs/recovery/{clusters/2-remote-clients.yaml debug/mds_client.yaml mounts/ceph-fuse.yaml tasks/client-limits.yaml}

rbd¶

./virtualenv/bin/teuthology-suite --priority 101 --suite rbd --filter-out btrfs,ext4 --suite-branch hammer --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --owner loic@dachary.org --ceph hammer-backports

• green http://pulpito.ceph.com/loic-2015-05-03_14:55:24-rbd-hammer-backports---basic-multi/

rados¶

$ git ls-remote https://github.com/dachary/ceph-qa-suite.git wip-rados-no-thrash-hammer
8233eae8329d1ef80dece59a2fe0518c13e51135    refs/heads/wip-rados-no-thrash-hammer
$ git clone -b wip-rados-no-thrash-hammer https://github.com/dachary/ceph-qa-suite.git /tmp/ceph-qa-suite
$ ./virtualenv/bin/teuthology-suite --filter-out ext4,btrfs --priority 101 --suite rados --suite-dir /tmp/ceph-qa-suite --machine-type plana,burnupi,mira --distro ubuntu --email abhishek.lekshmanan@gmail.com --owner loic@dachary.org  --ceph hammer-backports

• red http://pulpito.ceph.com/loic-2015-05-04_11:18:05-rados-hammer-backports---basic-multi/
  • manager
    • rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml workloads/ec-small-objects.yaml}
    • rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml workloads/ec-radosbench.yaml}
    • rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml}
    • rados/thrash-erasure-code/{clusters/fixed-2.yaml fs/xfs.yaml workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml}

QE Validation¶