Ceph

wip-7215-quorum-features
Going to have to do some manual testing; not sure if there's any feasible automated testing we can set up or should perform on this. :/

Greg, we have some tests in the upgrade:parallel suite [ceph-qa-suite/suites/upgrade/parallel/stress-split] to do this,

the config file would look like:

- chef: null
- clock.check: 
- install:
    branch: dumpling
- ceph:
    fs: xfs
- install.upgrade:
    osd.0: next
- ceph.restart:
    daemons:
    - osd.0
    - osd.1
    - osd.2
- thrashosds:
    chance_pgnum_grow: 1
    chance_pgpnum_fix: 1
    timeout: 1200
- ceph.restart:
    daemons:
    - mon.a
    wait-for-healthy: false
    wait-for-osds-up: true
- rados:
    clients:
    - client.0
    objects: 50
    op_weights:
      delete: 50
      read: 100
      rollback: 50
      snap_create: 50
      snap_remove: 50
      write: 100
    ops: 4000
- ceph.restart:
    daemons:
    - mon.b
    wait-for-healthy: false
    wait-for-osds-up: true
- workunit:
    branch: dumpling
    clients:
      client.0:
      - rados/test.sh
- install.upgrade:
    mon.c: null
- ceph.restart:
    daemons:
    - mon.c
    wait-for-healthy: false
    wait-for-osds-up: true
- ceph.wait_for_mon_quorum:
  - a
  - b
  - c
- workunit:
    branch: dumpling
    clients:
      client.0:
      - rados/test.sh

Nifty, thanks Tamil! That should let me cover a bunch of it.