Ceph

Steps:

1) install stable jewel release
2) upgrade to dev branch of luminous
3) try starting/stopping various ceph daemons
on one of the nodes following issue is seen

2017-10-12T23:33:54.066 INFO:teuthology.orchestra.run.vpm199:Running: 'sudo systemctl start ceph-mgr@vpm199.service'
2017-10-12T23:33:58.153 INFO:teuthology.orchestra.run.vpm141:Running: 'sudo ps -eaf | grep ceph'
2017-10-12T23:33:58.180 INFO:teuthology.orchestra.run.vpm141.stdout:ubuntu   25706 16821  0 23:33 ?        00:00:00 bash -c sudo ps -eaf | grep ceph
2017-10-12T23:33:58.180 INFO:teuthology.orchestra.run.vpm141.stdout:ubuntu   25708 25706  0 23:33 ?        00:00:00 grep ceph
2017-10-12T23:33:58.180 INFO:teuthology.orchestra.run.vpm141.stdout:ceph     26662     1  0 23:15 ?        00:00:03 /usr/bin/ceph-osd -f --cluster ceph --id 3 --setuser ceph --setgroup ceph
2017-10-12T23:33:58.180 INFO:teuthology.orchestra.run.vpm141.stdout:ceph     28126     1  0 23:15 ?        00:00:03 /usr/bin/ceph-osd -f --cluster ceph --id 4 --setuser ceph --setgroup ceph
2017-10-12T23:33:58.180 INFO:teuthology.orchestra.run.vpm141.stdout:ceph     29584     1  0 23:15 ?        00:00:02 /usr/bin/ceph-osd -f --cluster ceph --id 5 --setuser ceph --setgroup ceph
2017-10-12T23:33:58.181 INFO:teuthology.orchestra.run.vpm141:Running: 'sudo systemctl list-units | grep ceph'
2017-10-12T23:33:58.289 INFO:teuthology.orchestra.run.vpm141.stdout:  var-lib-ceph-osd-ceph\x2d3.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-3
2017-10-12T23:33:58.289 INFO:teuthology.orchestra.run.vpm141.stdout:  var-lib-ceph-osd-ceph\x2d4.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-4
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  var-lib-ceph-osd-ceph\x2d5.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-5
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-osd@3.service                                                                  loaded    active running   Ceph object storage daemon
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-osd@4.service                                                                  loaded    active running   Ceph object storage daemon
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-osd@5.service                                                                  loaded    active running   Ceph object storage daemon
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  system-ceph\x2ddisk.slice                                                           loaded    active active    system-ceph\x2ddisk.slice
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  system-ceph\x2dosd.slice                                                            loaded    active active    system-ceph\x2dosd.slice
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-mds.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-mds@.service instances at once
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-mon.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-mon@.service instances at once
2017-10-12T23:33:58.290 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-osd.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-osd@.service instances at once
2017-10-12T23:33:58.291 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph-radosgw.target                                                                 loaded    active active    ceph target allowing to start/stop all ceph-radosgw@.service instances at once
2017-10-12T23:33:58.291 INFO:teuthology.orchestra.run.vpm141.stdout:  ceph.target                                                                         loaded    active active    ceph target allowing to start/stop all ceph*@.service instances at once
2017-10-12T23:33:58.291 INFO:tasks.systemd:  var-lib-ceph-osd-ceph\x2d3.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-3
  var-lib-ceph-osd-ceph\x2d4.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-4
  var-lib-ceph-osd-ceph\x2d5.mount                                                    loaded    active mounted   /var/lib/ceph/osd/ceph-5
  ceph-osd@3.service                                                                  loaded    active running   Ceph object storage daemon
  ceph-osd@4.service                                                                  loaded    active running   Ceph object storage daemon
  ceph-osd@5.service                                                                  loaded    active running   Ceph object storage daemon
  system-ceph\x2ddisk.slice                                                           loaded    active active    system-ceph\x2ddisk.slice
  system-ceph\x2dosd.slice                                                            loaded    active active    system-ceph\x2dosd.slice
  ceph-mds.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-mds@.service instances at once
  ceph-mon.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-mon@.service instances at once
  ceph-osd.target                                                                     loaded    active active    ceph target allowing to start/stop all ceph-osd@.service instances at once
  ceph-radosgw.target                                                                 loaded    active active    ceph target allowing to start/stop all ceph-radosgw@.service instances at once
  ceph.target                                                                         loaded    active active    ceph target allowing to start/stop all ceph*@.service instances at once

2017-10-12T23:33:58.291 INFO:tasks.systemd:Ceph services in failed state
2017-10-12T23:33:58.291 INFO:tasks.systemd:Stopping all Ceph services
2017-10-12T23:33:58.291 INFO:teuthology.orchestra.run.vpm141:Running: 'sudo systemctl stop ceph.target'
2017-10-12T23:33:58.310 INFO:teuthology.orchestra.run.vpm141.stderr:Failed to stop ceph.target: Transaction order is cyclic. See system logs for details.
2017-10-12T23:33:58.310 INFO:teuthology.orchestra.run.vpm141.stderr:See system logs and 'systemctl status ceph.target' for details.
2017-10-12T23:33:58.311 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):