Ceph

/a/master-2012-01-27_13:29:47/9361

2012-01-27T16:23:52.280 INFO:teuthology.task.ceph:Shutting down osd daemons...
2012-01-27T16:23:52.280 INFO:teuthology.task.ceph:Shutting down mon daemons...
2012-01-27T16:23:52.282 INFO:teuthology.task.ceph.mon.a.err:*** Caught signal (Terminated) **
2012-01-27T16:23:52.282 INFO:teuthology.task.ceph.mon.a.err: in thread 7f8993043780. Shutting down.
2012-01-27T16:23:52.295 INFO:teuthology.task.ceph.mon.a:mon.a: Stopped
2012-01-27T16:23:52.328 INFO:teuthology.task.ceph.mon.c.err:*** Caught signal (Terminated) **
2012-01-27T16:23:52.328 INFO:teuthology.task.ceph.mon.c.err: in thread 7f0cbd3cc780. Shutting down.
2012-01-27T16:23:52.341 INFO:teuthology.task.ceph.mon.c:mon.c: Stopped
2012-01-27T16:23:52.342 INFO:teuthology.task.ceph.mon.b.err:*** Caught signal (Terminated) **
2012-01-27T16:23:52.343 INFO:teuthology.task.ceph.mon.b.err: in thread 7fb9c5f48780. Shutting down.
2012-01-27T16:23:52.363 INFO:teuthology.task.ceph.mon.b:mon.b: Stopped
2012-01-27T16:23:52.363 INFO:teuthology.task.ceph:Grabbing cluster log from ubuntu@sepia5.ceph.dreamhost.com mon.a...
2012-01-27T16:23:52.363 DEBUG:teuthology.orchestra.run:Running: 'cat -- /tmp/cephtest/data/mon.a/log'
2012-01-27T16:23:52.385 INFO:teuthology.task.ceph:Checking cluster ceph.log for badness...
2012-01-27T16:23:52.385 DEBUG:teuthology.orchestra.run:Running: "egrep '\\[ERR\\]|\\[WRN\\]|\\[SEC\\]' /tmp/cephtest/data/mon.a/log | egrep -v 'clocks not synchronized' | head -n 1" 
2012-01-27T16:23:52.428 INFO:teuthology.task.ceph:Unmounting /tmp/cephtest/data/osd.0.data on ubuntu@sepia5.ceph.dreamhost.com
2012-01-27T16:23:52.428 DEBUG:teuthology.orchestra.run:Running: 'sudo umount -f /tmp/cephtest/data/osd.0.data'
2012-01-27T16:23:52.491 INFO:teuthology.orchestra.run.err:umount2: Device or resource busy
2012-01-27T16:23:52.492 INFO:teuthology.orchestra.run.err:umount: /tmp/cephtest/data/osd.0.data: device is busy.
2012-01-27T16:23:52.492 INFO:teuthology.orchestra.run.err:        (In some cases useful info about processes that use
2012-01-27T16:23:52.492 INFO:teuthology.orchestra.run.err:         the device is found by lsof(8) or fuser(1))
2012-01-27T16:23:52.492 INFO:teuthology.orchestra.run.err:umount2: Device or resource busy

Note that there is a "shutting down osd daemons..." but no

2012-01-27T14:57:32.944 INFO:teuthology.task.ceph:Shutting down osd daemons...
2012-01-27T14:57:33.306 INFO:teuthology.task.ceph.osd.1:osd.1: Stopped
2012-01-27T14:57:33.332 INFO:teuthology.task.ceph.osd.0:osd.0: Stopped

that is present in successful runs. Is there a join/wait missing?

seeing this pretty often, causing qa failure noise...