Ceph

Another "my OSDs don't come up on reboot" bug:

ceph-disk unconditionally calls setup_statedir() during initialisation, which manipulates /var/lib/ceph/tmp :

def setup_statedir(dir):
    # XXX The following use of globals makes linting
    # really hard. Global state in Python is iffy and
    # should be avoided.
    global STATEDIR
    STATEDIR = dir

    if not os.path.exists(STATEDIR):
        os.mkdir(STATEDIR)
    if not os.path.exists(STATEDIR + "/tmp"):
        os.mkdir(STATEDIR + "/tmp")

    global prepare_lock
    prepare_lock = FileLock(STATEDIR + '/tmp/ceph-disk.prepare.lock')

    global activate_lock
    activate_lock = FileLock(STATEDIR + '/tmp/ceph-disk.activate.lock')
...

This may fail for users with a separately mounted /var filesystem, where /var hasn't been mounted prior to 95-ceph-osd.rules being triggered.