Ceph

Random ceph-osd processes doesn't get started up when the OSD node restarts. This can happen in any of the OSD nodes, and for any random OSD.

The OSD however is mounted at the mount point under /var/lib/ceph/osd/. Manually starting the OSD process after the machine is up and running, works fine.

/var/log/messages have logs on other OSD processes starting, but not anything on the ones that are not. No crashes, or anything that indicates a problem.

This is Hammer on RHEL7.1.

~~~
$ lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 7.1 (Maipo)
Release: 7.1
Codename: Maipo

$ uname -a
Linux <hostname> 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Aug 25 11:21:22 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

$ rpm -qa | grep ceph
ceph-0.94.3-0.el7.x86_64
libcephfs1-0.94.3-0.el7.x86_64
python-cephfs-0.94.3-0.el7.x86_64
ceph-common-0.94.3-0.el7.x86_64
~~~

Attaching /var/log/messages, as an example since the problem was reproducible on this OSD for the latest reboot.