Ceph

I have noticed when a scrub is running on an OSD that is backed by a JBOD spinning drive that has a single bad block (marked in S.M.A.R.T as 'Current_Pending_Sector' >1) causing the scrub to fail with an I/O error the OSD process will assert out.

Following the assert upstart will start the drive back up and the process will repeat again some time in the future.

As the drive only has a single bad block this can be repaired by doing a write to this block causing the Drive firmware to reallocate a reserved sector. I can do this manually using "hdparm --write-sector $SECTOR /dev/sdXX" but I think ceph should really be doing this in much the way a RAID card would do by reading a copy / parity from the other drives and writing back over the problem sector.