Ceph

I had some vm's with the qemu-rbd driver doing a trim operation. One of my osd's crashed and now has an inconsistent pg following restart. OSD 14 was the primary and crashed while OSD 6 is the secondary. The missing objects are like this:

2.37d osd.14 missing f30b0f7d/rb.0.105b.238e1f29.000000000ff4/head//2

However it appears to be on disk on both the primary and secondary, just in different places in the directory tree depending on the OSD

find /data/osd.14/current/2.37d_head/ -name 'rb.0.105b.238e1f29.000000000ff4*'
/data/osd.14/current/2.37d_head/DIR_D/rb.0.105b.238e1f29.000000000ff4__head_F30B0F7D__2

find /data/osd.6/current/2.37d_head/ -name 'rb.0.105b.238e1f29.000000000ff4*'
/data/osd.6/current/2.37d_head/DIR_D/DIR_7/rb.0.105b.238e1f29.000000000ff4__head_F30B0F7D__2