Ceph » RADOS

I have an Octopus (15.2.4) cluster with degraded and unfound objects, and PGs that have been stuck in the degraded and recovery_unfound states for several weeks. While debugging this issue on the ceph-users mailing list (https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/QHFOGEKXK7VDNNSKR74BA6IIMGGIXBXA/), I found that osdmaptool would consistenly core dump when running osdmaptool on the pool that contains the affected PGs:

# osdmaptool osd.map --test-map-pgs-dump --pool 7
Output: https://pastebin.com/HPtSqcS1

Of note is is one PG (7.39d) that has a missing OSD in the active set. We have a hunch that there is some internal corruption that is causing osdmaptool to core dump, and may also be the cause of the PGs stuck in unfound/degraded.