Ceph » RADOS

In looking at https://tracker.ceph.com/issues/23562, there were inactive PGs starting at

2018-04-04 16:57:43.702801 mon.reesi001 mon.0 10.8.130.101:6789/0 113 : cluster [WRN] Health check failed: Reduced data availability: 81 pgs inactive, 91 pgs peering (PG_AVAILABILITY)

immediately after the VDO OSD was turned on.
It settled down at

2018-04-04 16:59:38.517021 mon.reesi001 mon.0 10.8.130.101:6789/0 163 : cluster [WRN] overall HEALTH_WARN 463121/13876873 objects misplaced (3.337%); Reduced data availability: 61 pgs inactive; Degraded data redundancy: 14/13876873 objects degraded (0.000%), 152 pgs unclean, 6 pgs degraded; too many PGs per OSD (240 > max 200); clock skew detected on mon.reesi002, mon.reesi003

And then stayed pretty much that way indefinitely. It eventually transitioned to a HEALTH_ERR a couple hours later

2018-04-04 18:27:38.532992 mon.reesi001 mon.0 10.8.130.101:6789/0 1476 : cluster [WRN] overall HEALTH_WARN 405697/13877083 objects misplaced (2.924%); Reduced data availability: 61 pgs inactive; Degraded data redundancy: 13/13877083 objects degraded (0.000%), 139 pgs unclean, 2 pgs degraded; 1 slow requests are blocked > 32 sec; too many PGs per OSD (240 > max 200); clock skew detected on mon.reesi002, mon.reesi003
2018-04-04 18:28:38.533153 mon.reesi001 mon.0 10.8.130.101:6789/0 1494 : cluster [ERR] overall HEALTH_ERR 405508/13877089 objects misplaced (2.922%); Reduced data availability: 61 pgs inactive; Degraded data redundancy: 13/13877089 objects degraded (0.000%), 139 pgs unclean, 2 pgs degraded; 1 stuck requests are blocked > 4096 sec; too many PGs per OSD (240 > max 200); clock skew detected on mon.reesi002, mon.reesi003

But that seems to have been caused by the creation of stuck requests, not the inactive PGs.

I'm not quite sure what's going on here. Perhaps we only transition to HEALTH_ERR when PGs get stuck, but the primary for these inactive PGs was still sending in PGStats messages so that never happened?