Ceph

Copy the email from @ceph-devel to open a tracker:

Hi Sage,
Another potential problem with scrub scheduling, as observed in our deployment (2PB cluster, 70% full), was that some PGs hadn't been scrubbed for 1.5 months, even we have the configuration to do deep scrubbing weekly.

With our deployment and percentage of full of the cluster, as well as the conservative setting for scrubbing (osd_max_scrubs = 1), one round of scrubbing would not finish in 1 one week, so that we properly should schedule that monthly (with weekly shallow scrubbing).

Another problem, is that currently the scheduling of scrub is optimized locally at each OSD, that is, for each PG this OSD acts as the primary, it selects the one which hasn't been scheduled to scrubbing longest, put it as the candidate and request scrub reserver from all replicas. Since each OSD can only have 1 active scrubbing, that active slot could potentially always occupied by a replica, as a result, the PG whose primary is this OSD, fail to schedule and left behind.

Is this issue worth an enhancement?
[sage] Good point. Yeah, I think it's definitely worth fixing!