Actions
Bug #41526
closedChoosing the next PG for a deep scrubs wrong.
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
I have ceph cluster in this state:
# ceph health detail HEALTH_WARN 27 pgs not deep-scrubbed in time; PG_NOT_DEEP_SCRUBBED 27 pgs not deep-scrubbed in time pg 15.e4 not deep-scrubbed since 2019-08-14 19:15:30.699834 pg 2.db not deep-scrubbed since 2019-08-14 14:14:25.173851 pg 15.d8 not deep-scrubbed since 2019-08-14 20:19:03.937229 pg 9.d9 not deep-scrubbed since 2019-08-14 19:40:16.157361 pg 9.db not deep-scrubbed since 2019-08-15 04:24:14.865325 pg 2.5c not deep-scrubbed since 2019-08-14 20:22:04.605225 pg 17.45 not deep-scrubbed since 2019-08-15 01:43:06.099446 pg 6.51 not deep-scrubbed since 2019-08-14 13:22:51.959783 pg 6.4a not deep-scrubbed since 2019-08-15 03:39:44.701350 pg 2.4d not deep-scrubbed since 2019-08-14 23:54:40.245206 pg 7.33 not deep-scrubbed since 2019-08-15 03:32:24.927287 pg 17.27 not deep-scrubbed since 2019-08-14 14:15:42.543503 pg 17.17 not deep-scrubbed since 2019-08-14 23:09:42.728755 pg 6.3 not deep-scrubbed since 2019-08-14 14:10:45.757717 pg 10.12 not deep-scrubbed since 2019-08-15 03:11:14.487778 pg 17.c not deep-scrubbed since 2019-08-14 22:43:25.739869 pg 9.2f not deep-scrubbed since 2019-08-15 01:20:00.682925 pg 10.22 not deep-scrubbed since 2019-08-14 19:50:51.555694 pg 17.67 not deep-scrubbed since 2019-08-14 06:07:33.732999 pg 6.79 not deep-scrubbed since 2019-08-15 03:28:21.388018 pg 10.86 not deep-scrubbed since 2019-08-14 23:56:32.100535 pg 2.92 not deep-scrubbed since 2019-08-15 04:20:55.337923 pg 9.93 not deep-scrubbed since 2019-08-15 01:06:08.958664 pg 2.9b not deep-scrubbed since 2019-08-15 04:28:54.398524 pg 6.a7 not deep-scrubbed since 2019-08-15 04:05:45.886923 pg 10.a7 not deep-scrubbed since 2019-08-14 12:34:06.675859 pg 17.a4 not deep-scrubbed since 2019-08-14 22:21:07.467654
But ceph select strange (random?) pg for deep scrubing.
I do next command:
ceph pg dump | awk '$1 ~ /[0-9a-f]+\.[0-9a-f]+/ {print $25, $26, $1}' | sort > b1
After time again:
ceph pg dump | awk '$1 ~ /[0-9a-f]+\.[0-9a-f]+/ {print $25, $26, $1}' | sort > b2
And now - diff between:
diff b1 b2 558d557 < 2019-08-17 03:24:30.058471 17.18 2821a2821 > 2019-08-27 11:21:32.744345 17.18
As you can see - for deep scrubing will be selected not so old pg 17.18 although there are older pg.
Actions