Actions
Bug #8752
closedfirefly: scrub/repair stat mismatch
Status:
Can't reproduce
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Two dozen PGs are in "active+clean+inconsistent" state.
Attempted "ceph pg repair" reports fixed error(s) but next scrub or deep-scrub reveal the same (or similar) problem.
All 12 OSDs were replaced but problem do not go away and it is not clear what to expect (corruption?) or how to recover. Please advise.
2014-07-06 09:44:42.632205 osd.1 [ERR] 20.e deep-scrub stat mismatch, got 3280/3280 objects, 0/0 clones, 1634/1634 dirty, 0/0 omap, 4/4 hit_set_archive, 1871/1871 whiteouts, 893452192/893452181 bytes. 2014-07-06 09:44:42.632212 osd.1 [ERR] 20.e deep-scrub 1 errors 2014-07-06 09:53:10.496110 osd.1 [ERR] 20.e repair stat mismatch, got 3281/3281 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1874/1874 whiteouts, 889179125/889179115 bytes. 2014-07-06 09:53:10.496176 osd.1 [ERR] 20.e repair 1 errors, 1 fixed 2014-07-06 16:24:06.753233 osd.1 [ERR] 20.e scrub stat mismatch, got 3330/3330 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1751/1751 whiteouts, 1994079525/1994079526 bytes. 2014-07-06 16:24:06.753237 osd.1 [ERR] 20.e scrub 1 errors 2014-07-06 16:32:03.587865 osd.1 [ERR] 20.e repair stat mismatch, got 3333/3333 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1751/1751 whiteouts, 2006662452/2006662455 bytes. 2014-07-06 16:32:03.587944 osd.1 [ERR] 20.e repair 1 errors, 1 fixed 2014-07-06 17:07:09.114170 osd.1 [ERR] 20.e scrub stat mismatch, got 3333/3333 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1751/1751 whiteouts, 2006662455/2006662452 bytes. 2014-07-06 17:07:09.114176 osd.1 [ERR] 20.e scrub 1 errors 2014-07-06 17:10:26.163036 osd.1 [ERR] 20.e repair stat mismatch, got 3333/3333 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1751/1751 whiteouts, 2006662455/2006662452 bytes. 2014-07-06 17:10:26.163211 osd.1 [ERR] 20.e repair 1 errors, 1 fixed 2014-07-06 19:55:31.549075 osd.1 [ERR] 20.e scrub stat mismatch, got 3334/3334 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1729/1729 whiteouts, 2201876424/2201876433 bytes. 2014-07-06 19:55:31.549079 osd.1 [ERR] 20.e scrub 1 errors 2014-07-06 20:16:19.560180 osd.1 [ERR] 20.e repair stat mismatch, got 3315/3315 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1725/1725 whiteouts, 2159604413/2159604425 bytes. 2014-07-06 20:16:19.560267 osd.1 [ERR] 20.e repair 1 errors, 1 fixed 2014-07-06 20:23:14.901710 osd.1 [ERR] 20.e repair stat mismatch, got 3222/3222 objects, 0/0 clones, 1635/1635 dirty, 0/0 omap, 4/4 hit_set_archive, 1719/1719 whiteouts, 1832294131/1832294124 bytes. 2014-07-06 20:23:14.901808 osd.1 [ERR] 20.e repair 1 errors, 1 fixed
# ceph pg map 20.e osdmap e53790 pg 20.e (20.e) -> up [1,8,12,4] acting [1,8,12,4]
All PGs seems to belong to replicated pool.
Actions