Project

General

Profile

Actions

Bug #23228

closed

scrub mismatch on objects

Added by Sage Weil about 6 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

[ERR] 2.12s1 scrub stat mismatch, got 6/5 objects, 4/3 clones, 6/5 dirty, 0/0 omap, 0/0 pinned, 0/0 hit_set_archive, 0/0 whiteouts, 13601636/10392205 bytes, 0/0 hit_set_archive bytes." in cluster log

/a/sage-2018-03-05_15:17:42-rados-wip-sage3-testing-2018-03-04-1532-distro-basic-smithi/2252411

Related issues 2 (1 open1 closed)

Related to RADOS - Bug #22656: scrub mismatch on bytes (cache pools)New

Actions
Related to RADOS - Bug #23339: Scrub errors after ec-small-objects-overwrites testResolvedSage Weil03/13/2018

Actions
Actions #1

Updated by Sage Weil about 6 years ago

  • Project changed from Ceph to RADOS
Actions #2

Updated by David Zafman about 6 years ago

/a/dzafman-2018-03-21_09:57:19-rados:thrash-wip-zafman-testing2-distro-basic-smithi/2312125

rados:thrash/{0-size-min-size-overrides/3-size-2-min-size.yaml 1-pg-log-overrides/short_pg_log.yaml backoff/peering_and_degraded.yaml ceph.yaml clusters/{fixed-2.yaml openstack.yaml} d-balancer/upmap.yaml msgr-failures/osd-delay.yaml msgr/simple.yaml objectstore/filestore-xfs.yaml rados.yaml rocksdb.yaml thrashers/morepggrow.yaml thrashosds-health.yaml workloads/small-objects.yaml}

Actions #3

Updated by David Zafman about 6 years ago

Just bytes

dzafman-2018-03-28_18:21:29-rados-wip-zafman-testing-distro-basic-smithi/2332093

[ERR] 3.0 scrub stat mismatch, got
51/51 objects, 4/4 clones, 24/24 dirty, 0/0 omap, 0/0 pinned, 1/1 hit_set_archive,
7/7 whiteouts, 119951885/120679921 bytes, 0/0 manifest objects, 269/269 hit_set_archive
bytes

UPDATE: this is cache tiering and bytes, moved to #22656

Actions #4

Updated by Greg Farnum about 6 years ago

  • Related to Bug #22656: scrub mismatch on bytes (cache pools) added
Actions #5

Updated by Sage Weil about 6 years ago

  • Related to Bug #23339: Scrub errors after ec-small-objects-overwrites test added
Actions #6

Updated by Sage Weil about 6 years ago

The original report was an EC test, so it looks like a dup of #23339.

David, your failures are not EC. Could they be from your testing branch? I don't think I've seen them in master.

Actions #7

Updated by David Zafman about 6 years ago

My change only affects the scrub error counts in the stats. However, if setting dirty_info in proc_primary_info() would cause wrong stats to be written along with the scrub change, then that could cause this problem. So this could be caused by my change, but I'm not sure how it would be fixed.

If this is because dirty_info is set it would mean this is a latent defect where if history or purge_snaps need to be updated then we could run into this too.

Actions #8

Updated by Sage Weil about 6 years ago

  • Status changed from 12 to Closed

The failure in comment (2) looks unrelated, but i twas a test branch. let's see if it happens again.

The original issue was a dup of #23339

Actions

Also available in: Atom PDF