Project

General

Profile

Actions

Bug #63307

closed

crimson: SnapTrimObjSubEvent doesn't actually seem to submit delta_stats

Added by Samuel Just 6 months ago. Updated 19 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
good-first-issue
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

This tends to result in stat mismatches in scrub:

Create a vstart cluster, run

```
./bin/ceph osd pool delete $POOL_NAME $POOL_NAME --yes-i-really-really-mean-it
./bin/ceph osd pool create $POOL_NAME 1 1 replicated replicated_rule 2 2 2
./bin/ceph_test_rados --max-ops 1000 --objects 64 --max-in-flight 32 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 0 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 0 --op copy_from 0 --op write_excl 0 --pool $POOL_NAME
```

identify the pgid for the single PG in the above pool. Run

```
./bin/ceph tell <pgid> scrub
```

The log for the primary should contain log lines like:

```
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result:
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_objects: scrubbed value: 49, stored pg value: 250
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_object_clones: scrubbed value: 9, stored pg value: 241
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_whiteouts: scrubbed value: 4, stored pg value: 81
```


Related issues 1 (1 open0 closed)

Precedes crimson - Bug #65113: crimson: SnapTrimObjSubEvent num_bytes stats calculationFix Under Review

Actions
Actions #1

Updated by Radoslaw Zarzynski 6 months ago

  • Project changed from RADOS to crimson
Actions #2

Updated by Matan Breizman 3 months ago

SnapTrimObjSubEvent::remove_or_update:

      // Stats reporting - Set number of objects trimmed
      if (num_objects_before_trim > delta_stats.num_objects) {
    //int64_t num_objects_trimmed =
    //  num_objects_before_trim - delta_stats.num_objects;
    //add_objects_trimmed_count(num_objects_trimmed);
      }

Actions #3

Updated by Matan Breizman 3 months ago

  • Tags set to good-first-issue
Actions #4

Updated by junxiang mu about 2 months ago

let me try solve this issue

Actions #5

Updated by Matan Breizman about 1 month ago

  • Precedes Bug #65113: crimson: SnapTrimObjSubEvent num_bytes stats calculation added
Actions #6

Updated by Matan Breizman about 1 month ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 56378
Actions #7

Updated by Matan Breizman 19 days ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF