Bug #63307
closedcrimson: SnapTrimObjSubEvent doesn't actually seem to submit delta_stats
0%
Description
This tends to result in stat mismatches in scrub:
Create a vstart cluster, run
```
./bin/ceph osd pool delete $POOL_NAME $POOL_NAME --yes-i-really-really-mean-it
./bin/ceph osd pool create $POOL_NAME 1 1 replicated replicated_rule 2 2 2
./bin/ceph_test_rados --max-ops 1000 --objects 64 --max-in-flight 32 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 0 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 0 --op copy_from 0 --op write_excl 0 --pool $POOL_NAME
```
identify the pgid for the single PG in the above pool. Run
```
./bin/ceph tell <pgid> scrub
```
The log for the primary should contain log lines like:
```
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result:
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_objects: scrubbed value: 49, stored pg value: 250
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_object_clones: scrubbed value: 9, stored pg value: 241
ERROR 2023-10-24 21:29:28,451 [shard 0] osd - pg_epoch 1586 pg[15.0( v 1586'4230 (0'0,1586'4230] local-lis/les=1125/1126 n=250 ec=1125/1125 lis/c=1125/1125 les/c/f=1126/1126/0 sis=1125) [2,1] r=0 lpr=1125 lua=0'
0 crt=1586'4230 mlcod 1586'4230 active+clean+scrubbing PGScrubber::emit_scrub_result: stat mismatch for num_whiteouts: scrubbed value: 4, stored pg value: 81
```
Updated by Radoslaw Zarzynski 6 months ago
- Project changed from RADOS to crimson
Updated by Matan Breizman 3 months ago
SnapTrimObjSubEvent::remove_or_update:
// Stats reporting - Set number of objects trimmed
if (num_objects_before_trim > delta_stats.num_objects) {
//int64_t num_objects_trimmed =
// num_objects_before_trim - delta_stats.num_objects;
//add_objects_trimmed_count(num_objects_trimmed);
}
Updated by Matan Breizman about 1 month ago
- Precedes Bug #65113: crimson: SnapTrimObjSubEvent num_bytes stats calculation added
Updated by Matan Breizman about 1 month ago
- Status changed from New to Fix Under Review
- Pull request ID set to 56378
Updated by Matan Breizman 19 days ago
- Status changed from Fix Under Review to Resolved