Ceph » RADOS

/a/yuriw-2022-06-29_18:22:37-rados-wip-yuri2-testing-2022-06-29-0820-distro-default-smithi/6906226

The error looks similar to the one reported in an existing tracker: https://tracker.ceph.com/issues/49525
But the fix for the existing tracker is already merged and therefore this may be something new. In this
case the error was reported when the pg 3.bs0 was being scrubbed as shown below:

2022-06-30T04:02:32.039+0000 7f120425d700 20 osd.3 pg_epoch: 1051 pg[3.bs0( v 1042'663 (1031'653,1042'663] local-lis/les=1034/1035 n=36 ec=23/23 lis/c=1034/1034 les/c/f=1035/1035/0 sis=1034) [3,1,4]p3(0) r=0 lpr=1034 crt=1042'663 lcod 1039'661 mlcod 1039'661 active+clean+scrubbing [ 3.bs0:  ]  ps=122] scrubber <WaitReplicas/>:  b.e.: scan_snaps 3:d81a0fb3:::smithi10749189-30:137
2022-06-30T04:02:32.039+0000 7f120425d700 15 bluestore(/var/lib/ceph/osd/ceph-3) omap_get_values meta oid #-1:c0371625:::snapmapper:0#
2022-06-30T04:02:32.039+0000 7f120425d700 20 _pin 0x5615857cc000   #-1:c0371625:::snapmapper:0# pinned
2022-06-30T04:02:32.039+0000 7f120425d700 20 bluestore.onode(0x561585a8e580).flush flush done
2022-06-30T04:02:32.039+0000 7f120425d700 10 bluestore(/var/lib/ceph/osd/ceph-3) omap_get_values meta oid #-1:c0371625:::snapmapper:0# = 0
2022-06-30T04:02:32.039+0000 7f120425d700 20 _unpin 0x5615857cc000   #-1:c0371625:::snapmapper:0# unpinned
2022-06-30T04:02:32.039+0000 7f120425d700 20 snap_mapper.get_snaps 3:d81a0fb3:::smithi10749189-30:137 got.empty()

...

2022-06-30T04:02:32.039+0000 7f120425d700 15 osd.3 pg_epoch: 1051 pg[3.bs0( v 1042'663 (1031'653,1042'663] local-lis/les=1034/1035 n=36 ec=23/23 lis/c=1034/1034 les/c/f=1035/1035/0 sis=1034) [3,1,4]p3(0) r=0 lpr=1034 crt=1042'663 lcod 1039'661 mlcod 1039'661 active+clean+scrubbing [ 3.bs0:  ]  ps=122] scrubber <WaitReplicas/>:  b.e.: scan_snaps 1 fix orders
2022-06-30T04:02:32.039+0000 7f120425d700 15 osd.3 pg_epoch: 1051 pg[3.bs0( v 1042'663 (1031'653,1042'663] local-lis/les=1034/1035 n=36 ec=23/23 lis/c=1034/1034 les/c/f=1035/1035/0 sis=1034) [3,1,4]p3(0) r=0 lpr=1034 crt=1042'663 lcod 1039'661 mlcod 1039'661 active+clean+scrubbing [ 3.bs0:  ]  ps=122] scrubber <WaitReplicas/>: apply_snap_mapper_fixes 1 fixes
2022-06-30T04:02:32.039+0000 7f120425d700 -1 log_channel(cluster) log [ERR] : osd.3(0) found snap mapper error on pg 3.bs0> oid 3:d81a0fb3:::smithi10749189-30:137 snaps missing in mapper, should be: 132,137 ...repaired