Ceph » RADOS

2017-06-20 07:29:44.582239 7f5d60e90280 10 bluestore(/var/lib/ceph/osd/ceph-3).collection(0.4_head 0x557472d6fa00) open_shared_blob sbid 0x4143 opened SharedBlob(0x55747fd97260 sbid 0x4143)
2017-06-20 07:29:44.582240 7f5d60e90280 20 bluestore.blob(0x55747fd971f0) get_ref 0x0~10000 Blob(0x55747fd971f0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x0 0x0) SharedBlob(0x55747fd97260 sbid 0x4143))
2017-06-20 07:29:44.582249 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_onode 0x55747f6979c0 #1:23ec1e7d:::smithi0469966-402:10b# nid 4491 size 0x29cd97 (2739607) expected_object_size 0 expected_write_size 0 in 4 shards, 0 spanning blobs
2017-06-20 07:29:44.582337 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_extent_map  0x24c000~10000: 0x0~10000 Blob(0x55747fd971f0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x4*0x4000 0x[4000,4000,4000,4000]) SharedBlob(0x55747fd97260 sbid 0x4143))

2017-06-20 07:29:44.584863 7f5d60e90280 10 bluestore(/var/lib/ceph/osd/ceph-3).collection(0.4_head 0x557472d6fa00) open_shared_blob sbid 0x4143 had SharedBlob(0x55747fd97260 sbid 0x4143)
2017-06-20 07:29:44.584864 7f5d60e90280 20 bluestore.blob(0x55747fd98bc0) get_ref 0x1064~ef9c Blob(0x55747fd98bc0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x0 0x0) SharedBlob(0x55747fd97260 sbid 0x4143))
2017-06-20 07:29:44.584865 7f5d60e90280 20 bluestore.blob(0x55747fd98bc0) get_ref init 0x10000, 4000
2017-06-20 07:29:44.584929 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_onode 0x55747f697c80 #1:23ec1e7d:::smithi0469966-402:11f# nid 8442 size 0x3a385f (3815519) expected_object_size 0 expected_write_size 0 in 6 shards, 1 spanning blobs
2017-06-20 07:29:44.585049 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_extent_map  0x24d064~ef9c: 0x1064~ef9c Blob(0x55747fd98bc0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x4*0x4000 0x[2f9c,4000,4000,4000]) SharedBlob(0x55747fd97260 sbid 0x4143))

2017-06-20 07:29:44.588466 7f5d60e90280 10 bluestore(/var/lib/ceph/osd/ceph-3).collection(0.4_head 0x557472d6fa00) open_shared_blob sbid 0x4143 had SharedBlob(0x55747fd97260 sbid 0x4143)
2017-06-20 07:29:44.588468 7f5d60e90280 20 bluestore.blob(0x55747eafe4d0) get_ref 0x1064~ef9c Blob(0x55747eafe4d0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x0 0x0) SharedBlob(0x55747fd97260 sbid 0x4143))
2017-06-20 07:29:44.588470 7f5d60e90280 20 bluestore.blob(0x55747eafe4d0) get_ref init 0x10000, 4000
2017-06-20 07:29:44.588529 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_onode 0x55747f694580 #1:23ec1e7d:::smithi0469966-402:head# nid 8447 size 0x3a385f (3815519) expected_object_size 0 expected_write_size 0 in 7 shards, 2 spanning blobs
2017-06-20 07:29:44.588658 7f5d60e90280 30 bluestore(/var/lib/ceph/osd/ceph-3) _dump_extent_map  0x24d064~ef9c: 0x1064~ef9c Blob(0x55747eafe4d0 blob([0x1f9c4000~10000] csum+shared crc32c/0x1000) use_tracker(0x4*0x4000 0x[2f9c,4000,4000,4000]) SharedBlob(0x55747fd97260 sbid 0x4143))

2017-06-20 07:29:45.925369 7f5d60e90280 -1 bluestore(/var/lib/ceph/osd/ceph-3) fsck error: shared blob 0x4143 ref_map ref_map(0x1f9c4000~10000=2) != expected ref_map(0x1f9c4000~10000=3)

the problem

2017-06-20 07:27:53.176177 7fe207557c80 10 bluestore(/var/lib/ceph/osd/ceph-3).collection(1.4_head 0x5565a1c73c00) open_shared_blob sbid 0x4143 opened SharedBlob(0x5565a9c71170 sbid 0x4143)
2017-06-20 07:27:53.554181 7fe1ee706700 20 bluestore(/var/lib/ceph/osd/ceph-3).collection(1.4_head 0x5565a1c73c00) split_cache  moving SharedBlob(0x5565a9c71170 sbid 0x4143)
2017-06-20 07:28:00.950171 7fe1eb700700 10 bluestore(/var/lib/ceph/osd/ceph-3).collection(1.44_head 0x5565a29a6c00) open_shared_blob sbid 0x4143 opened SharedBlob(0x55659d7fc4c0 sbid 0x4143)

/a/sage-2017-06-20_06:19:52-rados-wip-sage-testing-distro-basic-smithi/1304153