Project

General

Profile

Actions

Bug #42576

closed

[rbd-mirror] remote image status lookup is expensive

Added by Jason Dillaman over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jason Dillaman
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The "mirror_image_status_list" command lists mirrored images, then for each one it pulls the image status. Pulling the image status involves a call to 'get_remote_image_status_fsids' which attempts to only list the fsids but actually iterates through all keys due to a logic bug where it never breaks out of the filter immediately ...

2019-10-30T17:52:18.948+0000 37b85700 10 osd.5 pg_epoch: 122 pg[28.6( v 120'2165 (0'0,120'2165] local-lis/les=121/122 n=8 ec=118/118 lis/c=121/118 les/c/f=122/119/0 sis=121) [5] r=0 lpr=121 pi=[118,121)/1 crt=120'2165 lcod 120'2164 mlcod 0'0 active+undersized+degraded mbc={}] do_osd_op 28:67cae2ed:::rbd_mirroring:head [omap-get-keys in=39b]
2019-10-30T17:52:18.949+0000 37b85700 10 osd.5 pg_epoch: 122 pg[28.6( v 120'2165 (0'0,120'2165] local-lis/les=121/122 n=8 ec=118/118 lis/c=121/118 les/c/f=122/119/0 sis=121) [5] r=0 lpr=121 pi=[118,121)/1 crt=120'2165 lcod 120'2164 mlcod 0'0 active+undersized+degraded mbc={}] do_osd_op  omap-get-keys in=39b
2019-10-30T17:52:18.950+0000 37b85700 10 bluestore(/var/lib/ceph/osd/ceph-5) get_omap_iterator 28.6_head #28:67cae2ed:::rbd_mirroring:head#
2019-10-30T17:52:18.950+0000 37b85700 20 bluestore.onode(0x3ce4a730).flush flush done
2019-10-30T17:52:18.950+0000 37b85700 10 bluestore(/var/lib/ceph/osd/ceph-5) get_omap_iterator has_omap = 1
2019-10-30T17:52:18.951+0000 37b85700  1 RefCountedObject::get 0x2075e390 4 -> 5
2019-10-30T17:52:18.951+0000 37b85700  1 RefCountedObject::get 0x2075e390 5 -> 6
2019-10-30T17:52:18.951+0000 37b85700  1 RefCountedObject::put 0x2075e390 6 -> 5
2019-10-30T17:52:18.951+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) upper_bound after remote_status_global_uuid0_ key 0x000000000000001c0000000000000579'.remote_status_global_uuid0_'
2019-10-30T17:52:18.951+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid0'
2019-10-30T17:52:18.952+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid1'
2019-10-30T17:52:18.952+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid10'
2019-10-30T17:52:18.952+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid100'
2019-10-30T17:52:18.952+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid1000'
2019-10-30T17:52:18.952+0000 37b85700 20 bluestore.OmapIteratorImpl(0x3d124d00) valid is at 0x000000000000001c0000000000000579'.status_global_uuid1001'
...

... it then re-pulls the remote status that it just retrieved the keys against which should be unnecessary if it just pulled the keys + values together originally.

Actions #1

Updated by Jason Dillaman over 4 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 31865
Actions #2

Updated by Sage Weil over 4 years ago

  • Status changed from Fix Under Review to Resolved

\o/

Actions

Also available in: Atom PDF