Bug #55410
closed[rgw-ms][dbr]:Large omap object found in the buckets.index pool, corresponding to resharded buckets
0%
Description
We observe warning like large omap found in the index pool corresponding to resharded buckets that did not completely sync.
[WRN] LARGE_OMAP_OBJECTS: 2 large omap objects
2 large objects found in pool 'east.rgw.buckets.index'
Search the cluster log for 'Large omap object found' for more details.
---------------------------
snippet from the osd.log
----------------------------
grep -C 200 Large /var/log/ceph/e16ec086-bfb0-11ec-8537-002590fbd524/ceph-osd.*.log | egrep 'Large omap'
/var/log/ceph/e16ec086-bfb0-11ec-8537-002590fbd524/ceph-osd.2.log:2022-04-22T05:14:32.703+0000 7fcc52a8e700 0 log_channel(cluster) log [WRN] : Large omap object found. Object: 7:16ead4f9:::.dir.167fc967-e35e-4eb4-bc09-f71b324f77d8.31813.1.2:head PG: 7.9f2b5768 (7.8) Key count: 536197 Size (bytes): 141281185Updated by Vidushi Mishra about 2 years ago
1. ceph version 17.0.0-10783-ge38464a1 (e38464a10ae9e8c7b43bae5a9a7395eb2cbb2444) quincy (dev)
2. Steps to reproduce:
i. Create a multi-site with 14 rgws on each site [4 for ms sync and 10 for client IO.]
ii. The 4 rgws for ms sync are not behind any LB.
iii. create a bucket 'test-sync-no-lb-1' and upload 20M objects [10M from either site.]
Result:
We observe Large omap objects found in the index pool on the primary site.
Updated by Casey Bodley almost 2 years ago
Vidushi Mishra wrote:
1. ceph version 17.0.0-10783-ge38464a1 (e38464a10ae9e8c7b43bae5a9a7395eb2cbb2444) quincy (dev)
2. Steps to reproduce:
i. Create a multi-site with 14 rgws on each site [4 for ms sync and 10 for client IO.]
ii. The 4 rgws for ms sync are not behind any LB.
iii. create a bucket 'test-sync-no-lb-1' and upload 20M objects [10M from either site.]Result:
We observe Large omap objects found in the index pool on the primary site.
this is probably just because bilog trim can't keep up with your 20M object workload. large omap can be expected here