Bug #21542
openluminous: index gets heavy read IOPS with index-less RGW pool
0%
Description
Hi All,
Recently, we've noticed a strange behaviour on one of our test clusters.
The cluster was configured to serve RGW and is running Luminous. Our standard procedure is to create blind (non-indexed) buckets so that our software manages the metadata by itself and we get less load on the index pool, with the zone configured as follows (notice the index type 1, which stands for blind buckets).
What we're seeing is periodical read spikes on the index pool, which are getting so bad the cluster starts seeing slow requests with barely any load. If I'm right in this configuration the index pool should not be getting any IOPS at all (at least on Kraken it worked this way).
Our Kraken sites don't have any objects in this pool, but for some reason our Luminous site has ~65K objects there - seems like RGW created them automatically?
Here's the relevant info from the cluster: https://pastebin.ca/3876293
Previously, we were running Kraken at multiple sites in the same configuration and never noticed this happening. Any ideas?
Files
Updated by Yuri Gorshkov over 6 years ago
- File index_stat.txt.xz index_stat.txt.xz added
Updated by John Spray over 6 years ago
- Project changed from Ceph to rgw
- Category deleted (
22)
Updated by Yehuda Sadeh over 6 years ago
Is this still happening? Can you list the pool, what type of objects do you have there?