Add custom user data support in bucket index
One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.
#3 Updated by Pavan Rallabhandi almost 2 years ago
This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.