Feature #19644: Add custom user data support in bucket index - rgw - Ceph

Actions

Copy link

Feature #19644

closed

Add custom user data support in bucket index

Added by Pavan Rallabhandi about 7 years ago. Updated over 6 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Pavan Rallabhandi

Target version:

% Done:

Source:

Community (dev)

Tags:

Backport:

kraken, jewel

Reviewed:

Affected Versions:

Pull request ID:

Description

One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by Pavan Rallabhandi about 7 years ago

Status changed from New to Fix Under Review

https://github.com/ceph/ceph/pull/14592

Actions

Copy link

Updated by wenjun jing about 7 years ago

can you take a example to explain this feature detailedly? Or
is there existing some docs about this feature? Thanks!

Actions

Copy link

Updated by Pavan Rallabhandi about 7 years ago

This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.

Actions

Copy link