Project

General

Profile

Feature #19644

Add custom user data support in bucket index

Added by Pavan Rallabhandi about 1 month ago. Updated 23 days ago.

Status:
Pending Backport
Priority:
Normal
Target version:
-
Start date:
04/18/2017
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
kraken, jewel
Reviewed:
User Impact:
Affected Versions:
Release:
Needs Doc:
No

Description

One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.


Related issues

Copied to Backport #19843: kraken: Add custom user data support in bucket index New
Copied to Backport #19844: jewel: Add custom user data support in bucket index New

History

#1 Updated by Pavan Rallabhandi about 1 month ago

  • Status changed from New to Need Review

#2 Updated by wenjun jing about 1 month ago

can you take a example to explain this feature detailedly? Or
is there existing some docs about this feature? Thanks!

#3 Updated by Pavan Rallabhandi about 1 month ago

This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.

#4 Updated by Orit Wasserman 23 days ago

  • Backport set to kraken, jewel

#5 Updated by Orit Wasserman 23 days ago

  • Status changed from Need Review to Pending Backport

#6 Updated by Nathan Cutler 20 days ago

  • Copied to Backport #19843: kraken: Add custom user data support in bucket index added

#7 Updated by Nathan Cutler 20 days ago

  • Copied to Backport #19844: jewel: Add custom user data support in bucket index added

Also available in: Atom PDF