Project

General

Profile

Feature #19644

Add custom user data support in bucket index

Added by Pavan Rallabhandi 2 months ago. Updated about 2 months ago.

Status:
Pending Backport
Priority:
Normal
Target version:
-
Start date:
04/18/2017
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
kraken, jewel
Reviewed:
User Impact:
Affected Versions:
Release:
Needs Doc:
No

Description

One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.


Related issues

Copied to rgw - Backport #19843: kraken: Add custom user data support in bucket index New
Copied to rgw - Backport #19844: jewel: Add custom user data support in bucket index Need More Info

History

#1 Updated by Pavan Rallabhandi 2 months ago

  • Status changed from New to Need Review

#2 Updated by wenjun jing 2 months ago

can you take a example to explain this feature detailedly? Or
is there existing some docs about this feature? Thanks!

#3 Updated by Pavan Rallabhandi 2 months ago

This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.

#4 Updated by Orit Wasserman about 2 months ago

  • Backport set to kraken, jewel

#5 Updated by Orit Wasserman about 2 months ago

  • Status changed from Need Review to Pending Backport

#6 Updated by Nathan Cutler about 2 months ago

  • Copied to Backport #19843: kraken: Add custom user data support in bucket index added

#7 Updated by Nathan Cutler about 2 months ago

  • Copied to Backport #19844: jewel: Add custom user data support in bucket index added

Also available in: Atom PDF