Project

General

Profile

Actions

Bug #50974

open

rgw: storage class: GLACIER lifecycle don't worked when STANDARD pool and GLACIER pool are equal

Added by Konstantin Shalygin almost 3 years ago. Updated over 1 year ago.

Status:
Triaged
Priority:
Normal
Assignee:
-
Target version:
% Done:

0%

Source:
Community (user)
Tags:
GLACIER
Backport:
pacific quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The case is simple:

  • storage classes is:
rgw zone placement configuration: radosgw-admin zone get --rgw-zone=default | jq '.placement_pools[] | select(.key=="default-placement")'
{
  "key": "default-placement",
  "val": {
    "index_pool": "default.rgw.buckets.index",
    "storage_classes": {
      "GLACIER": {
        "data_pool": "default.rgw.buckets.data",
        "compression_type": "lz4" 
      },
      "STANDARD": {
        "data_pool": "default.rgw.buckets.data" 
      }
    },
    "data_extra_pool": "default.rgw.buckets.non-ec",
    "index_type": 0
  }
}
  • user use STANDARD class by default
  • user set LC policy - move objects to GLACIER class
    <?xml version="1.0" ?>
    <LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
        <Rule>
            <ID>Move_to_GLACIER</ID>
            <Prefix/>
            <Status>Enabled</Status>
            <Transition>
                <Days>0</Days>
                <StorageClass>GLACIER</StorageClass>
            </Transition>
        </Rule>
    </LifecycleConfiguration>
    
  • when LC is complete, all objects will have a GLACIER class, but still not compressed, bucket stats diff:
--- glacier_pre 2021-05-23 19:08:36.121857545 +0700
+++ glacier_post        2021-05-24 03:27:56.771780018 +0700
@@ -13,7 +13,7 @@
     "marker": "f9e6d5e1-208a-4bf2-961f-d747a652b2f8.772483578.1",
     "index_type": "Normal",
     "owner": "k0ste",
-    "ver": "0#39541",
+    "ver": "0#71312",
     "master_ver": "0#0",
     "mtime": "2021-05-23 11:55:06.870920Z",
     "max_marker": "0#",

I perform test:

  • upload bunch of test data (git repos mostly) - with STANDARD class, then move to GLACIER class with LC

  • upload this data again, to another bucket with --storage-class=GLACIER option - data is really compressed


Files

GLACIER_compressed.png (171 KB) GLACIER_compressed.png Konstantin Shalygin, 05/25/2021 06:58 PM
GLACIER_uncompressed.png (173 KB) GLACIER_uncompressed.png Konstantin Shalygin, 05/25/2021 06:58 PM
Actions #1

Updated by Casey Bodley almost 3 years ago

sharing pools between storage classes is a bit unexpected, but not unreasonable. lifecycle transitions use the same logic as CopyObject, where a copy into the same pool can share the data instead of making a copy. it sounds like this logic needs to compare storage classes instead of just the pool names

Actions #2

Updated by Casey Bodley almost 3 years ago

  • Status changed from New to Triaged
Actions #3

Updated by Konstantin Shalygin almost 3 years ago

it sounds like this logic needs to compare storage classes instead of just the pool names

Thanks, seems something like that

Actions #4

Updated by Konstantin Shalygin over 1 year ago

  • Target version set to v18.0.0
  • Backport set to pacific quincy
Actions

Also available in: Atom PDF