Project

General

Profile

Actions

Bug #62737

open

RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage

Added by Shreyansh Sancheti 8 months ago. Updated 8 months ago.

Status:
Pending Backport
Priority:
Normal
Target version:
% Done:

0%

Source:
Tags:
backport_processed
Backport:
pacific quincy reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I have tried to get bucket usage via various methods, through curl or while accessing rgw api endpoint HEAD /{bucket}/?usage doesn't return the updated information and I have a doubt it doesn't return the bucket quota instead it returns the user quota.

Reproducing steps:

  • While in a vstart cluster after setting the rgw user and permissions such as
./bin/radosgw-admin caps add --uid=testid --caps="users=*;buckets=*;metadata=*;usage=*;zone=*" | jq
  • now you can check the status with :
    radosgw-admin user info --uid=testid
    
  • Create a test bucket using s3cmd and listed the bucket quota
    ./bin/radosgw-admin bucket stats --bucket=test-bucket | jq '.bucket_quota'
    
  • I have then tried to update the bucket quota using curl with the following script
    cat file.json
            {
              "enabled": true,
              "max_size": 12345,
              "max_size_kb": 0,
              "max_objects": 123
            }
    

    #!/bin/bash
    
    # Usage : rgw_curl.sh $access_key $secret_key $rgw_host:$rgw_port $http_request $admin_resource $query_string   
    access_key=$1
    secret_key=$2 
    rgw_endpoint="$3" 
    http_request=$4
    resource="/admin/$5" 
    query_string=$6
    
    contentType="application/json" 
    dateTime=`date -R -u`
    
    headerToSign="${http_request}
    
    ${contentType}
    ${dateTime}
    ${resource}" 
    
    signature=`echo -en "$headerToSign" | openssl sha1 -hmac ${secret_key} -binary | base64`
    
    curl -X ${http_request} -H "Content-Type: ${contentType}" -H "Date: ${dateTime}" -H "Authorization: AWS ${access_key}:${signature}" -H "Host: ${rgw_endpoint}" "http://${rgw_endpoint}${resource}?${query_string}" 
    

    bash -x ~/rgw_curl.sh "0555b35654ad1656d804" "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==" localhost:8000 PUT bucket "quota&uid=testid&bucket=bucket1&quota"|jq
    

*Above script should update the quota values and also enable the bucket quota but if you look at the bucket stats it will be not updated:

endpoint="localhost:8000" 
resource="/bucket1/" 
S3KEY="0555b35654ad1656d804" 
S3SECRET="h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==" 
dateValue=$(date -R -u)

stringToSign="HEAD\n\n\n${dateValue}\n${resource}" 

signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${S3SECRET} -binary | base64`

curl -sk -H "authorization: AWS ${S3KEY}:${signature}" -H "date: ${dateValue}" "http:///${endpoint}${resource}?usage" -v -XHEAD
< HTTP/1.1 200 OK
        < X-RGW-Object-Count: 0
        < X-RGW-Bytes-Used: 0
        < X-RGW-Quota-User-Size: -1
        < X-RGW-Quota-User-Objects: -1
        < X-RGW-Quota-Max-Buckets: 1000
        < X-RGW-Quota-Bucket-Size: -1
        < X-RGW-Quota-Bucket-Objects: -1
        < x-amz-request-id: tx0000004ae5b7e133832df-0064d2879c-37be-default
        < Content-Length: 0
        < Date: Tue, 08 Aug 2023 18:21:16 GMT
        < Connection: Keep-Alive

  • but if checked with the cmdline option :
    ./bin/radosgw-admin bucket stats --bucket=test-bucket | jq '.bucket_quota'
    
    "bucket_quota": {
            "enabled": true,
            "check_on_raw": false,
            "max_size": 12345,
            "max_size_kb": 13,
            "max_objects": 123
        }
    

From what I understood from the code base for this endpoint it is always returning the user quota values which you can find from

./bin/radosgw-admin user info --uid=testid |jq '.bucket_quota'

{
  "enabled": false,
  "check_on_raw": false,
  "max_size": -1,
  "max_size_kb": 0,
  "max_objects": -1
}


Related issues 3 (1 open2 closed)

Copied to rgw - Backport #62823: pacific: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usageResolvedKonstantin ShalyginActions
Copied to rgw - Backport #62824: quincy: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usageIn ProgressKonstantin ShalyginActions
Copied to rgw - Backport #62825: reef: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usageResolvedKonstantin ShalyginActions
Actions #1

Updated by Sridhar Seshasayee 8 months ago

  • Private changed from Yes to No
Actions #2

Updated by Shreyansh Sancheti 8 months ago

  • Status changed from New to In Progress
  • Pull request ID set to 53391
Actions #3

Updated by Konstantin Shalygin 8 months ago

  • Status changed from In Progress to Pending Backport
  • Target version set to v19.0.0
  • Backport set to pacific quincy reef
Actions #4

Updated by Backport Bot 8 months ago

  • Copied to Backport #62823: pacific: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage added
Actions #5

Updated by Backport Bot 8 months ago

  • Copied to Backport #62824: quincy: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage added
Actions #6

Updated by Backport Bot 8 months ago

  • Copied to Backport #62825: reef: RadosGW API: incorrect bucket quota in response to HEAD /{bucket}/?usage added
Actions #7

Updated by Backport Bot 8 months ago

  • Tags set to backport_processed
Actions #8

Updated by Konstantin Shalygin 8 months ago

  • Assignee set to Shreyansh Sancheti
Actions

Also available in: Atom PDF