Project

General

Profile

Bug #39155

Quota is ignored on uploading by POST

Added by Dmitry Plyakin almost 5 years ago. Updated over 4 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
easy first bug
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

When I upload files by POST method, RGW ignores quota.
For example I set max_objects = 5 and max_size_kb = 1024:
radosgw-admin metadata get user:da13b83b-2279-422b-ba58-3cec84c094ed

{
    "key": "user:da13b83b-2279-422b-ba58-3cec84c094ed",
    "ver": {
        "tag": "_TdYS9FCv8keXXQUGW75d_3y",
        "ver": 12
    },
    "mtime": "2019-04-09 08:19:47.496260Z",
    "data": {
        "user_id": "da13b83b-2279-422b-ba58-3cec84c094ed",
        "display_name": "limit@name",
        "email": "",
        "suspended": 0,
        "max_buckets": 5,
        "auid": 0,
        "subusers": [],
        "keys": [
            {
                "user": "da13b83b-2279-422b-ba58-3cec84c094ed",
                "access_key": "...",
                "secret_key": "..." 
            }
        ],
        "swift_keys": [],
        "caps": [],
        "op_mask": "read, write, delete",
        "default_placement": "",
        "placement_tags": [],
        "bucket_quota": {
            "enabled": true,
            "check_on_raw": false,
            "max_size": 1048576,
            "max_size_kb": 1024,
            "max_objects": 5
        },
        "user_quota": {
            "enabled": false,
            "check_on_raw": false,
            "max_size": -1,
            "max_size_kb": 0,
            "max_objects": -1
        },
        "temp_url_keys": [],
        "type": "rgw",
        "attrs": [
            {
                "key": "user.rgw.idtag",
                "val": "" 
            }
        ]
    }
}

And after that I can exceed that quota when upload by POST:
request: POST /limit/ HTTP/1.1 ...
reply: HTTP/1.1 204 No Content ...
radosgw-admin bucket stats --bucket limit
{
    "bucket": "limit",
    "zonegroup": "f6fee20e-f2fa-4bbd-96cf-54d612f96523",
    "placement_rule": "default-placement",
    "explicit_placement": {
        "data_pool": "",
        "data_extra_pool": "",
        "index_pool": "" 
    },
    "id": "ad0d24cf-ea4c-49f3-af1e-5da9aa1b51a1.4116.110",
    "marker": "ad0d24cf-ea4c-49f3-af1e-5da9aa1b51a1.4116.110",
    "index_type": "Normal",
    "owner": "da13b83b-2279-422b-ba58-3cec84c094ed",
    "ver": "0#81",
    "master_ver": "0#0",
    "mtime": "2019-04-08 19:27:04.262236",
    "max_marker": "0#",
    "usage": {
        "rgw.main": {
            "size": 2171332,
            "size_actual": 2174976,
            "size_utilized": 2171332,
            "size_kb": 2121,
            "size_kb_actual": 2124,
            "size_kb_utilized": 2121,
            "num_objects": 8
        }
    },
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    }
}

When upload files by PUT quota works as expected:
request: PUT /limit/ke555y4 HTTP/1.1 ...
response: 403 Forbidden
<Error><Code>QuotaExceeded</Code><BucketName>limit</BucketName><RequestId>tx0000000000000000464fc-005cac6b7b-1014-default</RequestId><HostId>1014-default-default</HostId></Error>

Reproduced in ceph version 12.2.5

History

#1 Updated by Casey Bodley almost 5 years ago

  • Tags set to easy first bug

#2 Updated by Matthew Oliver over 4 years ago

OK so I'm new to RGW, when you say upload by POST. Do you mean via s3 or swift's formpost equiv?

Just want to confirm before I go off on a rabbit hole. Just want to make sure I'm not missing anything.

I can just go test this in my own devenv to see if it's so. I know from a Swift POV quota is always a soft quota, so there is a chance of going above before it's enforced. Though no idea if it's the same with RGW. However, if we can just keep uploading objs via post then that definitely is a bug.

I do see check_quota calls in the RGWPostObj::execute ie (for bytes I assume):

op_ret = store->getRados()->check_quota(s->bucket_owner.get_id(), s->bucket,
                            user_quota, bucket_quota, s->obj_size);
if (op_ret < 0) {
  return;
}

So hopefully this will just be debugging why they aren't working as expected.

#3 Updated by Dmitry Plyakin over 4 years ago

Matthew Oliver wrote:

OK so I'm new to RGW, when you say upload by POST. Do you mean via s3 or swift's formpost equiv?

I dont use swift API so I caught such bug only in S3.

Also available in: Atom PDF