Bug #39155
Quota is ignored on uploading by POST
0%
Description
When I upload files by POST method, RGW ignores quota.
For example I set max_objects = 5 and max_size_kb = 1024:
radosgw-admin metadata get user:da13b83b-2279-422b-ba58-3cec84c094ed
{ "key": "user:da13b83b-2279-422b-ba58-3cec84c094ed", "ver": { "tag": "_TdYS9FCv8keXXQUGW75d_3y", "ver": 12 }, "mtime": "2019-04-09 08:19:47.496260Z", "data": { "user_id": "da13b83b-2279-422b-ba58-3cec84c094ed", "display_name": "limit@name", "email": "", "suspended": 0, "max_buckets": 5, "auid": 0, "subusers": [], "keys": [ { "user": "da13b83b-2279-422b-ba58-3cec84c094ed", "access_key": "...", "secret_key": "..." } ], "swift_keys": [], "caps": [], "op_mask": "read, write, delete", "default_placement": "", "placement_tags": [], "bucket_quota": { "enabled": true, "check_on_raw": false, "max_size": 1048576, "max_size_kb": 1024, "max_objects": 5 }, "user_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 }, "temp_url_keys": [], "type": "rgw", "attrs": [ { "key": "user.rgw.idtag", "val": "" } ] } }
And after that I can exceed that quota when upload by POST:
request: POST /limit/ HTTP/1.1 ...
reply: HTTP/1.1 204 No Content ...
radosgw-admin bucket stats --bucket limit
{ "bucket": "limit", "zonegroup": "f6fee20e-f2fa-4bbd-96cf-54d612f96523", "placement_rule": "default-placement", "explicit_placement": { "data_pool": "", "data_extra_pool": "", "index_pool": "" }, "id": "ad0d24cf-ea4c-49f3-af1e-5da9aa1b51a1.4116.110", "marker": "ad0d24cf-ea4c-49f3-af1e-5da9aa1b51a1.4116.110", "index_type": "Normal", "owner": "da13b83b-2279-422b-ba58-3cec84c094ed", "ver": "0#81", "master_ver": "0#0", "mtime": "2019-04-08 19:27:04.262236", "max_marker": "0#", "usage": { "rgw.main": { "size": 2171332, "size_actual": 2174976, "size_utilized": 2171332, "size_kb": 2121, "size_kb_actual": 2124, "size_kb_utilized": 2121, "num_objects": 8 } }, "bucket_quota": { "enabled": false, "check_on_raw": false, "max_size": -1, "max_size_kb": 0, "max_objects": -1 } }
When upload files by PUT quota works as expected:
request: PUT /limit/ke555y4 HTTP/1.1 ...
response: 403 Forbidden
<Error><Code>QuotaExceeded</Code><BucketName>limit</BucketName><RequestId>tx0000000000000000464fc-005cac6b7b-1014-default</RequestId><HostId>1014-default-default</HostId></Error>
Reproduced in ceph version 12.2.5
History
#1 Updated by Casey Bodley almost 5 years ago
- Tags set to easy first bug
#2 Updated by Matthew Oliver over 4 years ago
OK so I'm new to RGW, when you say upload by POST. Do you mean via s3 or swift's formpost equiv?
Just want to confirm before I go off on a rabbit hole. Just want to make sure I'm not missing anything.
I can just go test this in my own devenv to see if it's so. I know from a Swift POV quota is always a soft quota, so there is a chance of going above before it's enforced. Though no idea if it's the same with RGW. However, if we can just keep uploading objs via post then that definitely is a bug.
I do see check_quota calls in the RGWPostObj::execute ie (for bytes I assume):
op_ret = store->getRados()->check_quota(s->bucket_owner.get_id(), s->bucket,
user_quota, bucket_quota, s->obj_size);
if (op_ret < 0) {
return;
}
So hopefully this will just be debugging why they aren't working as expected.
#3 Updated by Dmitry Plyakin over 4 years ago
Matthew Oliver wrote:
OK so I'm new to RGW, when you say upload by POST. Do you mean via s3 or swift's formpost equiv?
I dont use swift API so I caught such bug only in S3.