Project

General

Profile

Bug #23817

Bucket policy and colons in filename

Added by Vladimir Buyanov over 1 year ago. Updated 7 months ago.

Status:
Pending Backport
Priority:
High
Assignee:
Target version:
Start date:
04/21/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
luminous mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:

Description

Hello.
I see strange behavior on files with colons in filename. Bucket policy not applied for its.
Example:
1. Create policy like this and set it to bucket.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::admin"]},
    "Action": "*",
    "Resource": [
      "arn:aws:s3:::test/*" 
    ]
  },
  {
    "Effect": "Allow",
    "Principal": "*",
    "Action": ["s3:GetObject"],
    "Resource": [
      "arn:aws:s3:::test/*" 
    ]
  }]
}

2. Upload file without colons in name and try to get it from anonymous user. It should work fine.
3. Upload file with colons in name and try to get it from anonymous user. You should get 403 error:
>> s3cmd put /tmp/file s3://test/test:file
>> curl http://rgw:7480/test/test:file -D-                                                                                                                                                                                              
HTTP/1.1 403 Forbidden
Content-Length: 214
x-amz-request-id: tx00000000000000017737a-005adb746a-ff06-default
Accept-Ranges: bytes
Content-Type: application/xml
Date: Sat, 21 Apr 2018 17:27:06 GMT

<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><BucketName>test</BucketName><RequestId>tx00000000000000017737a-005adb746a-ff06-default</RequestId><HostId>ff06-default-default</HostId></Error>

I found workaround, files uploaded with public acl (-P key for s3cmd) works fine, but this is not a good solution.
My ceph version: 12.2.4


Related issues

Copied to rgw - Backport #37482: luminous: Bucket policy and colons in filename Resolved
Copied to rgw - Backport #37483: mimic: Bucket policy and colons in filename In Progress

History

#1 Updated by Casey Bodley about 1 year ago

  • Assignee set to Adam Emerson

#2 Updated by Orit Wasserman about 1 year ago

  • Status changed from New to In Progress

#3 Updated by Herve Rousseau 9 months ago

Hello,

We've just been hit by this issue on a Ceph Cluster running version 12.2.8.

We can easily provide reproducers and/or test a fix

Thanks !

#4 Updated by Casey Bodley 9 months ago

  • Priority changed from Normal to High

#5 Updated by Adam Emerson 9 months ago

I was not able to reproduce this, so if you could give me reproducers I would greatly appreciate it!

#6 Updated by Adrian Mönnich 9 months ago

[adrian@blackhole:~]> s3cmd info s3://adrian-test/foo:bar                                                                                                                                                                                                                                   [4/1120]
s3://adrian-test/foo:bar (object):
   File size: 11
   Last mod:  Wed, 07 Nov 2018 13:03:24 GMT
   MIME type: text/plain
   Storage:   STANDARD
   MD5 sum:   d73b04b0e696b0945283defa3eee4538
   SSE:       none
   Policy:    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:GetObject",
        "s3:ListBucket" 
      ],
      "Effect": "Allow",
      "Resource": "*",
      "Principal": {
        "AWS": [
          "arn:aws:iam:::user/indicoread" 
        ]
      }
    }
  ]
}

   CORS:      none
   ACL:       Indico Service: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1541595795/ctime:1541595795/gid:100/gname:users/md5:d73b04b0e696b0945283defa3eee4538/mode:33188/mtime:1541595795/uid:1000/uname:adrian

[adrian@blackhole:~]> s3cmd info s3://adrian-test/foobar
s3://adrian-test/foobar (object):
   File size: 10
   Last mod:  Wed, 07 Nov 2018 13:03:50 GMT
   MIME type: text/plain
   Storage:   STANDARD
   MD5 sum:   9938590a16231655b7737aec3be9a55f
   SSE:       none
   Policy:    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:GetObject",
        "s3:ListBucket" 
      ],
      "Effect": "Allow",
      "Resource": "*",
      "Principal": {
        "AWS": [
          "arn:aws:iam:::user/indicoread" 
        ]
      }
    }
  ]
}

   CORS:      none
   ACL:       Indico Service: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1541595826/ctime:1541595826/gid:100/gname:users/md5:9938590a16231655b7737aec3be9a55f/mode:33188/mtime:1541595826/uid:1000/uname:adrian

[adrian@blackhole:~]> s3cmd -c ~/.s3cfg.indicoread get s3://adrian-test/foo:bar
download: 's3://adrian-test/foo:bar' -> './foo:bar'  [1 of 1]
ERROR: S3 error: 403 (AccessDenied)

[adrian@blackhole:~]> s3cmd -c ~/.s3cfg.indicoread get s3://adrian-test/foobar
download: 's3://adrian-test/foobar' -> './foobar'  [1 of 1]
 10 of 10   100% in    0s   252.73 B/s  done

#7 Updated by Adam Emerson 8 months ago

  • Status changed from In Progress to Need Review

#10 Updated by Nathan Cutler 8 months ago

@Adam: next time, please check out src/script/backport-create-issue for your backport issue creation needs - thanks!

#11 Updated by Nathan Cutler 8 months ago

@Adam - the general idea is to wait until the master PR is merged, and then:

1. fill in the Backport field (e.g. "mimic, luminous")
2. change issue status to "Pending Backport"
3. (optional) run src/script/backport-create-issue $MASTER_ISSUE_NUMBER

But note that 3 is not needed unless you're really in a hurry. I regularly run the script and it has a mode where it loops over all issues in Pending Backport status.

#12 Updated by Nathan Cutler 8 months ago

(I deleted the backport issues for now since they weren't properly formed and the master PR hasn't merged yet. Thanks!)

#13 Updated by Casey Bodley 8 months ago

  • Status changed from Need Review to Testing

#14 Updated by Casey Bodley 8 months ago

  • Status changed from Testing to Pending Backport
  • Backport set to luminous mimic

#15 Updated by Nathan Cutler 8 months ago

  • Copied to Backport #37482: luminous: Bucket policy and colons in filename added

#16 Updated by Nathan Cutler 8 months ago

  • Copied to Backport #37483: mimic: Bucket policy and colons in filename added

#17 Updated by Behnam Loghmani 7 months ago

When can we have this bug fix on mimic versions?
I am using 13.2.2 and I need this bug fix...

#18 Updated by Nathan Cutler 7 months ago

Behnam Loghmani wrote:

When can we have this bug fix on mimic versions?
I am using 13.2.2 and I need this bug fix...

Barring unforeseen circumstances, it looks like it should make it into 13.2.4.

Also available in: Atom PDF