Bug #20201
radosgw refuses upload when Content-Type missing from POST policy
0%
Description
When doing an S3 upload, if Content-Type is missing from the policy part of the upload, then radosgw refuses the upload with a 403 error, "Policy missing condition: Content-Type". The same upload to AWS S3 is successful, despite the Content-Type not being present in the uploaded policy. radosgw should accept this, for better interoperability.
Context: this was encountered when using Fine Uploader https://github.com/FineUploader/fine-uploader to get files into an S3 bucket. When a user tries to upload a file which the browser can't identify (because it has an unrecognised or missing filename extension) then the Content-Type is not added to the POST'ed policy.
This is a show-stopper for uploading files which the browser isn't able to identify.
Attachments:
options-and-failing-post-20170531.txt is a radosgw debug = 20 log, showing the OPTIONS pre-flight call succeeding, and the POST failing at "env var missing in policy: Content-Type"
s3-upload-policy-missing-content-type-20170531.txt is cut-and-paste from the browser dev tools, showing the request headers and payload.
This is using Chrome 58.0.3029.110 on Linux; Ceph is 10.2.3-0ubuntu0.16.04.2 (Ubuntu packaged).
Related issues
History
#1 Updated by Yehuda Sadeh over 1 year ago
- Priority changed from Normal to High
#2 Updated by Matt Benjamin over 1 year ago
- Assignee set to Matt Benjamin
#3 Updated by Matt Benjamin over 1 year ago
Have a working setup for testing this. Proposed quick-fix by Y. doesn't seem quite right, but working through it. Will update 7/5.
#4 Updated by Matt Benjamin over 1 year ago
- Priority changed from High to Normal
#5 Updated by Matt Benjamin over 1 year ago
- Priority changed from Normal to High
#6 Updated by Orit Wasserman over 1 year ago
- Status changed from New to In Progress
#7 Updated by Matt Benjamin about 1 year ago
#reproduced with this
import requests
import os
rgw_host = os.environ['RGW_HOST']
rgw_port = int(os.environ['RGW_PORT'])
access_key = os.environ['RGW_ACCESS_KEY']
secret_key = os.environ['RGW_SECRET_KEY']
endpoint_url = "http://%s:%d" % (rgw_host, rgw_port)
s3 = boto3.client(service_name='s3',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
endpoint_url=endpoint_url,
use_ssl=False,
verify=False)
connection_type = 'client'
try:
bucket_name = os.environ['RGW_NFS_BUCKET']
except:
bucket_name = 'sorrydave'
try:
object_name = os.environ['RGW_NFS_OBJECT']
except:
object_name = 'jocamlpost'
- Generate the POST attributes
post = s3.generate_presigned_post(
Bucket=bucket_name,
Key=object_name
)
files = {"file": "file_content"}
response = requests.post(post["url"], data=post["fields"], files=files)
#8 Updated by Matt Benjamin about 1 year ago
- Status changed from In Progress to Need Review
- Backport set to luminous
#9 Updated by Vikhyat Umrao about 1 year ago
- Backport changed from luminous to luminous,jewel
Had a discussion with Matt. Marking it for jewel backport.
#10 Updated by Yehuda Sadeh 12 months ago
- Status changed from Need Review to Need Test
#11 Updated by Dave Holland 12 months ago
(I am the original reporter) I confirm that the patch at https://github.com/ceph/ceph/pull/18658 fixes the bug for me.
Thank you!
Dave
#12 Updated by Nathan Cutler 12 months ago
- Status changed from Need Test to Pending Backport
#13 Updated by Nathan Cutler 12 months ago
- Status changed from Pending Backport to Need Test
This is waiting for a test case to be added to https://github.com/ceph/s3-tests
Once that is in place, we can do the ceph/ceph.git and ceph/s3-tests.git backports together.
#14 Updated by Matt Benjamin 11 months ago
Candidate test s3tests.functional.test_s3:test_post_object_authenticated_no_content_type pushed https://github.com/ceph/s3-tests/pull/204
#15 Updated by Matt Benjamin 11 months ago
- Status changed from Need Test to Need Review
#16 Updated by Orit Wasserman 10 months ago
- Status changed from Need Review to Pending Backport
#17 Updated by Nathan Cutler 10 months ago
The s3-tests patch/commit will need to be backported to the "ceph-jewel" and "ceph-luminous" branches along with the fix, right?
#18 Updated by Nathan Cutler 10 months ago
- Copied to Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policy added
#19 Updated by Nathan Cutler 10 months ago
- Copied to Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policy added
#20 Updated by Yuri Weinstein 9 months ago
#21 Updated by Nathan Cutler 8 months ago
- Status changed from Pending Backport to Resolved