Bug #20201
closedradosgw refuses upload when Content-Type missing from POST policy
0%
Description
When doing an S3 upload, if Content-Type is missing from the policy part of the upload, then radosgw refuses the upload with a 403 error, "Policy missing condition: Content-Type". The same upload to AWS S3 is successful, despite the Content-Type not being present in the uploaded policy. radosgw should accept this, for better interoperability.
Context: this was encountered when using Fine Uploader https://github.com/FineUploader/fine-uploader to get files into an S3 bucket. When a user tries to upload a file which the browser can't identify (because it has an unrecognised or missing filename extension) then the Content-Type is not added to the POST'ed policy.
This is a show-stopper for uploading files which the browser isn't able to identify.
Attachments:
options-and-failing-post-20170531.txt is a radosgw debug = 20 log, showing the OPTIONS pre-flight call succeeding, and the POST failing at "env var missing in policy: Content-Type"
s3-upload-policy-missing-content-type-20170531.txt is cut-and-paste from the browser dev tools, showing the request headers and payload.
This is using Chrome 58.0.3029.110 on Linux; Ceph is 10.2.3-0ubuntu0.16.04.2 (Ubuntu packaged).
Files
Updated by Matt Benjamin almost 7 years ago
Have a working setup for testing this. Proposed quick-fix by Y. doesn't seem quite right, but working through it. Will update 7/5.
Updated by Matt Benjamin almost 7 years ago
- Priority changed from High to Normal
Updated by Matt Benjamin almost 7 years ago
- Priority changed from Normal to High
Updated by Orit Wasserman almost 7 years ago
- Status changed from New to In Progress
Updated by Matt Benjamin over 6 years ago
#reproduced with this
import requests
import os
rgw_host = os.environ['RGW_HOST']
rgw_port = int(os.environ['RGW_PORT'])
access_key = os.environ['RGW_ACCESS_KEY']
secret_key = os.environ['RGW_SECRET_KEY']
endpoint_url = "http://%s:%d" % (rgw_host, rgw_port)
s3 = boto3.client(service_name='s3',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
endpoint_url=endpoint_url,
use_ssl=False,
verify=False)
connection_type = 'client'
try:
bucket_name = os.environ['RGW_NFS_BUCKET']
except:
bucket_name = 'sorrydave'
try:
object_name = os.environ['RGW_NFS_OBJECT']
except:
object_name = 'jocamlpost'
- Generate the POST attributes
post = s3.generate_presigned_post(
Bucket=bucket_name,
Key=object_name
)
files = {"file": "file_content"}
response = requests.post(post["url"], data=post["fields"], files=files)
Updated by Matt Benjamin over 6 years ago
- Status changed from In Progress to Fix Under Review
- Backport set to luminous
Updated by Vikhyat Umrao over 6 years ago
- Backport changed from luminous to luminous,jewel
Had a discussion with Matt. Marking it for jewel backport.
Updated by Yehuda Sadeh over 6 years ago
- Status changed from Fix Under Review to 17
Updated by Dave Holland over 6 years ago
(I am the original reporter) I confirm that the patch at https://github.com/ceph/ceph/pull/18658 fixes the bug for me.
Thank you!
Dave
Updated by Nathan Cutler over 6 years ago
- Status changed from 17 to Pending Backport
Updated by Nathan Cutler over 6 years ago
- Status changed from Pending Backport to 17
This is waiting for a test case to be added to https://github.com/ceph/s3-tests
Once that is in place, we can do the ceph/ceph.git and ceph/s3-tests.git backports together.
Updated by Matt Benjamin over 6 years ago
Candidate test s3tests.functional.test_s3:test_post_object_authenticated_no_content_type pushed https://github.com/ceph/s3-tests/pull/204
Updated by Matt Benjamin over 6 years ago
- Status changed from 17 to Fix Under Review
Updated by Orit Wasserman over 6 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Nathan Cutler over 6 years ago
The s3-tests patch/commit will need to be backported to the "ceph-jewel" and "ceph-luminous" branches along with the fix, right?
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policy added
Updated by Nathan Cutler over 6 years ago
- Copied to Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policy added
Updated by Yuri Weinstein about 6 years ago
Updated by Nathan Cutler about 6 years ago
- Status changed from Pending Backport to Resolved