Project

General

Profile

Bug #20201

radosgw refuses upload when Content-Type missing from POST policy

Added by Dave Holland about 2 years ago. Updated about 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
Start date:
06/06/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
luminous,jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

When doing an S3 upload, if Content-Type is missing from the policy part of the upload, then radosgw refuses the upload with a 403 error, "Policy missing condition: Content-Type". The same upload to AWS S3 is successful, despite the Content-Type not being present in the uploaded policy. radosgw should accept this, for better interoperability.

Context: this was encountered when using Fine Uploader https://github.com/FineUploader/fine-uploader to get files into an S3 bucket. When a user tries to upload a file which the browser can't identify (because it has an unrecognised or missing filename extension) then the Content-Type is not added to the POST'ed policy.

This is a show-stopper for uploading files which the browser isn't able to identify.

Attachments:

options-and-failing-post-20170531.txt is a radosgw debug = 20 log, showing the OPTIONS pre-flight call succeeding, and the POST failing at "env var missing in policy: Content-Type"

s3-upload-policy-missing-content-type-20170531.txt is cut-and-paste from the browser dev tools, showing the request headers and payload.

This is using Chrome 58.0.3029.110 on Linux; Ceph is 10.2.3-0ubuntu0.16.04.2 (Ubuntu packaged).

options-and-failing-post-20170531.txt View (27.3 KB) Dave Holland, 06/06/2017 03:00 PM

s3-upload-policy-missing-content-type-20170531.txt View (2.02 KB) Dave Holland, 06/06/2017 03:00 PM


Related issues

Copied to rgw - Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policy Resolved
Copied to rgw - Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policy Resolved

History

#1 Updated by Yehuda Sadeh about 2 years ago

  • Priority changed from Normal to High

#2 Updated by Matt Benjamin almost 2 years ago

  • Assignee set to Matt Benjamin

#3 Updated by Matt Benjamin almost 2 years ago

Have a working setup for testing this. Proposed quick-fix by Y. doesn't seem quite right, but working through it. Will update 7/5.

#4 Updated by Matt Benjamin almost 2 years ago

  • Priority changed from High to Normal

#5 Updated by Matt Benjamin almost 2 years ago

  • Priority changed from Normal to High

#6 Updated by Orit Wasserman almost 2 years ago

  • Status changed from New to In Progress

#7 Updated by Matt Benjamin over 1 year ago

#reproduced with this

import requests
import os

rgw_host = os.environ['RGW_HOST']
rgw_port = int(os.environ['RGW_PORT'])
access_key = os.environ['RGW_ACCESS_KEY']
secret_key = os.environ['RGW_SECRET_KEY']

endpoint_url = "http://%s:%d" % (rgw_host, rgw_port)

s3 = boto3.client(service_name='s3',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
endpoint_url=endpoint_url,
use_ssl=False,
verify=False)
connection_type = 'client'

try:
bucket_name = os.environ['RGW_NFS_BUCKET']
except:
bucket_name = 'sorrydave'

try:
object_name = os.environ['RGW_NFS_OBJECT']
except:
object_name = 'jocamlpost'

  1. Generate the POST attributes
    post = s3.generate_presigned_post(
    Bucket=bucket_name,
    Key=object_name
    )

files = {"file": "file_content"}
response = requests.post(post["url"], data=post["fields"], files=files)

#8 Updated by Matt Benjamin over 1 year ago

  • Status changed from In Progress to Need Review
  • Backport set to luminous

#9 Updated by Vikhyat Umrao over 1 year ago

  • Backport changed from luminous to luminous,jewel

Had a discussion with Matt. Marking it for jewel backport.

#10 Updated by Yehuda Sadeh over 1 year ago

  • Status changed from Need Review to Need Test

#11 Updated by Dave Holland over 1 year ago

(I am the original reporter) I confirm that the patch at https://github.com/ceph/ceph/pull/18658 fixes the bug for me.

Thank you!

Dave

#12 Updated by Nathan Cutler over 1 year ago

  • Status changed from Need Test to Pending Backport

#13 Updated by Nathan Cutler over 1 year ago

  • Status changed from Pending Backport to Need Test

This is waiting for a test case to be added to https://github.com/ceph/s3-tests

Once that is in place, we can do the ceph/ceph.git and ceph/s3-tests.git backports together.

#14 Updated by Matt Benjamin over 1 year ago

Candidate test s3tests.functional.test_s3:test_post_object_authenticated_no_content_type pushed https://github.com/ceph/s3-tests/pull/204

#15 Updated by Matt Benjamin over 1 year ago

  • Status changed from Need Test to Need Review

#16 Updated by Orit Wasserman over 1 year ago

  • Status changed from Need Review to Pending Backport

#17 Updated by Nathan Cutler over 1 year ago

The s3-tests patch/commit will need to be backported to the "ceph-jewel" and "ceph-luminous" branches along with the fix, right?

#18 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policy added

#19 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policy added

#21 Updated by Nathan Cutler about 1 year ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF