Project

General

Profile

Actions

Bug #20201

closed

radosgw refuses upload when Content-Type missing from POST policy

Added by Dave Holland almost 7 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
luminous,jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

When doing an S3 upload, if Content-Type is missing from the policy part of the upload, then radosgw refuses the upload with a 403 error, "Policy missing condition: Content-Type". The same upload to AWS S3 is successful, despite the Content-Type not being present in the uploaded policy. radosgw should accept this, for better interoperability.

Context: this was encountered when using Fine Uploader https://github.com/FineUploader/fine-uploader to get files into an S3 bucket. When a user tries to upload a file which the browser can't identify (because it has an unrecognised or missing filename extension) then the Content-Type is not added to the POST'ed policy.

This is a show-stopper for uploading files which the browser isn't able to identify.

Attachments:

options-and-failing-post-20170531.txt is a radosgw debug = 20 log, showing the OPTIONS pre-flight call succeeding, and the POST failing at "env var missing in policy: Content-Type"

s3-upload-policy-missing-content-type-20170531.txt is cut-and-paste from the browser dev tools, showing the request headers and payload.

This is using Chrome 58.0.3029.110 on Linux; Ceph is 10.2.3-0ubuntu0.16.04.2 (Ubuntu packaged).


Files


Related issues 2 (0 open2 closed)

Copied to rgw - Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policyResolvedNathan CutlerActions
Copied to rgw - Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policyResolvedMatt BenjaminActions
Actions #1

Updated by Yehuda Sadeh almost 7 years ago

  • Priority changed from Normal to High
Actions #2

Updated by Matt Benjamin almost 7 years ago

  • Assignee set to Matt Benjamin
Actions #3

Updated by Matt Benjamin almost 7 years ago

Have a working setup for testing this. Proposed quick-fix by Y. doesn't seem quite right, but working through it. Will update 7/5.

Actions #4

Updated by Matt Benjamin almost 7 years ago

  • Priority changed from High to Normal
Actions #5

Updated by Matt Benjamin almost 7 years ago

  • Priority changed from Normal to High
Actions #6

Updated by Orit Wasserman almost 7 years ago

  • Status changed from New to In Progress
Actions #7

Updated by Matt Benjamin over 6 years ago

#reproduced with this

import requests
import os

rgw_host = os.environ['RGW_HOST']
rgw_port = int(os.environ['RGW_PORT'])
access_key = os.environ['RGW_ACCESS_KEY']
secret_key = os.environ['RGW_SECRET_KEY']

endpoint_url = "http://%s:%d" % (rgw_host, rgw_port)

s3 = boto3.client(service_name='s3',
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
endpoint_url=endpoint_url,
use_ssl=False,
verify=False)
connection_type = 'client'

try:
bucket_name = os.environ['RGW_NFS_BUCKET']
except:
bucket_name = 'sorrydave'

try:
object_name = os.environ['RGW_NFS_OBJECT']
except:
object_name = 'jocamlpost'

  1. Generate the POST attributes
    post = s3.generate_presigned_post(
    Bucket=bucket_name,
    Key=object_name
    )

files = {"file": "file_content"}
response = requests.post(post["url"], data=post["fields"], files=files)

Actions #8

Updated by Matt Benjamin over 6 years ago

  • Status changed from In Progress to Fix Under Review
  • Backport set to luminous
Actions #9

Updated by Vikhyat Umrao over 6 years ago

  • Backport changed from luminous to luminous,jewel

Had a discussion with Matt. Marking it for jewel backport.

Actions #10

Updated by Yehuda Sadeh over 6 years ago

  • Status changed from Fix Under Review to 17
Actions #11

Updated by Dave Holland over 6 years ago

(I am the original reporter) I confirm that the patch at https://github.com/ceph/ceph/pull/18658 fixes the bug for me.

Thank you!

Dave

Actions #12

Updated by Nathan Cutler over 6 years ago

  • Status changed from 17 to Pending Backport
Actions #13

Updated by Nathan Cutler over 6 years ago

  • Status changed from Pending Backport to 17

This is waiting for a test case to be added to https://github.com/ceph/s3-tests

Once that is in place, we can do the ceph/ceph.git and ceph/s3-tests.git backports together.

Actions #14

Updated by Matt Benjamin over 6 years ago

Candidate test s3tests.functional.test_s3:test_post_object_authenticated_no_content_type pushed https://github.com/ceph/s3-tests/pull/204

Actions #15

Updated by Matt Benjamin over 6 years ago

  • Status changed from 17 to Fix Under Review
Actions #16

Updated by Orit Wasserman over 6 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #17

Updated by Nathan Cutler over 6 years ago

The s3-tests patch/commit will need to be backported to the "ceph-jewel" and "ceph-luminous" branches along with the fix, right?

Actions #18

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22591: luminous: radosgw refuses upload when Content-Type missing from POST policy added
Actions #19

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22592: jewel: radosgw refuses upload when Content-Type missing from POST policy added
Actions #21

Updated by Nathan Cutler about 6 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF