Ceph » rgw

We are using minio-py to upload files through RGW. Uploading files over 5MiB fails with `SignatureDoesNotMatch`. This is the threshold where minio-py switches to a multipart upload strategy.

The following code is enough to reproduce the issue with Python 3.6.6, Ceph 12.2.8 and minio-py 4.0.8:

import os
from minio import Minio

client = Minio('rgw.vis.ethz.ch',
                access_key='<our access key>',
                secret_key='<our secret key>',
                secure=True)

fname = '<path to some local file larger than 5MiB>'
with open(fname, 'rb') as file_data:
        file_stat = os.stat(fname)
        client.put_object('<our bucket name>', '<some remote path>', file_data,
                        file_stat.st_size, content_type='<some content type>')

It fails with the following stacktrace:

Traceback (most recent call last):
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1555, in _stream_put_object
    upload_result = pool.result()
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/thread_pool.py", line 92, in result
    raise self.exceptions_queue.get()
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/thread_pool.py", line 57, in run
    result = func(*args, **kargs)
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1502, in _upload_part_routine
    upload_id, part_number, sse=sse)
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1491, in _do_put_object
    content_sha256=sha256_hex
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1832, in _url_open
    object_name).get_exception()
minio.error.SignatureDoesNotMatch: SignatureDoesNotMatch: message: The request signature we calculated does not match the signature you provided.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 14, in <module>
    file_stat.st_size, content_type='application/pdf')
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 782, in put_object
    data, length, metadata=metadata, sse=sse)
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1561, in _stream_put_object
    upload_id)
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1618, in _remove_incomplete_upload
    headers={})
  File "/home/philippe/temp/minio-py-debug/.venv/lib/python3.6/site-packages/minio/api.py", line 1832, in _url_open
    object_name).get_exception()
minio.error.SignatureDoesNotMatch: SignatureDoesNotMatch: message: The request signature we calculated does not match the signature you provided.

The issue does not occur with files smaller than 5 MiB. The issue does not occur when using Minio as the S3 server. The issue does not occur when uploading files through RGW using the Minio CLI.

I have filed a corresponding issue in minio-py: https://github.com/minio/minio-py/issues/724