Feature #3494
closedceph S3 upload slowly
0%
Description
Hi, there!
Nowadays I have tried some tests on ceph S3 python API, But I have found that it is very slow to update a big file through the S3 API
Here is my upload program:
*_import boto
import boto.s3.connection
access_key = '70XK3XKRPNMCDOHAWUJY'
secret_key = 'epJ1UXT8hYdwxDnKeznxkCy+UL83TVvhhwhVp5uz'
conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = '192.168.0.18',
is_secure = False,
calling_format=boto.s3.connection.OrdinaryCallingFormat(), )
for bucket in conn.get_all_buckets():
key = bucket.new_key('movie.1')
key.set_contents_from_filename('/root/movie')
for key in bucket.list():
print "{name}\t{size}\t{modified}".format(
name=key.name,
size=key.size,
modified=key.last_modified,
)*
_
And the size of the movie.1 is 1GB, it takes 4 minutes to complete the upload…
It takes only 1m23s to complete the upload by ceph POSIX API after I have mounted the 192.168.0.18:/ to /mnt/ceph
I am wandering why?
Another question is that how the file is stored on the osds when I upload it through the S3 API?
After uploading an 1GB file, using du command in the /ceph_osd0/osd0/current directory, I can get this :
1082864 ./7.7_head
Is the file stored on the osd as a hole?
Can I configure some parameters to strip the file to make it stored like the file uploaded by the POSIX API?
Looking for your reply…
Thank you very much!