Ceph » rgw

When civetweb is configured with a request_timeout_ms, mg_write() may return a short write. The loop in RGWCivetWeb::write_data() will then try to rewrite the entire buffer, instead of only the unwritten bytes.

Reproduction steps provided by joke-lee in https://github.com/ceph/ceph/pull/30257#issuecomment-551396823:

this is a file 139MiB size named 201910101336581608-201909121510598784-CMCC_CN_P855A23V1.0.0B18_TO_CMCC_CN_P855A23V1.0.0B19.zip in bucket test

set the timeout to 1s

rgw frontends = "civetweb port=7480 request_timeout_ms=1000" 

restart rgw

and download file with follow script

import socket,hashlib
host = '192.168.198.137'
uri = '/test/201910101336581608-201909121510598784-CMCC_CN_P855A23V1.0.0B18_TO_CMCC_CN_P855A23V1.0.0B19.zip?AWSAccessKeyId=yly&Expires=2203897620&Signature=FUDeErIP2JHOEHNRE42eMflfIKQ%3D'
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect((host, 7480))

domain = host +":7480" 
mysock.send('GET %s HTTP/1.0\r\nHost:%s\r\nAccept-Encoding:gzip,deflate\r\nUser-Agent:yuliyangtest13\r\nRange:bytes=0-\r\n\r\n' % (uri, domain))
count = 0
all_data = "" 
index = 0
while True:
    data = mysock.recv(5120)
    if ( len(data) < 1 ) : break
    # time.sleep(0.25)
    count = count + len(data)
    print len(data), "%sMiB" % (count/1024/1024 , )
    all_data = all_data + data

mysock.close()
# Look for the end of the header (2 CRLF)
pos = all_data.find("\r\n\r\n");
print 'Header length',pos
print all_data[:pos]
# Skip past the header and save the picture data
body = all_data[pos+4:]

with open("download3.zip", "wb") as f:
    f.write(body)