Ceph » rgw

I uploaded a file of 55049805 bytes in chunked upload mode, and RGW returned 403 incorrectly. By tracing the execution process, I found that there was an error in get_padding_last_aws4_chunk_encoded().

In RGWPutObj_ObjStore:: get_data(), it first reads a piece of data with rgw_max_chunk_size bytes in length, which contains multiple chunks. In most cases, the last chunk is incomplete, so get_padding_last_aws4_chunk_encoded() is called here to find the remaining data length of the last chunk and read it out to pad the last chunk. In some special cases, this piece of data may end up with a truncated chunk header. If this happens, in get_padding_last_aws4_chunk_encoded(), at the beginning of the last loop, there is an incomplete header remaining in the buffer. When checking whether the remaining data length is less than the minimum chunk header length, an ERR_SIGNATURE_NO_MATCH error is returned. Eventually, the server returns 403.

I've fixed it by stashing the truncated header and splicing it at the head of the receive buffer in the next round of reading.

Pull request：https://github.com/ceph/ceph/pull/28746

More info about chunked upload: https://docs.aws.amazon.com/zh_cn/AmazonS3/latest/API/sigv4-streaming.html