Ceph » rgw

I made a bucket with a single object in on CEPH and s3 with the name "prefix with + plus/file.txt". I then issued a GET on the bucket with prefix to list the top level directory of the bucket. S3 returns the Key and the CommonPrefixes url encoded, whereas CEPH only encodes the Key.

Here is the XML returned. Note the URL encoding in the s3 response for CommonPrefixes and the lack of it in the CEPH response.

CEPH

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
 <Tenant>659259d53fc74389b9d71602f8a59bf8</Tenant>
 <Name>rclone-plus</Name>
 <Prefix>
 </Prefix>
 <Marker>
 </Marker>
 <MaxKeys>1000</MaxKeys>
 <Delimiter>/</Delimiter>
 <IsTruncated>false</IsTruncated>
 <EncodingType>url</EncodingType>
 <CommonPrefixes>
  <Prefix>prefix with + plus/</Prefix>
 </CommonPrefixes>
</ListBucketResult>

s3

<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
 <Name>rclone-plus</Name>
 <Prefix>
 </Prefix>
 <Marker>
 </Marker>
 <MaxKeys>1000</MaxKeys>
 <Delimiter>/</Delimiter>
 <EncodingType>url</EncodingType>
 <IsTruncated>false</IsTruncated>
 <CommonPrefixes>
  <Prefix>prefix+with+%2B+plus/</Prefix>
 </CommonPrefixes>
</ListBucketResult>

And here are the full HTTP request and response

2019/09/16 13:44:27 DEBUG : GET /rclone-plus?delimiter=%2F&encoding-type=url&max-keys=1000&prefix= HTTP/1.1
Host: s3.eu-west-2.amazonaws.com
User-Agent: rclone/v1.49.3-DEV
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190916T124427Z
Accept-Encoding: gzip

2019/09/16 13:44:27 DEBUG : HTTP RESPONSE (req 0xc000578f00)
2019/09/16 13:44:27 DEBUG : HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Mon, 16 Sep 2019 12:44:28 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-west-2
X-Amz-Id-2: ROeSmZNyUlxLSF1CYXlWlPkd54bUwD3wi99ziPKPuGbRJO1xrahGVcMTTMUvOLXi6pRvJ22k1zs=
X-Amz-Request-Id: 2887BDCA1729272E

16c
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>rclone-plus</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><Delimiter>/</Delimiter><EncodingType>url</EncodingType><IsTruncated>false</IsTruncated><CommonPrefixes><Prefix>prefix+with+%2B+plus/</Prefix></CommonPrefixes></ListBucketResult>
0

CEPH

2019/09/16 13:43:58 DEBUG : GET /rclone-plus?delimiter=%2F&encoding-type=url&max-keys=1000&prefix= HTTP/1.1
Host: storage.dunsfold-a.openstack.memset.com:8080
User-Agent: rclone/v1.49.3-DEV
Authorization: XXXX
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20190916T124358Z
Accept-Encoding: gzip

2019/09/16 13:43:58 DEBUG : HTTP RESPONSE (req 0xc000583500)
2019/09/16 13:43:58 DEBUG : HTTP/1.1 200 OK
Content-Length: 410
Access-Control-Allow-Headers: Origin,Content-Type,Cache-Control,Content-Language,Expires,Last-Modified,Pragma,X-Requested-With,User-Agent,If-Modified-Since,Accept,X-Auth-Token
Access-Control-Max-Age: 10800
Bucket: 659259d53fc74389b9d71602f8a59bf8%2Frclone-plus
Content-Type: application/xml
Date: Mon, 16 Sep 2019 12:43:58 GMT
X-Amz-Request-Id: tx000000000000001f6a4ba-005d7f838e-3fdcb85-dunsfold-a

<?xml version="1.0" encoding="UTF-8"?><ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Tenant>659259d53fc74389b9d71602f8a59bf8</Tenant><Name>rclone-plus</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><Delimiter>/</Delimiter><IsTruncated>false</IsTruncated><EncodingType>url</EncodingType><CommonPrefixes><Prefix>prefix with + plus/</Prefix></CommonPrefixes></ListBucketResult>

I tested this with 12.2.12, luminous as it is the only platform I have easy access to. However looking at the source code I think this bug is still present in master.