Ceph » rgw

(10:24:45 AM) Matt: attempting to enumerate all of his objects with boto goes on forever
(10:42:27 AM) Matt:
from boto import connect_s3;
c = connect_s3("[REDACTED]", "[REDACTED]", host="objects.dreamhost.com")
b = c.get_bucket('BUCKET')
h = b.list(marker="_FOLDER/FILE.example");
j = iter(h)
for n in range(1000):
j.next()
<get a thousand objects back, almost all of them coming before the marker object>
(10:43:06 AM) yehuda@newdream.net: might be something weird going on with the preceding underscore
(10:43:18 AM) Matt: that's what I was thinking
(10:43:32 AM) Matt:
<Key: BUCKET,__FOLDER/FILE.example>
<Key: BUCKET,_ARCHIVE/>

(10:43:38 AM) Matt: that seems to be where it wraps around
(10:43:46 AM) Matt: if you continue to enumerate
(10:44:51 AM) yehuda@newdream.net: the good news is that I just managed to reproduce the issue
(10:45:56 AM) Matt: excellent
(10:47:08 AM) yehuda@newdream.net: probably because we use the underscore as an escape char, we fail to escape it somewhere
(10:59:18 AM) yehuda@newdream.net: didn't look at it, but it makes sense .. internally we translate a preceding underscore into two underscores
(11:00:24 AM) yehuda@newdream.net: so if we don't follow that with the marker, we will just rewind the listing
(11:06:05 AM) Matt: that would do it

DHO would like the fix for this to be backported to Argonaut