Bug #4628
Updated by Tyler Brekke about 11 years ago
(10:24:45 AM) Matt: attempting to enumerate all of his objects with boto goes on forever (10:42:27 AM) Matt: from boto import connect_s3; c = connect_s3("[REDACTED]", "[REDACTED]", host="objects.dreamhost.com") b = c.get_bucket('BUCKET') h = b.list(marker="_FOLDER/FILE.example"); j = iter(h) for n in range(1000): j.next() <get a thousand objects back, almost all of them coming before the marker object> (10:43:06 AM) yehuda@newdream.net: might be something weird going on with the preceding underscore (10:43:18 AM) Matt: that's what I was thinking (10:43:32 AM) Matt: <Key: BUCKET,__FOLDER/FILE.example> <Key: BUCKET,_ARCHIVE/> (10:43:38 AM) Matt: that seems to be where it wraps around (10:43:46 AM) Matt: if you continue to enumerate (10:44:51 AM) yehuda@newdream.net: the good news is that I just managed to reproduce the issue (10:45:56 AM) Matt: excellent (10:47:08 AM) yehuda@newdream.net: probably because we use the underscore as an escape char, we fail to escape it somewhere (10:59:18 AM) yehuda@newdream.net: didn't look at it, but it makes sense .. internally we translate a preceding underscore into two underscores (11:00:24 AM) yehuda@newdream.net: so if we don't follow that with the marker, we will just rewind the listing (11:06:05 AM) Matt: that would do it DHO would like the fix for this to be backported to Argonaut