Bug #55288
closedrgw/dbstore: handle prefix/delim in Bucket::list operation
0%
Description
GET operation over a bucket on dbstore implementation, is producing a wrong set of items.
For example, if you feed two different backend, one using rados and the second using dbstore, with the same data, you will get two different output for GET:
RADOS
s3cmd ls s3://test/inner/
2022-04-08 15:17 0 s3://test/inner/myf3
2022-04-08 15:17 0 s3://test/inner/myf4
2022-04-08 15:17 0 s3://test/inner/myf5
2022-04-08 15:17 25 s3://test/inner/myfile
DBSTORE
s3cmd ls s3://test/inner/
2022-04-08 15:16 0 s3://test/inner/myf3
2022-04-08 15:16 0 s3://test/inner/myf4
2022-04-08 15:16 0 s3://test/inner/myf5
2022-04-08 15:16 25 s3://test/inner/myfile
2022-04-08 15:16 0 s3://test/myf6
2022-04-08 15:16 0 s3://test/myf7
As you can see, dbstore is wrongly returning also the parent's objects, those inside the parent directory.
It seems that GET for rados has the concept of directory and it is able to present objects as a DIR; this is not happening with dbstore:
RADOS
s3cmd ls s3://test
DIR s3://test/inner/
2022-04-08 15:17 0 s3://test/myf6
2022-04-08 15:17 0 s3://test/myf7
DBSTORE
s3cmd ls s3://test
2022-04-08 15:16 0 s3://test/inner/myf3
2022-04-08 15:16 0 s3://test/inner/myf4
2022-04-08 15:16 0 s3://test/inner/myf5
2022-04-08 15:16 25 s3://test/inner/myfile
2022-04-08 15:16 0 s3://test/myf6
2022-04-08 15:16 0 s3://test/myf7
This is leading to a quite notable and dangerous behavior on delete operations that, when are recursive, they rely on the set computed by a GET.
So, if you issue the following command:
s3cmd rm --recursive --force s3://test/inner
on dbstore, you will have the entire bucket's content being deleted.
I'm working on a patch that is preventing the most dangerous effects, but that is not covering all the aspects.
Solving this entirely probably will involve some design choices.