Ceph

Note that the filtering is being done on the osd side.

Yeah. If the filtering were done on the librados side, there would be little point to the API.

However, even though the filtering is done by the Objecter, we still need to export a reasonable API.

The Objecter is client-side. The filtering is done by the OSD, so as to preserve network bandwidth and reduce client memory use. So I don't recall what the filtering options are, but changing them will require work on the OSD and a communications protocol for them, not just Objecter stuff.

The original librados list_filter() had the following:

void Rados::list_filter(Rados::ListCtx& ctx, bufferlist& filter, bufferlist *extra_info);

• plain (IMO should be renamed to 'xattr'): filters only objects with a specific xattr value. Passed parameters are xattr name and value.
• parent: searches for entries that have a specific parent ino. Passed parameter is the ino (as integer).

We should also be able to filter entries by their name. A more generic parameter passing for the name/xattr filtering should be:
<value type, e.g., xattr or name> <filter type, e.g. prefix/contains/suffix> <value> <...>

The extra_info parameter is for extra information in the response. In the 'parent' filter it holds the back pointer.

I assume that we can define a base class for the pgls_filter iterators and specialize it according to the filter type. However, we should also make a lower level generic api available.

I assume that we can define a base class for the pgls_filter iterators and
specialize it according to the filter type.

I think my favorite approach would be having functions like this:

librados::ObjectIterator begin_objects();
librados::ObjectIterator begin_objects_filtered(const PGListFilter &filter);

Then you have a class PGListFilter with some static methods

PGListFilter PGListFilter::create_by_name(const std::string &name);
PGListFilter PGListFilter::create_by_xattr(const std::string &xattr_key, const std::string &xattr_val);

I wouldn't make any kind of lower level API available in librados. We want the flexibility to change the implementation later. Letting people fool with binary buffers will confuse users and potentially lock us in to a suboptimal implementation.

Just my $0.02

C.