Ceph » rgw

If resharding is started with the parameters 'verbose=true' and 'out=nullptr', like in the example:

ret = br.execute(entry.new_num_shards, max_entries, true, nullptr, formatter, this);
                                                    ^ verb|ose
                                                          ^out

Then in src/rgw/rgw_reshard.cc , function:

int RGWBucketReshard::do_reshard(...
...
for (auto iter = entries.begin(); iter != entries.end(); ++iter) {
    rgw_cls_bi_entry& entry = *iter;
    if (verbose) {
      formatter->open_object_section("entry");

      encode_json("shard_id", i, formatter);
      encode_json("num_entry", total_entries, formatter);
----> encode_json("entry", entry, formatter); 
    }

The formatter is never flushed if out=nullptr
and the "encode_json("entry", entry, formatter);"
is accumulating 1268 byte per object or roughly 1.2 GB Ram per 1000000 objects while resharding

For example content of entry, entry len is 1268 bytes:

entry":{"name":"myobjects27225067","instance":"","ver":{"pool":6,"epoch":151045},"locator":"","exists":"true","meta":{"category":1,"size":4000,"mtime":"2019-09-22 15:06:53.754838Z","etag":"6c1b4100d743a6937ad6c59244d535b3","owner":"cosbench","owner_display_name":"Test Id","content_type":"application/octet-stream","accounted_size":4000,"user_data":""},"tag":"597bae0b-2c94-4fa5-9d46-a0ee7a8c630a.4121.1205919","flags":0,"pending_map":[],"versioned_epoch":0}}}