Ceph

Hmm, just a another approach.
Maybe we can use another interface called "get_range" for the same goal.

It we call make_contiguous(1024*1024-4, 8), it need call realloc and memcpy to copy 2M bytes. As for ObjectStore::Transaction usage, most of cases we only need several bytes for one argument and it's only needed for once! So here get_range will only join two or more parts and return.

"get_range" could be used more popular for most of cases and require caller only acquire a few bytes.
"make_contiguous" should be only used when we will use this buffer many times.

What do you think?

Haomai Wang wrote:

Hmm, just a another approach.
Maybe we can use another interface called "get_range" for the same goal.

1M bytes

1M bytes

1M bytes

It we call make_contiguous(1024*1024-4, 8), it need call realloc and memcpy to copy 2M bytes. As for ObjectStore::Transaction usage, most of cases we only need several bytes for one argument and it's only needed for once! So here get_range will only join two or more parts and return.

"get_range" could be used more popular for most of cases and require caller only acquire a few bytes.
"make_contiguous" should be only used when we will use this buffer many times.

What do you think?

get_range sounds like the existing copy_out()?

In the make_contiguous case, it would be for making the entire transaction contiguous so that the op offsets can just be data base pointer + offset. Note that in practice we should make this basically never result in copying any data: when we read a message off the network it is a single contiguous block of memory. Same when we read it off disk.

Note that for the data blob payload I think we still need to use substr() since it'll be non-contiguous on teh primary (where we built it up from scraps). We'll feed the buffer bits into writev() in the end so no data will be copied. It's probably just a matter of making the buffer::list manipulation as efficient as possible.