Feature #9954
closedbuffer: method to ensure an extent is contiguous
0%
Description
Add a method to assure that an extent in a bufferlist is contigous. Something like
bufferlist bl;
...
char *ptr = bl.make_contiguous(offset, length);
If it is already contiguous, return a pointer to the position at offset.
If it is not contiguous, reallocate that range of the buffer (or do a full rebuild()). Then return a pointer to the offset.
After this call, it's safe to use ptr directly for up to length bytes from the start position.
Updated by Haomai Wang over 9 years ago
Hmm, just a another approach.
Maybe we can use another interface called "get_range" for the same goal.
1M bytes | 1M bytes | 1M bytes |
It we call make_contiguous(1024*1024-4, 8), it need call realloc and memcpy to copy 2M bytes. As for ObjectStore::Transaction usage, most of cases we only need several bytes for one argument and it's only needed for once! So here get_range will only join two or more parts and return.
"get_range" could be used more popular for most of cases and require caller only acquire a few bytes.
"make_contiguous" should be only used when we will use this buffer many times.
What do you think?
Updated by Sage Weil over 9 years ago
Haomai Wang wrote:
Hmm, just a another approach.
Maybe we can use another interface called "get_range" for the same goal.
1M bytes 1M bytes 1M bytes It we call make_contiguous(1024*1024-4, 8), it need call realloc and memcpy to copy 2M bytes. As for ObjectStore::Transaction usage, most of cases we only need several bytes for one argument and it's only needed for once! So here get_range will only join two or more parts and return.
"get_range" could be used more popular for most of cases and require caller only acquire a few bytes.
"make_contiguous" should be only used when we will use this buffer many times.What do you think?
get_range sounds like the existing copy_out()?
In the make_contiguous case, it would be for making the entire transaction contiguous so that the op offsets can just be data base pointer + offset. Note that in practice we should make this basically never result in copying any data: when we read a message off the network it is a single contiguous block of memory. Same when we read it off disk.
Note that for the data blob payload I think we still need to use substr() since it'll be non-contiguous on teh primary (where we built it up from scraps). We'll feed the buffer bits into writev() in the end so no data will be copied. It's probably just a matter of making the buffer::list manipulation as efficient as possible.
Updated by Sage Weil over 9 years ago
- Status changed from New to 4
- Source changed from other to Development
how about wip-buffer? can certainly optimize the rebuild case, but i think here we expect to never hit it. https://github.com/ceph/ceph/pull/2849
Updated by Sage Weil over 9 years ago
- Subject changed from buffer: method to ensure an extent is contiguous to buffer: method to ensure an extent is contiguous
- Status changed from 4 to Resolved