Subtask #4589
closed
Feature #3761: kernel messenger: need to support multiple ops per request
Subtask #4125: kernel messenger: support multiple sources of data
libceph: consolidate maintenance of message data length
Added by Alex Elder about 11 years ago.
Updated about 11 years ago.
Description
Right now the total length of the data in a message is
modified in different places throughout the code.
This makes it difficult to be able to be sure it
is recorded consistently It is also maintained in
wire byte order in the header of the message.
This is a little tricky, because the file system
(in its writepages) actually sets up a message for
an osd request early, then fills it in, then
updates the length values (including both the
extent length recorded for the osd op within the
request and the data length field in the header).
Anyway, I'm creating this issue to identify it as
a separate block of work that's being done in order
to allow a message's data length and the fields
in osd operations get updated only inside the
osd client code.
I've done a big block of this work but it's not
quite ready to post for review.
Just a comment on the work that leads up to getting
this to happen.
My aim is to have ceph_osdc_build_request() be
responsible for computing the length of the data
portion of a request message (and response too).
It will do that based on the content of the ops
array in the request.
In order to do this, the places throughout the
code need to avoid setting these fields, and
that mostly means rearranging blocks of code
so the length(s) get set (or updated) later
than they are now, so they can then just stop
setting them and let ceph_osdc_build_request()
do that work.
One more thing. I will be recording the message data length
(in CPU byte order) in a separate field in the message,
separate from what gets written into the message header.
What I'd like to do is have the outgoing message header
get formatted completely only by the messenger and
only right before it actually gets sent over the
wire. And similarly decoding an incoming message's
header should be done once by the messenger, and
thereafter others should use (CPU order) fields in
the message structure for information about the message.
Similarly, the data portion should be formatted by
its owner (e.g., the osd client) immediately prior
queueing the message with the messenger to be sent.
All that being said, I'm not doing all of this now,
I'm just stating that this is the target I'm aiming
for eventually, and that explains a bit about why
I'm doing this with the message data length field.
- Status changed from In Progress to Fix Under Review
The following patches have been posted for review:
[PATCH 3/9] libceph: record length of bio list with bio
[PATCH 4/9] libceph: record message data length
[PATCH 5/9] libceph: don't build request in ceph_osdc_new_request()
[PATCH 6/9] ceph: define ceph_writepages_osd_request()
[PATCH 7/9] ceph: kill ceph alloc_page_vec()
[PATCH 8/9] libceph: hold off building osd request
[PATCH 9/9] ceph: build osd request message later for writepages
There may be a few other patches that I have not yet
posted for review that might be considered related to
this but I won't wait for them to close this issue.
That is, once the above are committed, I'll mark
this issue resolved.
- Status changed from Fix Under Review to Resolved
- Translation missing: en.field_remaining_hours set to 0.00
The following have been committed to the ceph-client
"testing" repository:
fb3f962 libceph: record length of bio list with bio
15a0d7b libceph: record message data length
5a7d038 libceph: don't build request in ceph_osdc_new_request()
7faaaf9 ceph: define ceph_writepages_osd_request()
ae89db2 ceph: kill ceph alloc_page_vec()
64fa40f libceph: hold off building osd request
b0bb70d ceph: build osd request message later for writepages
There are some upcoming patches related to this length
consolidation, but they're more oriented toward using the
value now that it's available.
So I'm going to mark this issue complete.
- Status changed from Resolved to Fix Under Review
- Estimated time set to 0:00 h
One more patch that probably should have been included has
now been posted for review:
[PATCH] libceph: provide data length when preparing message
- Status changed from Fix Under Review to Resolved
The following has been committed to the "testing" branch
of the ceph-client git repository:
0af231f libceph: provide data length when preparing message
Also available in: Atom
PDF