libceph: support a list of data items in a message
This will require a number of patches to complete, but
it's a logical unit of work.
An abstracted data item is now used to represent a block
of data, either a list of pages, a page array, or a bio
list. What we will want for multiple ops is to allow
more than one such data item to be associated with a
message. That way, an osd request can build up a single
request message containing an array of ops, and each of
those ops can independently append a data item to be used
for outgoing or incoming data, in whatever form is required
(page list, bio, etc.).
This depends on completion of the work to abstract the
data item, and to use it for both read and write requests.
The general sequence I intend to follow is:
- collapse the separate structures representing data in
a message (now named "p", "l", and "b" for a page array,
page list, and bio list) into a single item. It turns
out only one of these is ever (currently) used for any
message (though that wasn't obvious before). They'll
all be represented by a structure "data" that will be
handled by the messenger according to its representation.
- Replace the single "data" structure with a pointer to such
a structure. Allocate that structure dynamically only
when a message will have a data portion.
- Implement a list of these structures, with the "data"
field in the message tracking the head (and tail) of
data items on that list.
- Have the callers that now "set" the data field instead
"add" it to the existing data in a message.
- Use this capability in the osd client for multiple ops
that carry data. (This may just be a contrived test
This is related to http://tracker.ceph.com/issues/3761.
#5 Updated by Alex Elder over 7 years ago
(The following really applies to this issue as well
as 4428, 4427, and 4426.)
The patch(es) for this were posted and reviewed by Josh.
Just before I went on vacation I was doing some final
testing of this before committing it and ran into some
strange crashes involving apparmor. Not having time
to investigate, I left things as they were.
Today I've updated the patches so they're now based on
the latest "testing" branch and have been testing with
that result. At this point I have seen no errors, but
I'm going to let my current testing run to completion,
and may run it through some additional tests.
If I see another crash while doing this I'll try to
get to the bottom of it. It may be a memory leak
or something (because I can't think of how apparmor
would be involved otherwise).
If I see no crash, I think I'll commit these changes
and hope for the best...
#6 Updated by Alex Elder over 7 years ago
This ran these tests, along with a "full" xfstests run
overnight with no problems:
I'm starting the same tests up again, iterating xfstests
3 times and reducing the injected socket frequency by
a factor of 10 (to get the testing done more quickly).
I'm prepared to check all this in but I'll wait to discuss
it at the stand up meeting (by which point much of the
above retesting will have been done).