Bug #35987
msg: AsyncConnection keeps previous message buffers until new message comes in
Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
mimic, luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Starting in the thread at http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-September/029780.html, and continuing on at http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-September/029783.html
The key claims (which appear accurate):
We can see that the memory is allocated in AsyncConnection::process, precisely here: https://github.com/ceph/ceph/blob/master/src/msg/async/AsyncConnection.cc#L614 >From my understanding - I have never read ceph code before, so my understanding might be flawed - this code is allocating a buffer to store the data contained in the message received from a ceph node and adding it to a buffers list `data_buf`. Note that this list of buffers is per AsyncConnection, that's important. Unfortunately, looking through the file shows that `data_buf` only gets cleared here: https://github.com/ceph/ceph/blob/master/src/msg/async/AsyncConnection.cc#L477 when it transitions to the STATE_OPEN_MESSAGE_HEADER state, IIUC only occurs when it starts receiving a new CEPH_MSGR_TAG_MSG message.
Related issues
History
#1 Updated by Greg Farnum over 4 years ago
- Subject changed from AsyncConnection keeps previous message buffers until new message comes in to msg: AsyncConnection keeps previous message buffers until new message comes in
- Category set to msgr
At first glance it looks we can just clear the local bufferlists we keep around once the Message is dispatched; we don't need our own copies. I'm putting together a simple patch.
#2 Updated by Greg Farnum over 4 years ago
- Status changed from New to 17
#3 Updated by Greg Farnum over 4 years ago
- Status changed from 17 to Pending Backport
- Backport set to mimic, luminous
#4 Updated by Greg Farnum over 4 years ago
- Assignee deleted (
Greg Farnum)
#5 Updated by Nathan Cutler over 4 years ago
- Copied to Backport #36126: luminous: msg: AsyncConnection keeps previous message buffers until new message comes in added
#6 Updated by Nathan Cutler over 4 years ago
- Copied to Backport #36127: mimic: msg: AsyncConnection keeps previous message buffers until new message comes in added
#7 Updated by Nathan Cutler over 4 years ago
- Status changed from Pending Backport to Resolved
#8 Updated by Greg Farnum almost 4 years ago
- Project changed from Ceph to Messengers
- Category deleted (
msgr)