Project

General

Profile

Bug #35987

msg: AsyncConnection keeps previous message buffers until new message comes in

Added by Greg Farnum 3 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
msgr
Target version:
-
Start date:
09/14/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
mimic, luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

Starting in the thread at http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-September/029780.html, and continuing on at http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-September/029783.html

The key claims (which appear accurate):

We can see that the memory is allocated in AsyncConnection::process,
precisely here:
https://github.com/ceph/ceph/blob/master/src/msg/async/AsyncConnection.cc#L614

>From my understanding - I have never read ceph code before, so my
understanding might be flawed - this code is allocating a buffer to store
the data contained in the message received from a ceph node and adding it
to a buffers list `data_buf`.
Note that this list of buffers is per AsyncConnection, that's important.

Unfortunately, looking through the file shows that `data_buf` only gets
cleared here:
https://github.com/ceph/ceph/blob/master/src/msg/async/AsyncConnection.cc#L477
when it transitions to the STATE_OPEN_MESSAGE_HEADER state, IIUC only
occurs when it starts receiving a new CEPH_MSGR_TAG_MSG message.


Related issues

Copied to Ceph - Backport #36126: luminous: msg: AsyncConnection keeps previous message buffers until new message comes in Resolved
Copied to Ceph - Backport #36127: mimic: msg: AsyncConnection keeps previous message buffers until new message comes in Resolved

History

#1 Updated by Greg Farnum 3 months ago

  • Subject changed from AsyncConnection keeps previous message buffers until new message comes in to msg: AsyncConnection keeps previous message buffers until new message comes in
  • Category set to msgr

At first glance it looks we can just clear the local bufferlists we keep around once the Message is dispatched; we don't need our own copies. I'm putting together a simple patch.

#2 Updated by Greg Farnum 3 months ago

  • Status changed from New to Need Test

#3 Updated by Greg Farnum 3 months ago

  • Status changed from Need Test to Pending Backport
  • Backport set to mimic, luminous

#4 Updated by Greg Farnum 3 months ago

  • Assignee deleted (Greg Farnum)

#5 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #36126: luminous: msg: AsyncConnection keeps previous message buffers until new message comes in added

#6 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #36127: mimic: msg: AsyncConnection keeps previous message buffers until new message comes in added

#7 Updated by Nathan Cutler 2 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF