Project

General

Profile

Fix #20330

msg : tcp backlog parameter increase

Added by red ref over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
msgr
Target version:
Start date:
06/16/2017
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

On a quite large cluster, during a cold restart phase, I noticed from tcp metrics some "ListenOverflows" and "ListenDrops" from /proc/net/netstat. On the other side, OSD's were failing to join MON's or peers (according to logs). Digging into, I tried to increase sysctl "net.core.somaxconn" kernel parameter (from some tutorials). Unfortunately, this parameter did not solved this issue. According to "ss -ln", nothing changed on the socket side.

Looking at linux documentation, I recompiled my packages modifying "128" hardcoded values from :
- https://github.com/ceph/ceph/blob/master/src/msg/simple/Accepter.cc : line 214
- https://github.com/ceph/ceph/blob/master/src/msg/async/PosixStack.cc : line 325
- https://github.com/ceph/ceph/blob/master/src/msg/async/rdma/RDMAServerSocketImpl.cc : line 59 (I do not use RDMA)

Values from "ss -ln" changed well this time and cluster reboot became smoother.

I think those values might evolve to something greater (> 1024) or even the maximum stock kernel accepts (65535) as it is capped by "somaxconn" value.

Another possibility would be to make it configurable.

History

#1 Updated by Kefu Chai over 1 year ago

  • Status changed from New to Need Review
  • Assignee set to Haomai Wang

#2 Updated by Sage Weil over 1 year ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF