Project

General

Profile

Fix #20330

msg : tcp backlog parameter increase

Added by red ref about 2 years ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
06/16/2017
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Reviewed:
ceph-qa-suite:
Pull request ID:

Description

On a quite large cluster, during a cold restart phase, I noticed from tcp metrics some "ListenOverflows" and "ListenDrops" from /proc/net/netstat. On the other side, OSD's were failing to join MON's or peers (according to logs). Digging into, I tried to increase sysctl "net.core.somaxconn" kernel parameter (from some tutorials). Unfortunately, this parameter did not solved this issue. According to "ss -ln", nothing changed on the socket side.

Looking at linux documentation, I recompiled my packages modifying "128" hardcoded values from :
- https://github.com/ceph/ceph/blob/master/src/msg/simple/Accepter.cc : line 214
- https://github.com/ceph/ceph/blob/master/src/msg/async/PosixStack.cc : line 325
- https://github.com/ceph/ceph/blob/master/src/msg/async/rdma/RDMAServerSocketImpl.cc : line 59 (I do not use RDMA)

Values from "ss -ln" changed well this time and cluster reboot became smoother.

I think those values might evolve to something greater (> 1024) or even the maximum stock kernel accepts (65535) as it is capped by "somaxconn" value.

Another possibility would be to make it configurable.

History

#1 Updated by Kefu Chai about 2 years ago

  • Status changed from New to Need Review
  • Assignee set to Haomai Wang

#2 Updated by Sage Weil almost 2 years ago

  • Status changed from Need Review to Resolved

#3 Updated by Greg Farnum 3 months ago

  • Project changed from Ceph to Messengers
  • Category deleted (msgr)

Also available in: Atom PDF