Project

General

Profile

Feature #9344

add a TCP_NODELAY option

Added by Ilya Dryomov over 9 years ago. Updated about 9 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:

Description

From Chaitanya Huilgol <>

In our benchmarking tests we observed that the ms_tcp_nodelay in ceph.conf option is not affecting the kernel rbd and as expected we see poor latency numbers for lower queue depths and 4K rand reads. There is significant increase in latency from qd=2  to 24 and starts tapering down for higher queue depths.
We did not find relevant kernel_setsockopt  with TCP_NODELAY in the kernel RBD/libceph (messenger.c) source. Unless we are missing something, looks like currently the kernel RBD is not setting this and this is affecting latency numbers are lower queue depths.

I have tested with userspace fio(rbd engine) and rados bench and we see similar latency behavior when ms_tcp_nodelay is set to false. However setting this to true gives consistent low latency numbers for all queue depths

I guess we can add a tcp_nodelay rbd map/mount option for this.


Related issues

Related to Linux kernel client - Feature #9345: look into using MSG_MORE and MSG_SENDPAGE_NOTLAST more Resolved 09/04/2014

History

#1 Updated by Ilya Dryomov about 9 years ago

  • Status changed from New to Resolved

Merged into 4.0-rc1.

Also available in: Atom PDF