Project

General

Profile

Feature #48292

cephadm: allow more than 60 OSDs per host

Added by Sebastian Wagner over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Category:
cephadm/osd
Target version:
-
% Done:

0%

Source:
Tags:
low-hanging-fruit
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

If the cluster is set to have very dense nodes (>60 OSDs per host) please make sure to assign sufficient ports for Ceph OSDs. The default (6800-7300) currently allows for no more than 62 OSDs per host.
For cluster with dense nodes like this please adjust the setting "ms_bind_port_max" to a suitable value. Each OSD will consume 8 additional ports.
For example, given a host that is set to run 96 OSDs, 768 ports will be needed. "ms_bind_port_max" should be set at least to 7568 by running

ceph config set osd.* ms_bind_port_max 7568.

Related issues

Related to Orchestrator - Bug #50526: OSD massive creation: OSDs not created Resolved
Related to Orchestrator - Bug #47873: /usr/lib/sysctl.d/90-ceph-osd.conf getting installed in container, rendering it ineffective Resolved

History

#1 Updated by Sebastian Wagner about 3 years ago

  • Tags set to low-hanging-fruit

#2 Updated by Sejal Welankar almost 3 years ago

Sebastian Wagner wrote:

If the cluster is set to have very dense nodes (>60 OSDs per host) please make sure to assign sufficient ports for Ceph OSDs. The default (6800-7300) currently allows for no more than 62 OSDs per host.
For cluster with dense nodes like this please adjust the setting "ms_bind_port_max" to a suitable value. Each OSD will consume 8 additional ports.
For example, given a host that is set to run 96 OSDs, 768 ports will be needed. "ms_bind_port_max" should be set at least to 7568 by running

[...]

Can I work on this issue?

#3 Updated by Sebastian Wagner almost 3 years ago

  • Assignee set to Sejal Welankar

#4 Updated by Sebastian Wagner almost 3 years ago

  • Related to Bug #50526: OSD massive creation: OSDs not created added

#5 Updated by Sebastian Wagner almost 3 years ago

  • Related to Bug #47873: /usr/lib/sysctl.d/90-ceph-osd.conf getting installed in container, rendering it ineffective added

#6 Updated by Sebastian Wagner over 2 years ago

  • Assignee deleted (Sejal Welankar)

#7 Updated by Sebastian Wagner over 2 years ago

  • Assignee set to Sebastian Wagner
  • Pull request ID set to 42210

#8 Updated by Sebastian Wagner over 2 years ago

  • Status changed from New to Fix Under Review

#9 Updated by Kefu Chai over 2 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF