Ceph needs mor than 32k pids
I kinda painfully discovered that one of my Hosts with 45 OSDs on it spawned 1.4 Million threads when starting it into a recovering cluster.
About 33k of those threads are persistent which is more than the default 32k pids a linux box provides.
In my opinion the documentation should contain a note that the amount of pids should be increased:
#sysctl -w kernel.pid_max=4194303
kernel.pid_max = 4194303
(4194303 is the maximum possible)
#3 Updated by David Moreau Simard about 5 years ago
FWIW there might be a bug to extract out of this. Adding this just for cross-reference: http://lists.openstack.org/pipermail/openstack-operators/2014-August/005015.html
#5 Updated by John Wilkins about 5 years ago
- Assignee changed from John Wilkins to Alfredo Deza
Added commentary in Hardware section and in troubleshooting.
There is a note here suggesting that ceph-deploy notifies a user if the number of OSDs per node exceeds n#. That is, to suggest increasing the max threadcount.