7 Best Practices to Maximize Your Ceph Cluster's Performance¶

Looking for ways to make your Ceph cluster run faster and stronger? Review this best practice checklist to make sure your cluster's working at its max.

1. Monitor nodes are critical for the proper operation of the cluster. Try and use dedicated monitor nodes to make sure they have exclusive access to resources or, if running in shared environments, fence off monitor processes. For redundancy, distribute monitor nodes across data centers or availability zones.
2. On-disk journals can halve write throughput to the cluster. Ideally, you should run operating systems, OSD data and OSD journals on separate drives to maximize overall throughput. Consider using SSD journals for high write throughput workloads.
3. Erasure coding is a data-durability feature for object storage. Use erasure coding when storing large amounts of write-once, read-infrequently data where performance is less critical than cost. But remember that there's a trade-off: erasure coding can substantially lower the cost per gigabyte but has lower IOPS performance vs replication.
4. Favoring dentry and inode cache can improve performance, especially on clusters with many small objects.
5. Use cache tiering to boost the performance of your cluster by automatically migrating data between hot and cold tiers based on demand. For maximum performance, use SSDs for the cache pool and host the pool on servers with lower latency.
6. Deploy an odd number of monitors (3 or 5) for quorum voting. Adding more monitors makes your cluster more durable; however, this can sometimes reduce performance because there's more data to keep in sync between monitors.
7. When diagnosing performance issues in your cluster, always start at the lowest level (the disks, network, or other hardware) and work your way up to the higher-level interfaces (block devices and object gateways).

Sources:
http://www.slideshare.net/Inktank_Ceph/dell-ceph-22nd-london-v5
http://ceph.com/docs/master/start/hardware-recommendations/
http://www.inktank.com/inktank-ceph-enterprise/inktank-ceph-enterprise-1-2-arrives-with-erasure-coding-and-cache-tiering