Project

General

Profile

Actions
Copy link

Bug #39173

closed

ceph-fuse: Ceph-fuse too slow compared to kernel client

Added by Shantur Rathore about 5 years ago. Updated about 5 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
-
Category:
Performance/Resource Usage
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v14.2.0
ceph-qa-suite:
Component(FS):
ceph-fuse
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I have a ceph cluster with 10 osd nodes, 50 osds, 5 mons, 5 mds (1 active, 4standby).
Ceph cluster is running latest nautilus 14.2

I recently started testing Ceph fs and find ceph-fuse client very slow.

The client machine is running ubuntu 18.04.2 with mainline kernel 4.18.0
I have cephfs mounted on same machine in 2 different folders one with ceph-fuse (/cephfs) and one kernel client (/cephfs-k)

root@gen8-5:/# mount | grep cephfs
ceph-fuse on /cephfs type fuse.ceph-fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
10.187.21.31:6789:/ on /cephfs-k type ceph (rw,relatime,name=admin,secret=<hidden>,acl)

root@gen8-5:/# dd if=/dev/zero of=/cephfs-k/test/test bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes (6.6 GB, 6.1 GiB) copied, 3.64737 s, 1.8 GB/s

root@gen8-5:/# dd if=/dev/zero of=/cephfs/test/test bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes (6.6 GB, 6.1 GiB) copied, 96.5499 s, 67.9 MB/s

root@gen8-5:/# ceph-fuse -v
ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus (stable)

Kernel client gives me speeds of 1.8GB/s while fuse client gives around 68MB/s. Not sure if i am missing something.

Not sure if its related but I also tried to export NFS via Ganesha using FSAL Ceph and get speed similar to ceph-fuse.

Actions
Copy link
 #1

Updated by Patrick Donnelly about 5 years ago

  • Status changed from New to Won't Fix

Sorry this isn't a bug and it's well documented everywhere. The userspace client libraries (and FUSE) are known to be slower than the kernel. There are no plans to improve this presently.

Actions
Copy link
 #2

Updated by Shantur Rathore about 5 years ago

Sure, It would help to mention the level of performance difference in the documents here http://docs.ceph.com/docs/master/cephfs/best-practices/#which-client

It says
"The FUSE client is the most accessible and the easiest to upgrade to the version of Ceph used by the storage cluster, while the kernel client will often give better performance."

This is quite confusing as the performance different is around 300x, in which case it should be clear that fuse client is really really slow.

Actions
Copy link

Also available in: Atom PDF