Project

General

Profile

Actions

Bug #39173

closed

ceph-fuse: Ceph-fuse too slow compared to kernel client

Added by Shantur Rathore about 5 years ago. Updated about 5 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
-
Category:
Performance/Resource Usage
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
ceph-fuse
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I have a ceph cluster with 10 osd nodes, 50 osds, 5 mons, 5 mds (1 active, 4standby).
Ceph cluster is running latest nautilus 14.2

I recently started testing Ceph fs and find ceph-fuse client very slow.

The client machine is running ubuntu 18.04.2 with mainline kernel 4.18.0
I have cephfs mounted on same machine in 2 different folders one with ceph-fuse (/cephfs) and one kernel client (/cephfs-k)

root@gen8-5:/# mount | grep cephfs
ceph-fuse on /cephfs type fuse.ceph-fuse (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)
10.187.21.31:6789:/ on /cephfs-k type ceph (rw,relatime,name=admin,secret=<hidden>,acl)

root@gen8-5:/# dd if=/dev/zero of=/cephfs-k/test/test bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes (6.6 GB, 6.1 GiB) copied, 3.64737 s, 1.8 GB/s

root@gen8-5:/# dd if=/dev/zero of=/cephfs/test/test bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes (6.6 GB, 6.1 GiB) copied, 96.5499 s, 67.9 MB/s

root@gen8-5:/# ceph-fuse -v
ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus (stable)

Kernel client gives me speeds of 1.8GB/s while fuse client gives around 68MB/s. Not sure if i am missing something.

Not sure if its related but I also tried to export NFS via Ganesha using FSAL Ceph and get speed similar to ceph-fuse.

Actions

Also available in: Atom PDF