Project

General

Profile

Actions
Copy link

Bug #22788

closed

ceph-fuse performance issues with rsync

Added by Robert Sander about 6 years ago. Updated about 6 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
Zheng Yan
Category:
Performance/Resource Usage
Target version:
Ceph - v12.2.2
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v12.2.2
ceph-qa-suite:
Component(FS):
ceph-fuse
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hi,

I have a performance issue when running rsync on a FUSE-mounted CephFS.

dd runs on "line speed" on my test cluster:

root@ceph05:/mnt/cephfs/test# dd if=/root/tester of=/mnt/cephfs/test/tester bs=4M
1024+0 records in
1024+0 records out
4294967296 bytes (4,3 GB, 4,0 GiB) copied, 64,0135 s, 67,1 MB/s

rsync just achieves 5% of that speed:

root@ceph05:/mnt/cephfs/test# rsync -avHPSA /root/tester .
sending incremental file list
tester
64,749,568 1% 3.02MB/s 0:22:47

I see a similar performance discrepancy on a production cluster.

Do you know if rsync and FUSE do not play well together?

Actions
Copy link
 #1

Updated by Zheng Yan about 6 years ago

It seems that the -H option causes low performance when destination is in cephfs. I still don't figure out why

Actions
Copy link
 #2

Updated by Zheng Yan about 6 years ago

  • Status changed from New to 4

with -H option, rsync does 1k write. without -H option, rsync does 4k write. ceph-fuse does not enable kernel writeback cache, each write syscall on ceph-fuse mount requires a fuse upcall. that's why rsync -H (1k write) is slow.

Actions
Copy link
 #3

Updated by Patrick Donnelly about 6 years ago

  • Subject changed from Performance FUSE-mounted CephFS with rsync to ceph-fuse performance issues with rsync
  • Status changed from 4 to Won't Fix
  • Assignee set to Zheng Yan
  • ceph-qa-suite deleted (fs)
  • Component(FS) ceph-fuse added
Actions
Copy link

Also available in: Atom PDF