Bug #22788
closedceph-fuse performance issues with rsync
0%
Description
Hi,
I have a performance issue when running rsync on a FUSE-mounted CephFS.
dd runs on "line speed" on my test cluster:
root@ceph05:/mnt/cephfs/test# dd if=/root/tester of=/mnt/cephfs/test/tester bs=4M
1024+0 records in
1024+0 records out
4294967296 bytes (4,3 GB, 4,0 GiB) copied, 64,0135 s, 67,1 MB/s
rsync just achieves 5% of that speed:
root@ceph05:/mnt/cephfs/test# rsync -avHPSA /root/tester .
sending incremental file list
tester
64,749,568 1% 3.02MB/s 0:22:47
I see a similar performance discrepancy on a production cluster.
Do you know if rsync and FUSE do not play well together?
Updated by Zheng Yan over 6 years ago
It seems that the -H option causes low performance when destination is in cephfs. I still don't figure out why
Updated by Zheng Yan over 6 years ago
- Status changed from New to 4
with -H option, rsync does 1k write. without -H option, rsync does 4k write. ceph-fuse does not enable kernel writeback cache, each write syscall on ceph-fuse mount requires a fuse upcall. that's why rsync -H (1k write) is slow.
Updated by Patrick Donnelly about 6 years ago
- Subject changed from Performance FUSE-mounted CephFS with rsync to ceph-fuse performance issues with rsync
- Status changed from 4 to Won't Fix
- Assignee set to Zheng Yan
- ceph-qa-suite deleted (
fs) - Component(FS) ceph-fuse added