Ceph » CephFS

# fio -filename=/tmp/mydir/lxb.txt -direct=0 -iodepth 1 -thread -rw=write -ioengine=psync -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file -bs=xyzK

With libfuse = 2.9.2 and set the big_writes = true:

4K   --> write: IOPS=406, BW=1626KiB/s (1665kB/s)(16.0MiB/10080msec)
8K   --> write: IOPS=420, BW=3368KiB/s (3449kB/s)(33.2MiB/10084msec)
16k  --> write: IOPS=453, BW=7258KiB/s (7432kB/s)(71.5MiB/10086msec)
32K  --> write: IOPS=391, BW=12.2MiB/s (12.8MB/s)(124MiB/10126msec)
64K  --> write: IOPS=376, BW=23.5MiB/s (24.7MB/s)(238MiB/10098msec)
128K --> write: IOPS=324, BW=40.6MiB/s (42.6MB/s)(410MiB/10091msec)
256K --> write: IOPS=268, BW=67.2MiB/s (70.5MB/s)(685MiB/10184msec)
512K --> write: IOPS=211, BW=106MiB/s (111MB/s)(1091MiB/10339msec)
1024K--> write: IOPS=122, BW=123MiB/s (128MB/s)(1280MiB/10445msec)

With libfuse = 2.9.2 and set the big_writes = false:

4K   --> write: IOPS=469, BW=1878KiB/s (1923kB/s)(18.5MiB/10086msec)
8K   --> write: IOPS=417, BW=3340KiB/s (3420kB/s)(33.0MiB/10131msec)
16k  --> write: IOPS=308, BW=4938KiB/s (5056kB/s)(48.9MiB/10149msec)
32K  --> write: IOPS=237, BW=7593KiB/s (7775kB/s)(76.1MiB/10266msec)
64K  --> write: IOPS=134, BW=8638KiB/s (8845kB/s)(87.6MiB/10388msec)
128K --> write: IOPS=68, BW=8705KiB/s (8914kB/s)(92.2MiB/10852msec)
256K --> write: IOPS=30, BW=7766KiB/s (7952kB/s)(92.8MiB/12230msec)
512K --> write: IOPS=15, BW=7960KiB/s (8151kB/s)(123MiB/15823msec)
1024K--> write: IOPS=4, BW=4968KiB/s (5087kB/s)(156MiB/32155msec)

The PR: https://github.com/ceph/ceph/pull/34531

As Zheng Yan adviced, the fix will set the big_writes options as true as default.

Then in ceph-fuse will just ignore setting the '-o big_writes' to fuse library when linking to libfuse >= 3.0.0.

About the big_writes option in libfuse: https://github.com/libfuse/libfuse/blob/master/ChangeLog.rst