Bug #16654
closedthe option 'rbd_cache_writethrough_until_flush=true' dosn't work
0%
Description
Env: my ceph cluster with 252 SATA osds. the test VM kernerl version is ' 3.13.0-86-generic'
test cmd: sudo fio -name iops -rw=randwrite -bs=4k -direct=1 -thread -runtime=60 -iodepth 32 -filename /dev/vdb -ioengine=libaio
Test steps:
1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 1000
2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 3000
3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still IOPS.
So it looks like the rbd cache didn't switch from the writethrough to writeback mode .
rbd cache writethrough until flush
Description: Start out in write-through mode, and switch to write-back after the first flush request is received. Enabling this is a conservative but safe setting in case VMs running on rbd are too old to send flushes, like the virtio driver in Linux before 2.6.32.
Type: Boolean
Required: No
Default: true
Updated by Qi Yu almost 8 years ago
Sorry, the correct test data is :
1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 3000
2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 1000
3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still 1000 IOPS.
Updated by cheng li over 7 years ago
I meet the same problem with ceph 10.2.2. After some investigate, I think the rbd cache write-back mode was not able to be enabled because the librbd didn't change the mode from write-through to write-back when dealing with the VM OS flush.
I added this step( m_image_ctx.user_flushed() ) manually before this line https://github.com/ceph/ceph/blob/jewel/src/librbd/AioImageRequest.cc#L476
and it worked.
My question is: why librbd don't change the cache mode when dealing with VM OS flush? Did the developer forget this step? Or this step is executed at somewhere else?
Updated by Jason Dillaman over 7 years ago
- Project changed from Ceph to rbd
- Category deleted (
librbd) - Status changed from New to In Progress
- Assignee set to Jason Dillaman
- Priority changed from Normal to High
- Backport set to jewel
Updated by Jason Dillaman over 7 years ago
- Status changed from In Progress to Fix Under Review
Updated by Mykola Golub over 7 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Loïc Dachary over 7 years ago
- Copied to Backport #17080: jewel: the option 'rbd_cache_writethrough_until_flush=true' dosn't work added
Updated by Loïc Dachary over 7 years ago
- Status changed from Pending Backport to Resolved