Project

General

Profile

Bug #16654

the option 'rbd_cache_writethrough_until_flush=true' dosn't work

Added by Qi Yu almost 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
High
Target version:
-
Start date:
07/12/2016
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
jewel
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rbd
Pull request ID:

Description

Env: my ceph cluster with 252 SATA osds. the test VM kernerl version is ' 3.13.0-86-generic'

test cmd: sudo fio -name iops -rw=randwrite -bs=4k -direct=1 -thread -runtime=60 -iodepth 32 -filename /dev/vdb -ioengine=libaio

Test steps:

1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 1000

2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 3000

3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still IOPS.

So it looks like the rbd cache didn't switch from the writethrough to writeback mode .

rbd cache writethrough until flush

Description: Start out in write-through mode, and switch to write-back after the first flush request is received. Enabling this is a conservative but safe setting in case VMs running on rbd are too old to send flushes, like the virtio driver in Linux before 2.6.32.
Type: Boolean
Required: No
Default: true


Related issues

Copied to rbd - Backport #17080: jewel: the option 'rbd_cache_writethrough_until_flush=true' dosn't work Resolved

History

#1 Updated by Qi Yu almost 3 years ago

Sorry, the correct test data is :

1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 3000

2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 1000

3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still 1000 IOPS.

#2 Updated by Nathan Cutler almost 3 years ago

  • Target version deleted (519)

#3 Updated by cheng li almost 3 years ago

I meet the same problem with ceph 10.2.2. After some investigate, I think the rbd cache write-back mode was not able to be enabled because the librbd didn't change the mode from write-through to write-back when dealing with the VM OS flush.

I added this step( m_image_ctx.user_flushed() ) manually before this line https://github.com/ceph/ceph/blob/jewel/src/librbd/AioImageRequest.cc#L476
and it worked.
My question is: why librbd don't change the cache mode when dealing with VM OS flush? Did the developer forget this step? Or this step is executed at somewhere else?

#4 Updated by Jason Dillaman almost 3 years ago

  • Project changed from Ceph to rbd
  • Category deleted (librbd)
  • Status changed from New to In Progress
  • Assignee set to Jason Dillaman
  • Priority changed from Normal to High
  • Backport set to jewel

#5 Updated by Jason Dillaman almost 3 years ago

  • Status changed from In Progress to Need Review

#6 Updated by Mykola Golub almost 3 years ago

  • Status changed from Need Review to Pending Backport

#7 Updated by Loic Dachary almost 3 years ago

  • Copied to Backport #17080: jewel: the option 'rbd_cache_writethrough_until_flush=true' dosn't work added

#8 Updated by Loic Dachary almost 3 years ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF