Bug #16654: the option 'rbd_cache_writethrough_until_flush=true' dosn't work - rbd - Ceph

Actions

Copy link

Bug #16654

closed

the option 'rbd_cache_writethrough_until_flush=true' dosn't work

Added by Qi Yu almost 8 years ago. Updated over 7 years ago.

Status:

Resolved

Priority:

High

Assignee:

Jason Dillaman

Target version:

% Done:

Source:

Community (user)

Tags:

Backport:

jewel

Regression:

Severity:

2 - major

Reviewed:

Affected Versions:

Ceph - v10.2.0

ceph-qa-suite:

rbd

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Env: my ceph cluster with 252 SATA osds. the test VM kernerl version is ' 3.13.0-86-generic'

test cmd: sudo fio -name iops -rw=randwrite -bs=4k -direct=1 -thread -runtime=60 -iodepth 32 -filename /dev/vdb -ioengine=libaio

Test steps:

1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 1000

2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 3000

3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still IOPS.

So it looks like the rbd cache didn't switch from the writethrough to writeback mode .

rbd cache writethrough until flush

Description: Start out in write-through mode, and switch to write-back after the first flush request is received. Enabling this is a conservative but safe setting in case VMs running on rbd are too old to send flushes, like the virtio driver in Linux before 2.6.32.
Type: Boolean
Required: No
Default: true

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Qi Yu almost 8 years ago

Sorry, the correct test data is :

1. Test in the VM with 'rbd_cache_writethrough_until_flush=false' , the randwrite IOPS is 3000

2. Test in the VM with 'rbd_cache_writethrough_until_flush=true' , the randwrite IOPS is 1000

3. Test in the VM with 'rbd_cache_writethrough_until_flush=true' ,and excute 'sync' cmd to get a OS flush . but the randwrite iops is still 1000 IOPS.

Actions

Copy link

Updated by Nathan Cutler over 7 years ago

Target version deleted (~~519~~)

Actions

Copy link

Updated by cheng li over 7 years ago

I meet the same problem with ceph 10.2.2. After some investigate, I think the rbd cache write-back mode was not able to be enabled because the librbd didn't change the mode from write-through to write-back when dealing with the VM OS flush.

I added this step( m_image_ctx.user_flushed() ) manually before this line https://github.com/ceph/ceph/blob/jewel/src/librbd/AioImageRequest.cc#L476
and it worked.
My question is: why librbd don't change the cache mode when dealing with VM OS flush? Did the developer forget this step? Or this step is executed at somewhere else?

Actions

Copy link