Project

General

Profile

Bug #18338

bluestore_preextend_wal_files appears to corrupt logs

Added by Sage Weil almost 3 years ago. Updated over 2 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
Start date:
12/22/2016
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

reported by ma jianpeng. working to reproduce.


Related issues

Related to bluestore - Bug #18375: bluestore: bluefs_preextend_wal_files=true is not crash consistent Resolved 12/31/2016

Associated revisions

Revision c703cf9a (diff)
Added by Vitaliy Filippov 7 months ago

osd/bluestore: Actually wait until completion in write_sync

This function is only used by RocksDB WAL writing so it must sync data.

This fixes #18338 and thus allows to actually set `bluefs_preextend_wal_files`
to true, gaining +100% single-thread write iops in disk-bound (HDD or bad SSD) setups.
To my knowledge it doesn't hurt performance in other cases.
Test it yourself on any HDD with `fio -ioengine=rbd -direct=1 -bs=4k -iodepth=1`.

Issue #18338 is easily reproduced without this patch by issuing a `kill -9` to the OSD
while doing `fio -ioengine=rbd -direct=1 -bs=4M -iodepth=16`.

Fixes: https://tracker.ceph.com/issues/18338 https://tracker.ceph.com/issues/38559
Signed-off-by: Vitaliy Filippov <>

Revision 47cd1afa (diff)
Added by Vitaliy Filippov 4 months ago

osd/bluestore: Actually wait until completion in write_sync

This function is only used by RocksDB WAL writing so it must sync data.

This fixes #18338 and thus allows to actually set `bluefs_preextend_wal_files`
to true, gaining +100% single-thread write iops in disk-bound (HDD or bad SSD) setups.
To my knowledge it doesn't hurt performance in other cases.
Test it yourself on any HDD with `fio -ioengine=rbd -direct=1 -bs=4k -iodepth=1`.

Issue #18338 is easily reproduced without this patch by issuing a `kill -9` to the OSD
while doing `fio -ioengine=rbd -direct=1 -bs=4M -iodepth=16`.

Fixes: https://tracker.ceph.com/issues/18338 https://tracker.ceph.com/issues/38559
Signed-off-by: Vitaliy Filippov <>
(cherry picked from commit c703cf9a7632cbd9f17e148ef203509549a28571)

Conflicts:
src/os/bluestore/KernelDevice.cc
- mimic has a single variable "fd_buffered" where master has an array "fd_buffereds"

History

#1 Updated by Sage Weil almost 3 years ago

In the meantime, disabling the option: https://github.com/ceph/ceph/pull/12617

#2 Updated by Sage Weil almost 3 years ago

  • Priority changed from Immediate to Urgent

#3 Updated by Sage Weil over 2 years ago

  • Related to Bug #18375: bluestore: bluefs_preextend_wal_files=true is not crash consistent added

#4 Updated by Sage Weil over 2 years ago

  • Status changed from Need More Info to Duplicate

Also available in: Atom PDF