Project

General

Profile

Actions

Bug #46411

closed

mimic: Disks associated to osds have small write io even on an idle ceph cluster

Added by hongsong wu almost 4 years ago. Updated about 3 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

  • 1.anomalies

When the ceph cluster is idle,you can found that the disks associated to osds have small write io every bluefs_balance_interval seconds. just like below:

Device:         rrqm/s   wrqm/s     r/s     w/s    rMB/s    wMB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     0.00    0.00    4.00     0.00     0.01     4.00     0.00    0.00    0.00    0.00   0.00   0.00
sdc               0.00     0.00    0.00    4.00     0.00     0.01     4.00     0.00    0.00    0.00    0.00   0.00   0.00
sdf               0.00     0.00    0.00    4.00     0.00     0.01     4.00     0.00    0.00    0.00    0.00   0.00   0.00
sdk               0.00     0.00    0.00    4.00     0.00     0.01     4.00     0.00    0.00    0.00    0.00   0.00   0.00

  • 2.reason

This abnormal phenomenon is caused by the pr(https://github.com/ceph/ceph/pull/26866 ),so ceph version of v12.2.12~~v12.2.13, v13.2.5 ~~ v13.2.9 has the problem.
The reason is that it will lead to an empty RocksDB transaction submite to RocksDB even on an idle system,the empty transaction lead to the small write.
More details can be seen in the discussion in that pr.

  • 3.solution

check if the transaction is empty,then it's will not be submitted to the rocksdb.

Actions #1

Updated by hongsong wu almost 4 years ago

Affected Versions: v12.2.12~~v12.2.13, v13.2.5 ~~ v13.2.9

Actions #2

Updated by Igor Fedotov almost 4 years ago

  • Project changed from Ceph to bluestore
Actions #3

Updated by Igor Fedotov almost 4 years ago

  • Assignee set to Adam Kupczyk

Adam, mind taking a look?

Actions #4

Updated by hongsong wu almost 4 years ago

hongsong wu wrote:

Affected Versions: v12.2.12~~v12.2.13, v13.2.5 ~~ v13.2.10

Actions #5

Updated by hongsong wu almost 4 years ago

I am sorry, There is a problem with the pr link given above( This abnormal phenomenon is caused by the pr(https://github.com/ceph/ceph/pull/26866 ) ).

Actions #6

Updated by hongsong wu almost 4 years ago

I have fixed this bug in pr: https://github.com/ceph/ceph/pull/36108 , can you help to have a review ? @Igor Gajowiak Fedotov @Adam DC949 Kupczyk

Actions #7

Updated by Kefu Chai almost 4 years ago

  • Description updated (diff)
Actions #8

Updated by Neha Ojha over 3 years ago

  • Subject changed from Disks associated to osds have small write io even on an idle ceph cluster to mimic: Disks associated to osds have small write io even on an idle ceph cluster
  • Status changed from New to Fix Under Review
  • Pull request ID set to 36108
Actions #9

Updated by Nathan Cutler about 3 years ago

  • Status changed from Fix Under Review to Rejected

mimic EOL

Actions

Also available in: Atom PDF