Project

General

Profile

Bug #48532

mgr/dashboard: OSD I/O stats freeze if OSD is DOWN

Added by Kiefer Chang about 2 years ago. Updated almost 2 years ago.

Status:
Triaged
Priority:
Normal
Assignee:
Category:
Component - OSDs
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Description of problem

The Read Bytes, Write Bytes, Read ops, and Write ops values freeze after an OSD is down.

How reproducible

  • Generate some I/Os to a pool.
  • Shutdown a host.
  • Observe the I/O stats on OSD list table.

Actual results

Screenshot after killing host node3 (I/Os are ongoing)

Screenshot after killing host node3 (I/O are canceled and everything is settled down)

Expected results

The dot-graph or Read-Write ops should become 0.

Additional info

Here are stats_history data return from backend:

DOWN OSD

"stats_history":{

    "op_w":[
        [
            1607575596.8253326,
            0.0
        ],
        [
            1607575601.8246965,
            0.0
        ],
        [
            1607575606.8268645,
            0.0
        ],
        [
            1607575611.8271983,
            0.0
        ],

IN/UP OSD

"stats_history":{

    "op_w":[
        [
            1607576260.8144872,
            0.0
        ],
        [
            1607576265.8149185,
            0.0
        ],
        [
            1607576270.815149,
            0.0
        ],
        [
            1607576275.8155153,
            0.0
        ],

The timestamps in the DOWN OSD freeze at the moment that the OSD is down.
The timestamps in the UP OSD advance with time.

a.png View (98.6 KB) Kiefer Chang, 12/10/2020 09:18 AM

b.png View (92.2 KB) Kiefer Chang, 12/10/2020 09:18 AM

History

#1 Updated by Ernesto Puerta about 2 years ago

  • Status changed from New to Triaged
  • Assignee set to Kiefer Chang

#2 Updated by Ernesto Puerta almost 2 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 138 to Component - OSDs

Also available in: Atom PDF