Project

General

Profile

Actions

Bug #45185

closed

mgr/dashboard: fix usage calculation to match "ceph df" way

Added by Ernesto Puerta about 4 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
High
Category:
Component - Pools
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
octopus, nautilus
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Currently Ceph-dashboard Pool list Usage % is calculated as:

usage = raw_used / (raw_used + max_avail)

While ceph df is:

usage = data_used / (data_used + max_avail)

Related issues 5 (2 open3 closed)

Related to mgr - Bug #41829: ceph df reports incorrect pool usageNew

Actions
Related to mgr - Bug #40203: ceph df shows incorrect usageNew06/07/2019

Actions
Has duplicate Dashboard - Bug #43384: mgr/dashboard: Pool size is calculated with data with different metricsDuplicate

Actions
Copied to Dashboard - Backport #46570: octopus: mgr/dashboard: fix usage calculation to match "ceph df" wayResolvedErnesto PuertaActions
Copied to Dashboard - Backport #47579: nautilus: mgr/dashboard: fix usage calculation to match "ceph df" wayResolvedErnesto PuertaActions
Actions #1

Updated by Lenz Grimmer almost 4 years ago

  • Translation missing: en.field_tag_list set to low-hanging-fruit, administration
Actions #2

Updated by Alfonso Martínez almost 4 years ago

  • Status changed from In Progress to New
  • Assignee changed from Ernesto Puerta to Alfonso Martínez
Actions #3

Updated by Kiefer Chang almost 4 years ago

To fully align with CLI output, we might need to also consider omap usage and rounding method differences:
CLI implementation: https://github.com/ceph/ceph/blob/e6fccf6f381e37b38956f3efc33f5c4938a663e1/src/mon/PGMap.cc#L914-L926

Actions #4

Updated by Ernesto Puerta almost 4 years ago

Just explored the ceph-mgr API with this and found:

# ceph mgr cli --help

 Monitor commands: 
 =================
...
mgr cli get <data_name>                                                Called by the plugin to fetch named cluster-wide objects from ceph-mgr.
                                                                        :param str data_name: Valid things to fetch are osd_crush_map_text, 
                                                                        osd_map, osd_map_tree, osd_map_crush, config, mon_map, fs_map, osd_
                                                                        metadata, pg_summary, io_rate, pg_dump, df, osd_stats, health, mon_
                                                                        status, devices, device <devid>, pg_stats, pool_stats, pg_ready, osd_
                                                                        ping_times. Note: All these structures have their own JSON 
                                                                        representations: experiment or look at the C++ ``dump()`` methods to 
                                                                        learn about the

It provides an in-memory cached get('df') command, whose output is:

{
    "pools": [
        {
            "id": 1,
            "name": "device_health_metrics",
            "stats": {
                "bytes_used": 0,
                "compress_bytes_used": 0,
                "compress_under_bytes": 0,
                "data_bytes_used": 0,
                "dirty": 0,
                "kb_used": 0,
                "max_avail": 105215721472,
                "objects": 0,
                "omap_bytes_used": 0,
                "percent_used": 0.0,
                "quota_bytes": 0,
                "quota_objects": 0,
                "rd": 0,
                "rd_bytes": 0,
                "stored": 0,
                "stored_data": 0,
                "stored_omap": 0,
                "stored_raw": 0,
                "wr": 0,
                "wr_bytes": 0
            }
        },
        {
            "id": 2,
            "name": "rbd-pool",
            "stats": {
                "bytes_used": 0,
                "compress_bytes_used": 0,
                "compress_under_bytes": 0,
                "data_bytes_used": 0,
                "dirty": 0,
                "kb_used": 0,
                "max_avail": 105215721472,
                "objects": 0,
                "omap_bytes_used": 0,
                "percent_used": 0.0,
                "quota_bytes": 0,
                "quota_objects": 0,
                "rd": 0,
                "rd_bytes": 0,
                "stored": 0,
                "stored_data": 0,
                "stored_omap": 0,
                "stored_raw": 0,
                "wr": 0,
                "wr_bytes": 0
            }
        }
    ],
...
}

'percent_used' is the exact calculation we're looking for.

Actions #5

Updated by Ernesto Puerta almost 4 years ago

  • Assignee changed from Alfonso Martínez to Ernesto Puerta
Actions #6

Updated by Ernesto Puerta almost 4 years ago

  • Status changed from New to Fix Under Review
Actions #7

Updated by Ernesto Puerta almost 4 years ago

  • Pull request ID set to 35768
Actions #8

Updated by Kefu Chai almost 4 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #10

Updated by Ernesto Puerta almost 4 years ago

  • Copied to Backport #46570: octopus: mgr/dashboard: fix usage calculation to match "ceph df" way added
Actions #11

Updated by Ernesto Puerta almost 4 years ago

  • Backport changed from nautilus, octopus to octopus
Actions #12

Updated by Nathan Cutler over 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions #13

Updated by Stephan Müller over 3 years ago

  • Related to Bug #41829: ceph df reports incorrect pool usage added
Actions #14

Updated by Stephan Müller over 3 years ago

  • Related to Bug #40203: ceph df shows incorrect usage added
Actions #15

Updated by Ernesto Puerta over 3 years ago

  • Status changed from Resolved to Pending Backport
  • Backport changed from octopus to octopus, nautilus
Actions #16

Updated by Ernesto Puerta over 3 years ago

  • Copied to Backport #47579: nautilus: mgr/dashboard: fix usage calculation to match "ceph df" way added
Actions #17

Updated by Nathan Cutler over 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions #18

Updated by Lenz Grimmer over 3 years ago

  • Has duplicate Bug #43384: mgr/dashboard: Pool size is calculated with data with different metrics added
Actions #19

Updated by Ernesto Puerta about 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 137 to Component - Pools
Actions

Also available in: Atom PDF