Project

General

Profile

Bug #45185

mgr/dashboard: fix usage calculation to match "ceph df" way

Added by Ernesto Puerta over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Category:
Component - Pools
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
octopus, nautilus
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Currently Ceph-dashboard Pool list Usage % is calculated as:

usage = raw_used / (raw_used + max_avail)

While ceph df is:

usage = data_used / (data_used + max_avail)

Related issues

Related to mgr - Bug #41829: ceph df reports incorrect pool usage New
Related to mgr - Bug #40203: ceph df shows incorrect usage New 06/07/2019
Duplicated by Dashboard - Bug #43384: mgr/dashboard: Pool size is calculated with data with different metrics Duplicate
Copied to Dashboard - Backport #46570: octopus: mgr/dashboard: fix usage calculation to match "ceph df" way Resolved
Copied to Dashboard - Backport #47579: nautilus: mgr/dashboard: fix usage calculation to match "ceph df" way Resolved

History

#1 Updated by Lenz Grimmer over 3 years ago

  • Tags set to low-hanging-fruit, administration

#2 Updated by Alfonso Martínez over 3 years ago

  • Status changed from In Progress to New
  • Assignee changed from Ernesto Puerta to Alfonso Martínez

#3 Updated by Kiefer Chang over 3 years ago

To fully align with CLI output, we might need to also consider omap usage and rounding method differences:
CLI implementation: https://github.com/ceph/ceph/blob/e6fccf6f381e37b38956f3efc33f5c4938a663e1/src/mon/PGMap.cc#L914-L926

#4 Updated by Ernesto Puerta over 3 years ago

Just explored the ceph-mgr API with this and found:

# ceph mgr cli --help

 Monitor commands: 
 =================
...
mgr cli get <data_name>                                                Called by the plugin to fetch named cluster-wide objects from ceph-mgr.
                                                                        :param str data_name: Valid things to fetch are osd_crush_map_text, 
                                                                        osd_map, osd_map_tree, osd_map_crush, config, mon_map, fs_map, osd_
                                                                        metadata, pg_summary, io_rate, pg_dump, df, osd_stats, health, mon_
                                                                        status, devices, device <devid>, pg_stats, pool_stats, pg_ready, osd_
                                                                        ping_times. Note: All these structures have their own JSON 
                                                                        representations: experiment or look at the C++ ``dump()`` methods to 
                                                                        learn about the

It provides an in-memory cached get('df') command, whose output is:

{
    "pools": [
        {
            "id": 1,
            "name": "device_health_metrics",
            "stats": {
                "bytes_used": 0,
                "compress_bytes_used": 0,
                "compress_under_bytes": 0,
                "data_bytes_used": 0,
                "dirty": 0,
                "kb_used": 0,
                "max_avail": 105215721472,
                "objects": 0,
                "omap_bytes_used": 0,
                "percent_used": 0.0,
                "quota_bytes": 0,
                "quota_objects": 0,
                "rd": 0,
                "rd_bytes": 0,
                "stored": 0,
                "stored_data": 0,
                "stored_omap": 0,
                "stored_raw": 0,
                "wr": 0,
                "wr_bytes": 0
            }
        },
        {
            "id": 2,
            "name": "rbd-pool",
            "stats": {
                "bytes_used": 0,
                "compress_bytes_used": 0,
                "compress_under_bytes": 0,
                "data_bytes_used": 0,
                "dirty": 0,
                "kb_used": 0,
                "max_avail": 105215721472,
                "objects": 0,
                "omap_bytes_used": 0,
                "percent_used": 0.0,
                "quota_bytes": 0,
                "quota_objects": 0,
                "rd": 0,
                "rd_bytes": 0,
                "stored": 0,
                "stored_data": 0,
                "stored_omap": 0,
                "stored_raw": 0,
                "wr": 0,
                "wr_bytes": 0
            }
        }
    ],
...
}

'percent_used' is the exact calculation we're looking for.

#5 Updated by Ernesto Puerta over 3 years ago

  • Assignee changed from Alfonso Martínez to Ernesto Puerta

#6 Updated by Ernesto Puerta over 3 years ago

  • Status changed from New to Fix Under Review

#7 Updated by Ernesto Puerta over 3 years ago

  • Pull request ID set to 35768

#8 Updated by Kefu Chai over 3 years ago

  • Status changed from Fix Under Review to Pending Backport

#10 Updated by Ernesto Puerta over 3 years ago

  • Copied to Backport #46570: octopus: mgr/dashboard: fix usage calculation to match "ceph df" way added

#11 Updated by Ernesto Puerta over 3 years ago

  • Backport changed from nautilus, octopus to octopus

#12 Updated by Nathan Cutler over 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

#13 Updated by Stephan Müller over 3 years ago

  • Related to Bug #41829: ceph df reports incorrect pool usage added

#14 Updated by Stephan Müller over 3 years ago

  • Related to Bug #40203: ceph df shows incorrect usage added

#15 Updated by Ernesto Puerta about 3 years ago

  • Status changed from Resolved to Pending Backport
  • Backport changed from octopus to octopus, nautilus

#16 Updated by Ernesto Puerta about 3 years ago

  • Copied to Backport #47579: nautilus: mgr/dashboard: fix usage calculation to match "ceph df" way added

#17 Updated by Nathan Cutler about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

#18 Updated by Lenz Grimmer about 3 years ago

  • Duplicated by Bug #43384: mgr/dashboard: Pool size is calculated with data with different metrics added

#19 Updated by Ernesto Puerta over 2 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 137 to Component - Pools

Also available in: Atom PDF