Project

General

Profile

Actions

Bug #49433

closed

mgr/dashboard: Grafana Error: matching labels must be unique on one side

Added by Gunther Heinrich about 3 years ago. Updated about 3 years ago.

Status:
Duplicate
Priority:
Normal
Category:
General
Target version:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

When opening the grafana performance details tab for any host on the page "Hosts", the displays for the "Disk IOPS" and the "Throughput by Disk" are emtpy. When clicking on the red exclamation mark in the upper left corner of one of the displays, a JSON error is displayed.
Here is an example of the error message:

error:Object

message:""found duplicate series for the match group {device="dm-3", instance="ceph_cluster"} on the right hand-side of the operation: 
[
{__name__="ceph_disk_occupation", ceph_daemon="osd.62", device="dm-3", exported_instance="iz-ceph-01-osd-04", instance="ceph_cluster", job="ceph"},  
{__name__="ceph_disk_occupation", ceph_daemon="osd.100", device="dm-3", exported_instance="iz-ceph-01-osd-01", instance="ceph_cluster", job="ceph"}
];
many-to-many matching not allowed: matching labels must be unique on one side"" 

refId:"B" 

status:422

statusText:"Unprocessable Entity" 

The system used is Ceph 15.2.7 on Ubuntu 20.04.2


Related issues 1 (0 open1 closed)

Is duplicate of Dashboard - Bug #47334: mgr/dashboard: many-to-many matching not allowed: matching labels must be unique on one sideResolvedPatrick Seidensal

Actions
Actions #1

Updated by Alfonso Martínez about 3 years ago

  • Target version changed from v17.0.0 to v15.2.9
Actions #2

Updated by Alfonso Martínez about 3 years ago

  • Assignee set to Patrick Seidensal
Actions #3

Updated by Patrick Seidensal about 3 years ago

  • Status changed from New to In Progress
Actions #4

Updated by Patrick Seidensal about 3 years ago

  • Status changed from In Progress to Duplicate
  • Assignee changed from Patrick Seidensal to Gunther Heinrich

Hi Gunther,

this looks very much like the Prometheus configuration does not contain `honor_labels` set to true for the scrape job named `ceph`, though exactly this template is being shipped in the Ceph Octopus since 15.2.8 , but not before. If you have Ceph newer or equal to 15.2.8 redeploying Prometheus should be sufficient to have a configuration created which works correctly.

Please note that only newly scraped metrics will be free of this problem. This means that, if you updated or set `honor_labels` to true yourself, and directly after that look at the Grafana dashboard for range of an hour, you will still see the issue. Only after five minutes have passed and only if you then look at a time range for the past five minutes, you'll notice that the problem has been resolved.

Actions #5

Updated by Patrick Seidensal about 3 years ago

  • Is duplicate of Bug #47334: mgr/dashboard: many-to-many matching not allowed: matching labels must be unique on one side added
Actions #6

Updated by Ernesto Puerta about 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 132 to General
Actions

Also available in: Atom PDF