Project

General

Profile

Bug #59505

mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format

Added by Anton Turetckii 11 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Normal
Category:
ceph-mgr
Target version:
-
% Done:

100%

Source:
Community (user)
Tags:
ceph-mgr backport_processed
Backport:
reef,quincy,pacific
Regression:
Yes
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Recreating Prometheus Metric object in a loop in the wrong way because as a result we may see messages like the below:
# HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
# TYPE ceph_pool_objects_repaired counter
ceph_pool_objects_repaired{poolid="5"} 0.0
# HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
# TYPE ceph_pool_objects_repaired counter
ceph_pool_objects_repaired{poolid="4"} 0.0
# HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
# TYPE ceph_pool_objects_repaired counter
ceph_pool_objects_repaired{poolid="3"} 0.0
# HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
# TYPE ceph_pool_objects_repaired counter
ceph_pool_objects_repaired{poolid="1"} 0.0
# HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
# TYPE ceph_pool_objects_repaired counter
ceph_pool_objects_repaired{poolid="2"} 0.0

The correct format must be like below:
  1. HELP ceph_pool_objects_repaired Number of objects repaired in a pool Count
  2. TYPE ceph_pool_objects_repaired counter

ceph_pool_objects_repaired{poolid="5"} 0.0
ceph_pool_objects_repaired{poolid="4"} 0.0
ceph_pool_objects_repaired{poolid="3"} 0.0
ceph_pool_objects_repaired{poolid="1"} 0.0
ceph_pool_objects_repaired{poolid="2"} 0.0

with only one header for each metric

I created the PR with the fix - https://github.com/ceph/ceph/pull/51090

Thanks


Related issues

Copied to mgr - Backport #61340: pacific: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format Resolved
Copied to mgr - Backport #61341: quincy: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format Resolved
Copied to mgr - Backport #61342: reef: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format Resolved

History

#1 Updated by Mykola Golub 11 months ago

  • Status changed from New to Fix Under Review
  • Backport set to quincy,pacific
  • Pull request ID set to 51090

#2 Updated by Mykola Golub 11 months ago

  • Backport changed from quincy,pacific to reef,quincy,pacific

#3 Updated by Ilya Dryomov 11 months ago

  • Regression changed from No to Yes

#4 Updated by Ilya Dryomov 11 months ago

  • Subject changed from mgr/prometheus plugin [bug]: expose daemon health metrics, repaired pgs metrics (implemented by PR#48204, PR#49519) to mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format
  • Target version deleted (v17.2.6)

This fixes a regression introduced in https://github.com/ceph/ceph/pull/47494 and https://github.com/ceph/ceph/pull/48843. While these metrics are new, they cause a general error:

error reading metrics for http://****:***/metrics: reading text format failed: text format parsing error in line 2010: second HELP line for metric name "ceph_pool_objects_repaired" 

#5 Updated by Konstantin Shalygin 11 months ago

  • Assignee set to Anton Turetckii
  • Source set to Community (user)

#6 Updated by Nizamudeen A 10 months ago

  • Status changed from Fix Under Review to Pending Backport

#7 Updated by Backport Bot 10 months ago

  • Copied to Backport #61340: pacific: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format added

#8 Updated by Backport Bot 10 months ago

  • Copied to Backport #61341: quincy: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format added

#9 Updated by Backport Bot 10 months ago

  • Copied to Backport #61342: reef: mgr/prometheus: fix pool_objects_repaired and daemon_health_metrics format added

#10 Updated by Backport Bot 10 months ago

  • Tags changed from ceph-mgr to ceph-mgr backport_processed

#11 Updated by Konstantin Shalygin 3 months ago

  • Status changed from Pending Backport to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF