Project

General

Profile

Actions

Bug #62165

closed

[pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs

Added by Kamoltat (Junior) Sirivadhna 9 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
High
Category:
pg_autoscaler module
Target version:
-
% Done:

100%

Source:
Tags:
backport_processed
Backport:
reef, quincy, pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

We have observed a performance issue regarding the autoscaler while scale testing using reef. The cpu peaking to up to a core every minute


Files

image.png (182 KB) image.png Kamoltat (Junior) Sirivadhna, 07/25/2023 02:57 PM

Related issues 3 (0 open3 closed)

Copied to mgr - Backport #62885: reef: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGsResolvedKamoltat (Junior) SirivadhnaActions
Copied to mgr - Backport #62886: quincy: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGsResolvedKamoltat (Junior) SirivadhnaActions
Copied to mgr - Backport #62887: pacific: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGsRejectedKamoltat (Junior) SirivadhnaActions
Actions #1

Updated by Kamoltat (Junior) Sirivadhna 9 months ago

  • Pull request ID set to 52633

The problem is due to how we have been extravagantly using osdmap get calls in the autoscaler module which becomes very costly when there is a lot of PGs.

Also attaching PR id.

Actions #2

Updated by Kamoltat (Junior) Sirivadhna 9 months ago

  • Subject changed from [pg-autoscaler] Peformance issue with 32768 taking up 1 whole cpu core to [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs
Actions #3

Updated by Neha Ojha 9 months ago

  • Project changed from Ceph to mgr
  • Category set to pg_autoscaler module
  • Status changed from New to Fix Under Review
Actions #4

Updated by Kamoltat (Junior) Sirivadhna 8 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Backport set to reef, quincy, pacific
Actions #5

Updated by Backport Bot 8 months ago

  • Copied to Backport #62885: reef: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs added
Actions #6

Updated by Backport Bot 8 months ago

  • Copied to Backport #62886: quincy: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs added
Actions #7

Updated by Backport Bot 8 months ago

  • Copied to Backport #62887: pacific: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs added
Actions #8

Updated by Backport Bot 8 months ago

  • Tags set to backport_processed
Actions #10

Updated by Konstantin Shalygin about 1 month ago

  • Status changed from Pending Backport to Resolved
  • % Done changed from 0 to 100
Actions

Also available in: Atom PDF