Project

General

Profile

Actions

Bug #53475

closed

progress: dump of pg_stats hold ClusterState::lock for long periods on large cluster

Added by Sage Weil over 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:

100%

Source:
Tags:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The progress module does get('pg_stats'), which does

  } else if (what == "pg_stats") {
    return cluster_state.with_pgmap([&](const PGMap &pg_map) {
      with_gil_t with_gil{no_gil};
      pg_map.dump_pg_stats(&f, false);
      return f.get();
    });

This hold ClusterState::lock for a looong time while PyFormatter generates a zillion objects for a full dump of PGMap over all PGs. This can take a long time (multiple seconds) on large clusters, which ends up blocking the ms_dispatch thread and/or reports that are coming in from osds etc.


Related issues 1 (0 open1 closed)

Copied to mgr - Backport #53634: pacific: progress: dump of pg_stats hold ClusterState::lock for long periods on large clusterResolvedCory SnyderActions
Actions #1

Updated by Sage Weil over 2 years ago

  • Status changed from New to Need More Info
  • Backport set to pacific
  • Pull request ID set to 44208
Actions #2

Updated by Sage Weil over 2 years ago

  • Status changed from Need More Info to Pending Backport
Actions #3

Updated by Backport Bot over 2 years ago

  • Copied to Backport #53634: pacific: progress: dump of pg_stats hold ClusterState::lock for long periods on large cluster added
Actions #4

Updated by Backport Bot over 1 year ago

  • Tags set to backport_processed
Actions #5

Updated by Konstantin Shalygin over 1 year ago

  • Status changed from Pending Backport to Resolved
  • % Done changed from 0 to 100
  • Tags deleted (backport_processed)
Actions

Also available in: Atom PDF