Project

General

Profile

Actions

Bug #52673

closed

rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation

Added by Vikhyat Umrao over 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
% Done:

100%

Source:
Tags:
Backport:
pacific, quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

- Currently, the RGW init function takes care of the creation/configuration for RGW omap usage pools - metadata pool.
- We need to clean up this code because for now RGW init function only sets pg-autoscaler bias to 4 to only `.meta` pools, not for .log, .index, or any other metadata pools. This is also reproducible in the pacific release.

- Suggestion is to move this code/logic out of RGW and put it in Ceph Orchestrator(ceph-adm), Please check - https://tracker.ceph.com/issues/52674

- Source File - src/rgw/rgw_tools.cc

int rgw_init_ioctx(const DoutPrefixProvider *dpp,
                   librados::Rados *rados, const rgw_pool& pool,
                   librados::IoCtx& ioctx, bool create,
                   bool mostly_omap)
{
  int r = rados->ioctx_create(pool.name.c_str(), ioctx);
  if (r == -ENOENT && create) {
    r = rados->pool_create(pool.name.c_str());
    if (r == -ERANGE) {
      ldpp_dout(dpp, 0)
        << __func__
        << " ERROR: librados::Rados::pool_create returned " << cpp_strerror(-r)
        << " (this can be due to a pool or placement group misconfiguration, e.g." 
        << " pg_num < pgp_num or mon_max_pg_per_osd exceeded)" 
        << dendl;
    }
    if (r < 0 && r != -EEXIST) {
      return r;
    }

    r = rados->ioctx_create(pool.name.c_str(), ioctx);
    if (r < 0) {
      return r;
    }

    r = ioctx.application_enable(pg_pool_t::APPLICATION_NAME_RGW, false);
    if (r < 0 && r != -EOPNOTSUPP) {
      return r;
    }

    if (mostly_omap) {
      // set pg_autoscale_bias
      bufferlist inbl;
      float bias = g_conf().get_val<double>("rgw_rados_pool_autoscale_bias");
      int r = rados->mon_command(
        "{\"prefix\": \"osd pool set\", \"pool\": \"" +
        pool.name + "\", \"var\": \"pg_autoscale_bias\", \"val\": \"" +
        stringify(bias) + "\"}",
        inbl, NULL, NULL);
      if (r < 0) {
        ldpp_dout(dpp, 10) << __func__ << " warning: failed to set pg_autoscale_bias on " 
                 << pool.name << dendl;
      }
      // set pg_num_min
      int min = g_conf().get_val<uint64_t>("rgw_rados_pool_pg_num_min");
      r = rados->mon_command(
        "{\"prefix\": \"osd pool set\", \"pool\": \"" +
        pool.name + "\", \"var\": \"pg_num_min\", \"val\": \"" +
        stringify(min) + "\"}",
        inbl, NULL, NULL);
      if (r < 0) {
        ldpp_dout(dpp, 10) << __func__ << " warning: failed to set pg_num_min on " 
                 << pool.name << dendl;
      }
      // set recovery_priority
      int p = g_conf().get_val<uint64_t>("rgw_rados_pool_recovery_priority");
      r = rados->mon_command(
        "{\"prefix\": \"osd pool set\", \"pool\": \"" +
        pool.name + "\", \"var\": \"recovery_priority\": \"" +
        stringify(p) + "\"}",
        inbl, NULL, NULL);
      if (r < 0) {
        ldpp_dout(dpp, 10) << __func__ << " warning: failed to set recovery_priority on " 
                 << pool.name << dendl;
      }
    }
  } else if (r < 0) {
    return r;
  }
  if (!pool.ns.empty()) {
    ioctx.set_namespace(pool.ns);
  }
  return 0;
}


Related issues 3 (1 open2 closed)

Related to Orchestrator - Feature #52674: Ceph orch/ceph-adm should create/configure RGW poolsNew

Actions
Copied to rgw - Backport #54277: quincy: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creationResolvedVikhyat UmraoActions
Copied to rgw - Backport #54278: pacific: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creationResolvedVikhyat UmraoActions
Actions #1

Updated by Vikhyat Umrao over 2 years ago

  • Description updated (diff)
Actions #2

Updated by Sebastian Wagner over 2 years ago

  • Related to Feature #52674: Ceph orch/ceph-adm should create/configure RGW pools added
Actions #3

Updated by Casey Bodley about 2 years ago

opened https://github.com/ceph/ceph/pull/44983 which removes rgw_rados_pool_pg_num_min

Actions #4

Updated by Vikhyat Umrao about 2 years ago

  • Status changed from New to In Progress
  • Assignee set to Casey Bodley
Actions #5

Updated by Vikhyat Umrao about 2 years ago

  • Subject changed from RGW pool creation code cleanup with pg autoscaler for mostly_omap use case to rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
Actions #6

Updated by Vikhyat Umrao about 2 years ago

  • Backport set to pacific, quincy
Actions #8

Updated by Neha Ojha about 2 years ago

  • Status changed from In Progress to Pending Backport
Actions #9

Updated by Backport Bot about 2 years ago

  • Copied to Backport #54277: quincy: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation added
Actions #10

Updated by Backport Bot about 2 years ago

  • Copied to Backport #54278: pacific: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation added
Actions #13

Updated by Backport Bot over 1 year ago

  • Tags set to backport_processed
Actions #14

Updated by Konstantin Shalygin over 1 year ago

  • Tracker changed from Cleanup to Bug
  • Status changed from Pending Backport to Resolved
  • % Done changed from 0 to 100
  • Tags deleted (backport_processed)
  • Regression set to No
  • Severity set to 3 - minor
Actions

Also available in: Atom PDF