Ceph » CephFS

Hi Victoria,

I am not very familiar with the osd configs but as per code if 'osd_pool_default_pg_autoscale_mode' is 'on', the 'pg_num' is set to 1. Please see the code below

int OSDMonitor::prepare_new_pool(....)
{
...
...
  if (pg_num == 0) {
    auto pg_num_from_mode =
      [pg_num=g_conf().get_val<uint64_t>("osd_pool_default_pg_num")]
      (const string& mode) {
      return mode == "on" ? 1 : pg_num;
    };
    pg_num = pg_num_from_mode(
      pg_autoscale_mode.empty() ?
      g_conf().get_val<string>("osd_pool_default_pg_autoscale_mode") :
      pg_autoscale_mode);
  }
...
...

According to your config 'osd_pool_default_pgp_num = 8' which is greater than the 'pg_num' set (i.e 8 > 1). Setting 'osd_pool_default_pgp_num = 1' should fix the issue.
Please see the code below.

int OSDMonitor::prepare_new_pool(....)
{
...
...
  if (pgp_num > pg_num) {
    *ss << "'pgp_num' must be greater than 0 and lower or equal than 'pg_num'" 
        << ", which in this case is " << pg_num;
    return -ERANGE;
  }
... 
</prev>

Tested the deployment with "osd_pool_default_pg_autoscale_mode = off" in bootstrap_conf and seems to fix the issue. Has this option been changed to on between Pacific and Quincy? Is setting this value to off advised or should I try a different config?

Current config and cluster status is

stack@devstack-quincy:~$ cat bootstrap_ceph.conf
[global]
log to file = true
osd crush chooseleaf type = 0
osd_pool_default_pg_num = 8
osd_pool_default_pgp_num = 8
osd_pool_default_size = 1
osd_pool_default_pg_autoscale_mode = off
[mon]
mon_warn_on_pool_no_redundancy = False
[osd]
osd_memory_target_autotune = true
osd_numa_auto_affinity = true
[mgr]
mgr/cephadm/autotune_memory_target_ratio = 0.2

stack@devstack-quincy:~/devstack$ sudo cephadm shell
Inferring fsid ddc1d35b-4f75-4956-a55d-6a9d7e77f1d1
Inferring config /var/lib/ceph/ddc1d35b-4f75-4956-a55d-6a9d7e77f1d1/mon.devstack-quincy/config
Using ceph image with id 'e5af760fa1c1' and tag 'v17.2.1' created on 2022-06-23 19:49:45 +0000 UTC
quay.io/ceph/ceph@sha256:d3f3e1b59a304a280a3a81641ca730982da141dad41e942631e4c5d88711a66b
root@devstack-quincy:/# ceph -s
cluster:
id: ddc1d35b-4f75-4956-a55d-6a9d7e77f1d1
health: HEALTH_OK

services:
    mon: 1 daemons, quorum devstack-quincy (age 15m)
    mgr: devstack-quincy.repaot(active, since 14m), standbys: devstack-quincy.marjpi
    mds: 1/1 daemons up, 1 standby
    osd: 1 osds: 1 up (since 13m), 1 in (since 14m)

data:
    volumes: 1/1 healthy
    pools:   4 pools, 25 pgs
    objects: 27 objects, 451 KiB
    usage:   21 MiB used, 30 GiB / 30 GiB avail
    pgs:     25 active+clean

io:
    client:   85 B/s rd, 0 op/s rd, 0 op/s wr

Once we confirm that removing osd_pool_default_pgp_num and osd_pool_default_pg_num in devstack-plugin-ceph works, we can close this tracker.
https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/851521