Project

General

Profile

Actions
Copy link

Bug #63912

open

mon cannot trim OSDMaps - due to creating_pgs.pgs is not empty

Added by jianwei zhang 4 months ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
01/02/2024
Affected Versions:
Ceph - v19.0.0
ceph-qa-suite:
rados
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

# date;ceph -s
Tue Dec 19 10:18:18 CST 2023
  cluster:
    id:     621820e9-f1e5-4bf5-a70d-fa98001f21fe
    health: HEALTH_WARN
            mons a,b,c are using a lot of disk space

  services:
    mon: 3 daemons, quorum a,b,c (age 3w)
    mgr: x(active, since 3M), standbys: y, z
    osd: 1920 osds: 1920 up (since 19h), 1920 in (since 3M)

  data:
    pools:   1920 pools, 245760 pgs
    objects: 9.42M objects, 1.1 TiB
    usage:   909 TiB used, 27 PiB / 28 PiB avail
    pgs:     245760 active+clean

# tail -f ceph-mon.a.log  | grep trim
2023-12-19T11:03:02.255+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0 
2023-12-19T11:03:17.282+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0 
2023-12-19T11:04:22.432+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0
Actions
Copy link
 #1

Updated by jianwei zhang 4 months ago

pool have the pg_pool_t::FLAG_CREATING flag, use the command can see:
ceph osd pool ls detail | grep creating

Actions
Copy link
 #2

Updated by jianwei zhang 4 months ago

This phenomenon is not accidental,
We have many clusters that have this problem

Actions
Copy link
 #4

Updated by jianwei zhang 4 months ago

I don't know how the MOSDPGCreated message sent to mon was lost,

Therefore, based on the source code of https://github.com/ceph/ceph/pull/55041 PR,
I forced not to send MOSDPGCreated to MON in the OSD source code.
The temporary code modification is as follows:

[root@zjw-cmain-dev build]# git diff
diff --git a/src/common/options.cc b/src/common/options.cc
index 58a4ab7b8..a74ad7cab 100644
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -3523,7 +3523,7 @@ std::vector<Option> get_global_options() {

     Option("osd_send_pg_created_interval", Option::TYPE_INT, Option::LEVEL_ADVANCED)
     .set_default(6_hr)
-    .set_min(1_hr)
+    .set_min(60)
     .set_description("The desired interval seconds between send MOSDPGCreated of OSD::tick."),

     Option("osd_max_snap_prune_intervals_per_epoch", Option::TYPE_UINT, Option::LEVEL_DEV)
diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index afb1f1d4b..fa6140f0a 100644
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -1225,22 +1225,27 @@ void OSDService::send_pg_temp()
 void OSDService::send_pg_created(pg_t pgid)
 {
   std::lock_guard l(pg_created_lock);
-  dout(20) << __func__ << dendl;
+  dout(0) << __func__ << dendl;
   auto o = get_osdmap();
   if (o->require_osd_release >= ceph_release_t::luminous) {
     pg_created.insert(pgid);
-    monc->send_mon_message(new MOSDPGCreated(pgid));
+    dout(0) << __func__ << ": pg_created_insert1 " << pgid << dendl;
+    //monc->send_mon_message(new MOSDPGCreated(pgid));
   }
 }

 void OSDService::send_pg_created()
 {
   std::lock_guard l(pg_created_lock);
-  dout(20) << __func__ << dendl;
+  dout(0) << __func__ << dendl;
   auto o = get_osdmap();
   if (o->require_osd_release >= ceph_release_t::luminous) {
     for (auto pgid : pg_created) {
-      monc->send_mon_message(new MOSDPGCreated(pgid));
+      dout(0) << __func__ << ": pg_created_insert2 " << pgid << dendl;
+      auto send_pg_created_min_interval = cct->_conf.get_val<int64_t>("osd_send_pg_created_interval");
+      if (send_pg_created_min_interval <= 300) {
+        monc->send_mon_message(new MOSDPGCreated(pgid));
+      }
     }
   }
 }
@@ -1248,16 +1253,16 @@ void OSDService::send_pg_created()
 void OSDService::prune_pg_created()
 {
   std::lock_guard l(pg_created_lock);
-  dout(20) << __func__ << dendl;
+  dout(0) << __func__ << dendl;
   auto o = get_osdmap();
   auto i = pg_created.begin();
   while (i != pg_created.end()) {
     auto p = o->get_pg_pool(i->pool());
     if (!p || !p->has_flag(pg_pool_t::FLAG_CREATING)) {
-      dout(20) << __func__ << " pruning " << *i << dendl;
+      dout(0) << __func__ << " pruning " << *i << dendl;
       i = pg_created.erase(i);
     } else {
-      dout(20) << __func__ << " keeping " << *i << dendl;
+      dout(0) << __func__ << " keeping " << *i << dendl;
       ++i;
     }
   }

1. create a vstart cluster

#!/bin/sh
set -x

rm -rf dev ceph.conf out

MON=3 MGR=3 OSD=3 FS=0 MDS=0 RGW=0 ../src/vstart.sh -n -l -X -b --msgr2

2. ceph.conf

vi ceph.conf
[osd]
osd_send_pg_created_interval = 21600

init-ceph restart osd

3. create test-pool

ceph osd pool create test-pool-0 128 128 replicated replicated_rule 0 1 off

4. ceph -s

# ceph -s
  cluster:
    id:     c6422188-5bff-48b7-bbec-c862cc6acb27
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum a,b,c (age 100m)
    mgr: x(active, since 100m), y, z
    osd: 3 osds: 3 up (since 58m), 3 in (since 106m)

  data:
    pools:   1 pools, 128 pgs
    objects: 0 objects, 0 B
    usage:   372 KiB used, 9.1 TiB / 9.1 TiB avail
    pgs:     128 active+clean

5. pool creating status remains

# ceph osd pool ls detail |grep creating
pool 1 'test-pool-0' replicated size 3 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 29 flags hashpspool,creating stripe_width 0 application rgw

6. Allow OSD to send MOSDPGCreated to MON via OSD::tick

vi ceph.conf
[osd]
osd_send_pg_created_interval = 300

init-ceph restart osd

# ceph daemon osd.0 config show |grep osd_send_pg_created_interval
    "osd_send_pg_created_interval": "300"

7. ceph osd pool ls detail 

# for i in {1..10000}; do date; ceph osd pool ls detail | grep  creating; sleep 10;done
Wed Jan  3 14:30:26 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw

Wed Jan  3 14:30:36 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw

Wed Jan  3 14:30:47 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw

// here, pool creating status is not show

Wed Jan  3 14:30:57 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 44 flags hashpspool stripe_width 0 application rgw

Wed Jan  3 14:31:07 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 44 flags hashpspool stripe_width 0 application rgw

8. osd.0.log
1. osd.0 calls OSDService::send_pg_created through OSD::tick to send X to mon
2. After osd.0 receives new OSDMap, pool not has pg_pool_t::FLAG_CREATING, erase the pgid of pool from pg_created
3. ceph osd pool ls detail , show that test-pool-0's creating status is not exists.

tail -f out/osd.0.log | grep -e 'ceph version'  -e prune_pg_created -e osd_pg_created -e send_pg_created
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created: pg_created_insert1 1.46
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created: pg_created_insert1 1.3c
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created
2024-01-03T14:25:52.550+0800 7f6cb4bc7700  0 osd.0 43 send_pg_created: pg_created_insert1 1.19
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.0
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7
2024-01-03T14:30:51.262+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.8
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.9
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.10
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.11
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.12
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.13
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.14
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.15
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.16
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.17
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.18
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.19
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.1f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.20
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.21
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.22
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.23
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.24
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.25
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.26
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.27
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.28
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.29
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.2f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.30
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.31
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.32
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.33
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.34
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.35
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.36
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.37
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.38
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.39
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.3f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.40
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.41
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.42
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.43
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.44
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.45
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.46
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.47
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.48
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.49
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.4f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.50
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.51
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.52
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.53
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.54
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.55
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.56
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.57
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.58
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.59
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.5f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.60
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.61
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.62
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.63
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.64
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.65
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.66
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.67
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.68
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.69
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.6f
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.70
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.71
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.72
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.73
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.74
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.75
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.76
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.77
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.78
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.79
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7a
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7b
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7c
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7d
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7e
2024-01-03T14:30:51.263+0800 7f6ccce06700  0 osd.0 43 send_pg_created: pg_created_insert2 1.7f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.0
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.8
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.9
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.10
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.11
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.12
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.13
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.14
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.15
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.16
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.17
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.18
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.19
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.1f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.20
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.21
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.22
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.23
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.24
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.25
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.26
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.27
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.28
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.29
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.2f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.30
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.31
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.32
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.33
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.34
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.35
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.36
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.37
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.38
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.39
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.3f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.40
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.41
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.42
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.43
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.44
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.45
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.46
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.47
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.48
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.49
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.4f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.50
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.51
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.52
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.53
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.54
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.55
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.56
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.57
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.58
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.59
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.5f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.60
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.61
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.62
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.63
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.64
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.65
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.66
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.67
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.68
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.69
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.6f
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.70
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.71
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.72
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.73
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.74
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.75
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.76
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.77
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.78
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.79
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7a
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7b
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7c
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7d
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7e
2024-01-03T14:30:51.760+0800 7f6cc0bdf700  0 osd.0 44 prune_pg_created pruning 1.7f
2024-01-03T14:35:52.857+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T14:40:54.106+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T14:45:55.882+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T14:50:56.936+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T14:55:58.137+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T15:00:59.403+0800 7f6ccce06700  0 osd.0 44 send_pg_created
2024-01-03T15:06:01.175+0800 7f6ccce06700  0 osd.0 44 send_pg_created

Actions
Copy link
 #5

Updated by jianwei zhang 4 months ago

osd force-create-pg <PGID> reproduce:

# ceph -s
  cluster:
    id:     db340f9a-d843-47b8-8280-6c724a6fa74e
    health: HEALTH_OK

  services:
    mon: 1 daemons, quorum a (age 77s)
    mgr: x(active, since 69s)
    osd: 1 osds: 1 up (since 63s), 1 in (since 5m)

  data:
    pools:   1 pools, 128 pgs
    objects: 0 objects, 0 B
    usage:   228 KiB used, 9.1 TiB / 9.1 TiB avail
    pgs:     128 active+clean

# ceph osd pool ls detail
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 application rgw


vi ceph.conf
[osd.0]
debug_osd = 20
debug_ms = 1

[mon.a]
debug_mon = 20
debug_ms = 1

init-ceph restart mon.a osd.0


# ceph osd pool ls detail
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 application rgw


ceph osd force-create-pg 1.0 --yes-i-really-mean-it
pg 1.0 now creating, ok


ceph osd pool ls detail
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 40 flags hashpspool,creating stripe_width 0 application rgw

osd.0.log

[root@zjw-cmain-dev build]# grep 7f71be537700 out/osd.0.log
2024-01-04T14:06:37.251+0800 7f71be537700  1 -- v2:127.0.0.1:6804/96647 <== mon.0 v2:127.0.0.1:40328/0 15 ==== pg_create2(e59 {1.0=58,2024-01-04T14:06:36.568113+0800}) v3 ==== 208+0+0 (crc 0 0 0) 0x55ee8ecd9a40 con 0x55ee8fb5bc00
2024-01-04T14:06:37.251+0800 7f71be537700  7 osd.0 59 handle_fast_pg_create pg_create2(e59 {1.0=58,2024-01-04T14:06:36.568113+0800}) v3 from mon.0
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 59 handle_fast_pg_create 1.0 e58@2024-01-04T14:06:36.568113+0800 history ec=58/58 lis/c=0/0 les/c/f=0/0/0 sis=58 pi ([0,0] all_participants= intervals=)
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 59 handle_fast_pg_create 1.0 e58@2024-01-04T14:06:36.568113+0800 (has history or past_intervals)
2024-01-04T14:06:37.251+0800 7f71be537700 15 osd.0 59 enqueue_peering_evt 1.0 epoch_sent: 59 epoch_requested: 59 NullEvt +create_info
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59)


2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {}
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59) queued
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process 1.0 to_process <OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59)> waiting <> waiting_peering {}
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59) pg 0x55ee8ffd3000
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 dequeue_peering_evt enter evt = epoch_sent: 59 epoch_requested: 59 NullEvt +create_info pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 advance_pg pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 advance_pg return true osd_epoch<=pg->get_osdmap_epoch() 59 pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 dequeue_peering_evt advance_pg=true evt = epoch_sent: 59 epoch_requested: 59 NullEvt +create_info pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean] do_peering_event: epoch_sent: 59 epoch_requested: 59 NullEvt +create_info
2024-01-04T14:06:37.255+0800 7f719e26b700 10 log is not dirty
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process empty q, waiting


# grep -nr send_pg_created out/osd.0.log
# //nothing
Actions
Copy link
 #6

Updated by jianwei zhang 4 months ago

In order to identify the creation triggered by ceph osd force-create-pg 1.0,

  1. A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55239)
  2. Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55041)
Actions
Copy link
 #7

Updated by jianwei zhang 4 months ago

jianwei zhang wrote:

In order to identify the creation triggered by ceph osd force-create-pg 1.0,

  1. A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55239)
  2. Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55041)
correct:
  1. Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55239)
  2. A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55041)
Actions
Copy link

Also available in: Atom PDF