Actions
Bug #63912
openmon cannot trim OSDMaps - due to creating_pgs.pgs is not empty
Added by jianwei zhang 5 months ago. Updated 4 months ago.
Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
01/02/2024
Affected Versions:
ceph-qa-suite:
rados
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
# date;ceph -s
Tue Dec 19 10:18:18 CST 2023
cluster:
id: 621820e9-f1e5-4bf5-a70d-fa98001f21fe
health: HEALTH_WARN
mons a,b,c are using a lot of disk space
services:
mon: 3 daemons, quorum a,b,c (age 3w)
mgr: x(active, since 3M), standbys: y, z
osd: 1920 osds: 1920 up (since 19h), 1920 in (since 3M)
data:
pools: 1920 pools, 245760 pgs
objects: 9.42M objects, 1.1 TiB
usage: 909 TiB used, 27 PiB / 28 PiB avail
pgs: 245760 active+clean
# tail -f ceph-mon.a.log | grep trim 2023-12-19T11:03:02.255+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0 2023-12-19T11:03:17.282+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0 2023-12-19T11:04:22.432+0800 7feea6b12700 10 mon.a@0(leader).osd e124692 get_trim_to pgs creating, trim_to = 0
Updated by jianwei zhang 5 months ago
pool have the pg_pool_t::FLAG_CREATING flag, use the command can see:
ceph osd pool ls detail | grep creating
Updated by jianwei zhang 5 months ago
This phenomenon is not accidental,
We have many clusters that have this problem
Updated by jianwei zhang 5 months ago
maybe a solution:
https://github.com/ceph/ceph/pull/55041
Updated by jianwei zhang 5 months ago
I don't know how the MOSDPGCreated message sent to mon was lost,
Therefore, based on the source code of https://github.com/ceph/ceph/pull/55041 PR,
I forced not to send MOSDPGCreated to MON in the OSD source code.
The temporary code modification is as follows:
[root@zjw-cmain-dev build]# git diff
diff --git a/src/common/options.cc b/src/common/options.cc
index 58a4ab7b8..a74ad7cab 100644
--- a/src/common/options.cc
+++ b/src/common/options.cc
@@ -3523,7 +3523,7 @@ std::vector<Option> get_global_options() {
Option("osd_send_pg_created_interval", Option::TYPE_INT, Option::LEVEL_ADVANCED)
.set_default(6_hr)
- .set_min(1_hr)
+ .set_min(60)
.set_description("The desired interval seconds between send MOSDPGCreated of OSD::tick."),
Option("osd_max_snap_prune_intervals_per_epoch", Option::TYPE_UINT, Option::LEVEL_DEV)
diff --git a/src/osd/OSD.cc b/src/osd/OSD.cc
index afb1f1d4b..fa6140f0a 100644
--- a/src/osd/OSD.cc
+++ b/src/osd/OSD.cc
@@ -1225,22 +1225,27 @@ void OSDService::send_pg_temp()
void OSDService::send_pg_created(pg_t pgid)
{
std::lock_guard l(pg_created_lock);
- dout(20) << __func__ << dendl;
+ dout(0) << __func__ << dendl;
auto o = get_osdmap();
if (o->require_osd_release >= ceph_release_t::luminous) {
pg_created.insert(pgid);
- monc->send_mon_message(new MOSDPGCreated(pgid));
+ dout(0) << __func__ << ": pg_created_insert1 " << pgid << dendl;
+ //monc->send_mon_message(new MOSDPGCreated(pgid));
}
}
void OSDService::send_pg_created()
{
std::lock_guard l(pg_created_lock);
- dout(20) << __func__ << dendl;
+ dout(0) << __func__ << dendl;
auto o = get_osdmap();
if (o->require_osd_release >= ceph_release_t::luminous) {
for (auto pgid : pg_created) {
- monc->send_mon_message(new MOSDPGCreated(pgid));
+ dout(0) << __func__ << ": pg_created_insert2 " << pgid << dendl;
+ auto send_pg_created_min_interval = cct->_conf.get_val<int64_t>("osd_send_pg_created_interval");
+ if (send_pg_created_min_interval <= 300) {
+ monc->send_mon_message(new MOSDPGCreated(pgid));
+ }
}
}
}
@@ -1248,16 +1253,16 @@ void OSDService::send_pg_created()
void OSDService::prune_pg_created()
{
std::lock_guard l(pg_created_lock);
- dout(20) << __func__ << dendl;
+ dout(0) << __func__ << dendl;
auto o = get_osdmap();
auto i = pg_created.begin();
while (i != pg_created.end()) {
auto p = o->get_pg_pool(i->pool());
if (!p || !p->has_flag(pg_pool_t::FLAG_CREATING)) {
- dout(20) << __func__ << " pruning " << *i << dendl;
+ dout(0) << __func__ << " pruning " << *i << dendl;
i = pg_created.erase(i);
} else {
- dout(20) << __func__ << " keeping " << *i << dendl;
+ dout(0) << __func__ << " keeping " << *i << dendl;
++i;
}
}
1. create a vstart cluster
#!/bin/sh set -x rm -rf dev ceph.conf out MON=3 MGR=3 OSD=3 FS=0 MDS=0 RGW=0 ../src/vstart.sh -n -l -X -b --msgr2
2. ceph.conf
vi ceph.conf [osd] osd_send_pg_created_interval = 21600 init-ceph restart osd
3. create test-pool
ceph osd pool create test-pool-0 128 128 replicated replicated_rule 0 1 off
4. ceph -s
# ceph -s
cluster:
id: c6422188-5bff-48b7-bbec-c862cc6acb27
health: HEALTH_OK
services:
mon: 3 daemons, quorum a,b,c (age 100m)
mgr: x(active, since 100m), y, z
osd: 3 osds: 3 up (since 58m), 3 in (since 106m)
data:
pools: 1 pools, 128 pgs
objects: 0 objects, 0 B
usage: 372 KiB used, 9.1 TiB / 9.1 TiB avail
pgs: 128 active+clean
5. pool creating status remains
# ceph osd pool ls detail |grep creating pool 1 'test-pool-0' replicated size 3 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 29 flags hashpspool,creating stripe_width 0 application rgw
6. Allow OSD to send MOSDPGCreated to MON via OSD::tick
vi ceph.conf
[osd]
osd_send_pg_created_interval = 300
init-ceph restart osd
# ceph daemon osd.0 config show |grep osd_send_pg_created_interval
"osd_send_pg_created_interval": "300"
7. ceph osd pool ls detail
# for i in {1..10000}; do date; ceph osd pool ls detail | grep creating; sleep 10;done
Wed Jan 3 14:30:26 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw
Wed Jan 3 14:30:36 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw
Wed Jan 3 14:30:47 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 35 flags hashpspool,creating stripe_width 0 application rgw
// here, pool creating status is not show
Wed Jan 3 14:30:57 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 44 flags hashpspool stripe_width 0 application rgw
Wed Jan 3 14:31:07 CST 2024
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 44 flags hashpspool stripe_width 0 application rgw
8. osd.0.log
1. osd.0 calls OSDService::send_pg_created through OSD::tick to send X to mon
2. After osd.0 receives new OSDMap, pool not has pg_pool_t::FLAG_CREATING, erase the pgid of pool from pg_created
3. ceph osd pool ls detail , show that test-pool-0's creating status is not exists.
tail -f out/osd.0.log | grep -e 'ceph version' -e prune_pg_created -e osd_pg_created -e send_pg_created 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created: pg_created_insert1 1.46 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created: pg_created_insert1 1.3c 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created 2024-01-03T14:25:52.550+0800 7f6cb4bc7700 0 osd.0 43 send_pg_created: pg_created_insert1 1.19 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.0 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7 2024-01-03T14:30:51.262+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.8 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.9 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.10 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.11 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.12 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.13 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.14 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.15 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.16 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.17 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.18 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.19 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.1f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.20 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.21 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.22 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.23 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.24 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.25 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.26 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.27 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.28 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.29 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.2f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.30 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.31 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.32 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.33 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.34 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.35 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.36 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.37 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.38 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.39 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.3f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.40 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.41 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.42 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.43 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.44 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.45 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.46 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.47 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.48 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.49 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.4f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.50 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.51 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.52 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.53 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.54 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.55 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.56 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.57 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.58 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.59 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.5f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.60 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.61 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.62 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.63 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.64 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.65 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.66 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.67 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.68 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.69 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.6f 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.70 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.71 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.72 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.73 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.74 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.75 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.76 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.77 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.78 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.79 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7a 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7b 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7c 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7d 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7e 2024-01-03T14:30:51.263+0800 7f6ccce06700 0 osd.0 43 send_pg_created: pg_created_insert2 1.7f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.0 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.8 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.9 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.10 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.11 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.12 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.13 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.14 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.15 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.16 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.17 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.18 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.19 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.1f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.20 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.21 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.22 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.23 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.24 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.25 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.26 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.27 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.28 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.29 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.2f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.30 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.31 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.32 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.33 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.34 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.35 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.36 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.37 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.38 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.39 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.3f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.40 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.41 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.42 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.43 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.44 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.45 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.46 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.47 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.48 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.49 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.4f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.50 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.51 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.52 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.53 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.54 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.55 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.56 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.57 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.58 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.59 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.5f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.60 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.61 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.62 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.63 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.64 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.65 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.66 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.67 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.68 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.69 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.6f 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.70 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.71 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.72 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.73 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.74 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.75 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.76 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.77 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.78 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.79 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7a 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7b 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7c 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7d 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7e 2024-01-03T14:30:51.760+0800 7f6cc0bdf700 0 osd.0 44 prune_pg_created pruning 1.7f 2024-01-03T14:35:52.857+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T14:40:54.106+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T14:45:55.882+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T14:50:56.936+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T14:55:58.137+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T15:00:59.403+0800 7f6ccce06700 0 osd.0 44 send_pg_created 2024-01-03T15:06:01.175+0800 7f6ccce06700 0 osd.0 44 send_pg_created
Updated by jianwei zhang 5 months ago
osd force-create-pg <PGID> reproduce:
# ceph -s
cluster:
id: db340f9a-d843-47b8-8280-6c724a6fa74e
health: HEALTH_OK
services:
mon: 1 daemons, quorum a (age 77s)
mgr: x(active, since 69s)
osd: 1 osds: 1 up (since 63s), 1 in (since 5m)
data:
pools: 1 pools, 128 pgs
objects: 0 objects, 0 B
usage: 228 KiB used, 9.1 TiB / 9.1 TiB avail
pgs: 128 active+clean
# ceph osd pool ls detail
pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 application rgw
vi ceph.conf [osd.0] debug_osd = 20 debug_ms = 1 [mon.a] debug_mon = 20 debug_ms = 1 init-ceph restart mon.a osd.0
# ceph osd pool ls detail pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 application rgw
ceph osd force-create-pg 1.0 --yes-i-really-mean-it pg 1.0 now creating, ok
ceph osd pool ls detail pool 1 'test-pool-0' replicated size 1 min_size 1 crush_rule 1 object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off last_change 40 flags hashpspool,creating stripe_width 0 application rgw
osd.0.log
[root@zjw-cmain-dev build]# grep 7f71be537700 out/osd.0.log
2024-01-04T14:06:37.251+0800 7f71be537700 1 -- v2:127.0.0.1:6804/96647 <== mon.0 v2:127.0.0.1:40328/0 15 ==== pg_create2(e59 {1.0=58,2024-01-04T14:06:36.568113+0800}) v3 ==== 208+0+0 (crc 0 0 0) 0x55ee8ecd9a40 con 0x55ee8fb5bc00
2024-01-04T14:06:37.251+0800 7f71be537700 7 osd.0 59 handle_fast_pg_create pg_create2(e59 {1.0=58,2024-01-04T14:06:36.568113+0800}) v3 from mon.0
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 59 handle_fast_pg_create 1.0 e58@2024-01-04T14:06:36.568113+0800 history ec=58/58 lis/c=0/0 les/c/f=0/0/0 sis=58 pi ([0,0] all_participants= intervals=)
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 59 handle_fast_pg_create 1.0 e58@2024-01-04T14:06:36.568113+0800 (has history or past_intervals)
2024-01-04T14:06:37.251+0800 7f71be537700 15 osd.0 59 enqueue_peering_evt 1.0 epoch_sent: 59 epoch_requested: 59 NullEvt +create_info
2024-01-04T14:06:37.251+0800 7f71be537700 20 osd.0 op_wq(0) _enqueue OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59)
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process 1.0 to_process <> waiting <> waiting_peering {}
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59) queued
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process 1.0 to_process <OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59)> waiting <> waiting_peering {}
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process OpSchedulerItem(1.0 PGPeeringEvent(epoch_sent: 59 epoch_requested: 59 NullEvt +create_info) prio 10 cost 10 e59) pg 0x55ee8ffd3000
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 dequeue_peering_evt enter evt = epoch_sent: 59 epoch_requested: 59 NullEvt +create_info pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 advance_pg pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 advance_pg return true osd_epoch<=pg->get_osdmap_epoch() 59 pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 59 dequeue_peering_evt advance_pg=true evt = epoch_sent: 59 epoch_requested: 59 NullEvt +create_info pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean]
2024-01-04T14:06:37.255+0800 7f719e26b700 10 osd.0 pg_epoch: 59 pg[1.0( empty local-lis/les=57/58 n=0 ec=24/24 lis/c=57/57 les/c/f=58/58/0 sis=57) [0] r=0 lpr=57 crt=0'0 mlcod 0'0 active+clean] do_peering_event: epoch_sent: 59 epoch_requested: 59 NullEvt +create_info
2024-01-04T14:06:37.255+0800 7f719e26b700 10 log is not dirty
2024-01-04T14:06:37.255+0800 7f719e26b700 20 osd.0 op_wq(0) _process empty q, waiting
# grep -nr send_pg_created out/osd.0.log # //nothing
Updated by jianwei zhang 4 months ago
In order to identify the creation triggered by ceph osd force-create-pg 1.0,
- A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55239)
- Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55041)
Updated by jianwei zhang 4 months ago
jianwei zhang wrote:
correct:In order to identify the creation triggered by ceph osd force-create-pg 1.0,
- A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55239)
- Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55041)
- Still use PgCreatEvt to identify which pgid sent MOSDPGCreate request. (Solution 1: https://github.com/ceph/ceph/pull/55239)
- A pg_creating queue may be needed on the OSD side to identify which pgid sent pg_create2.(Solution 2: https://github.com/ceph/ceph/pull/55041)
Actions