Actions
Bug #55101
openmon has slow op
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
there are 3 nodes in our cluster, 84 OSD each node.
after execute "systemctl restart ceph-osd.target" on node2, there is 1 slow op in mon.b
"ops": [
{
"description": "osd_alive(want up_thru 1348 have 1348)",
"initiated_at": "2022-03-29T12:10:11.016387+0800",
"age": 9036.0939306100008,
"duration": 9036.0939519370004,
"type_data": {
"events": [
{
"time": "2022-03-29T12:10:11.016387+0800",
"event": "initiated"
},
{
"time": "2022-03-29T12:10:11.016391+0800",
"event": "throttled"
},
{
"time": "2022-03-29T12:10:11.016387+0800",
"event": "header_read"
},
{
"time": "2022-03-29T12:10:11.016404+0800",
"event": "all_read"
},
{
"time": "2022-03-29T12:10:11.016667+0800",
"event": "dispatched"
},
{
"time": "2022-03-29T12:10:11.016675+0800",
"event": "mon:_ms_dispatch"
},
{
"time": "2022-03-29T12:10:11.016677+0800",
"event": "mon:dispatch_op"
},
{
"time": "2022-03-29T12:10:11.016678+0800",
"event": "psvc:dispatch"
},
{
"time": "2022-03-29T12:10:11.016699+0800",
"event": "osdmap:preprocess_query"
},
{
"time": "2022-03-29T12:10:11.016702+0800",
"event": "osdmap:preprocess_alive"
},
{
"time": "2022-03-29T12:10:11.016717+0800",
"event": "osdmap:wait_for_writeable"
},
{
"time": "2022-03-29T12:10:11.016718+0800",
"event": "osdmap:wait_for_finished_proposal"
},
{
"time": "2022-03-29T12:10:11.721484+0800",
"event": "callback finished"
},
{
"time": "2022-03-29T12:10:11.721485+0800",
"event": "psvc:dispatch"
},
{
"time": "2022-03-29T12:10:11.721499+0800",
"event": "osdmap:preprocess_query"
},
{
"time": "2022-03-29T12:10:11.721502+0800",
"event": "osdmap:preprocess_alive"
},
{
"time": "2022-03-29T12:10:11.721510+0800",
"event": "osdmap:prepare_update"
},
{
"time": "2022-03-29T12:10:11.721512+0800",
"event": "osdmap:prepare_alive"
},
{
"time": "2022-03-29T12:10:11.721515+0800",
"event": "osdmap:wait_for_finished_proposal"
},
{
"time": "2022-03-29T12:10:12.548334+0800",
"event": "callback retry"
},
{
"time": "2022-03-29T12:10:12.548335+0800",
"event": "psvc:dispatch"
},
{
"time": "2022-03-29T12:10:12.548350+0800",
"event": "osdmap:wait_for_readable"
},
{
"time": "2022-03-29T12:10:12.548352+0800",
"event": "osdmap:wait_for_readable/paxos"
},
{
"time": "2022-03-29T12:10:12.548360+0800",
"event": "paxos:wait_for_readable"
},
{
"time": "2022-03-29T12:10:12.775644+0800",
"event": "callback finished"
},
{
"time": "2022-03-29T12:10:12.775645+0800",
"event": "psvc:dispatch"
},
{
"time": "2022-03-29T12:10:12.775669+0800",
"event": "osdmap:preprocess_query"
},
{
"time": "2022-03-29T12:10:12.775671+0800",
"event": "osdmap:preprocess_alive"
},
{
"time": "2022-03-29T12:10:12.775683+0800",
"event": "forward_request_leader"
},
{
"time": "2022-03-29T12:10:12.775758+0800",
"event": "forwarded"
}
],
"info": {
"seq": 9009263,
"src_is_mon": false,
"source": "osd.128 v1:142.137.11.142:7263/1961019",
"forwarded_to_leader": true
}
}
Files
Updated by Radoslaw Zarzynski about 2 years ago
- Status changed from New to Need More Info
Hello! We would need to take a look on the mon.b's log, preferably also on the one preceding the restart.
Updated by liqun zhang about 2 years ago
the attachment includes:
ceph-mon.a.log ceph-mon.b.log ceph-mon.c.log
b.ops (ceph daemon mon.b ops)
c.ops(ceph daemon mon.c ops)
ceph_s.out(ceph -s)
Updated by Radoslaw Zarzynski about 2 years ago
- Status changed from Need More Info to New
- Assignee set to Siddharth Sharma
Actions