Project

General

Profile

Bug #24180

mon: slow op on log message

Added by Sage Weil 10 months ago. Updated 5 months ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
Target version:
-
Start date:
05/18/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
mimic, luminous, jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:

Description

2018-05-18 19:10:34.156 7fd045402700  1 -- 172.21.15.106:6789/0 <== mon.1 172.21.15.106:6790/0 745 ==== forward(log(4 entries from seq 28 at 2018-05-18 19:10:27.883036) v1 caps allow profile mgr tid 352 con_features 2305244844817448955) v3 ==== 1153+0+0 (271030207 0 0) 0x564893db7900 con 0x564893bbd500
2018-05-18 19:10:34.156 7fd045402700 20 mon.a@0(leader) e1 _ms_dispatch existing session 0x564893744d00 for mon.1 172.21.15.106:6790/0
2018-05-18 19:10:34.156 7fd045402700 20 mon.a@0(leader) e1  caps allow *
2018-05-18 19:10:34.156 7fd045402700 20 is_capable service=mon command= read on cap allow *
2018-05-18 19:10:34.156 7fd045402700 20  allow so far , doing grant allow *
2018-05-18 19:10:34.156 7fd045402700 20  allow all
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader) e1 received forwarded message from mgr.4103 172.21.15.106:0/12904 via mon.1 172.21.15.106:6790/0
2018-05-18 19:10:34.156 7fd045402700 20 is_capable service=mon command= exec on cap allow *
2018-05-18 19:10:34.156 7fd045402700 20  allow so far , doing grant allow *
2018-05-18 19:10:34.156 7fd045402700 20  allow all
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader) e1  caps are allow profile mgr
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader) e1  entity name 'mgr.x' type 16
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader) e1  mesg 0x564893d25680 from 172.21.15.106:6790/0
2018-05-18 19:10:34.156 7fd045402700 20 mon.a@0(leader) e1 _ms_dispatch existing session 0x564894771f80 for mgr.4103 -
2018-05-18 19:10:34.156 7fd045402700 20 mon.a@0(leader) e1  caps allow profile mgr
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader).paxosservice(logm 1..94) dispatch 0x564893d25680 log(4 entries from seq 28 at 2018-05-18 19:10:27.883036) v1 from mgr.4103 172.21.15.106:0/12904 con 0x564893e29440
2018-05-18 19:10:34.156 7fd045402700  5 mon.a@0(leader).paxos(paxos active c 1..188) is_readable = 1 - now=2018-05-18 19:10:34.162227 lease_expire=2018-05-18 19:10:38.559247 has v0 lc 188
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader).log v94 preprocess_query log(4 entries from seq 28 at 2018-05-18 19:10:27.883036) v1 from mgr.4103 172.21.15.106:0/12904
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader).log v94 preprocess_log log(4 entries from seq 28 at 2018-05-18 19:10:27.883036) v1 from mgr.4103
2018-05-18 19:10:34.156 7fd045402700 20 is_capable service=log command= write on cap allow profile mgr
2018-05-18 19:10:34.156 7fd045402700 20  allow so far , doing grant allow profile mgr
2018-05-18 19:10:34.156 7fd045402700 20  match
2018-05-18 19:10:34.156 7fd045402700 10 mon.a@0(leader).log v94   nothing new

meanwhile, on the originating peon,
2018-05-18 19:11:04.280 7fc192a73700 -1 mon.b@1(peon) e1 get_health_metrics reporting 1 slow ops, oldest is log(4 entries from seq 28 at 2018-05-18 19:10:27.883036)

/a/sage-2018-05-18_16:20:24-rados-wip-sage-testing-2018-05-18-0817-distro-basic-smithi/2548220

Related issues

Copied to RADOS - Backport #24195: mimic: mon: slow op on log message Resolved
Copied to RADOS - Backport #24198: luminous: mon: slow op on log message Resolved
Copied to RADOS - Backport #24293: jewel: mon: slow op on log message Resolved

History

#1 Updated by Sage Weil 10 months ago

  • Status changed from Verified to Need Review

#2 Updated by Sage Weil 10 months ago

  • Backport set to mimic, luminous

#3 Updated by Kefu Chai 10 months ago

  • Status changed from Need Review to Pending Backport

#4 Updated by Kefu Chai 10 months ago

#5 Updated by Kefu Chai 10 months ago

#6 Updated by Patrick Donnelly 10 months ago

  • Backport changed from mimic, luminous to mimic, luminous, jewel

#7 Updated by Nathan Cutler 10 months ago

#8 Updated by Nathan Cutler 5 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF