Project

General

Profile

Actions

Bug #16237

closed

AsyncConnection::lockmsg/async lockdep cycle: AsyncMessenger::lock, MDSDaemon::mds_lock, AsyncConnection::lock

Added by Sage Weil almost 8 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

------------------------------------
existing dependency AsyncMessenger::lock (1) -> AsyncConnection::lock (13) at:

    -5> 2016-06-10 17:39:06.934839 7fe2d67fc700  0 existing intermediate dependency MDSDaemon::mds_lock (18) -> AsyncMessenger::lock (1) at:

    -4> 2016-06-10 17:39:06.936524 7fe2d67fc700  0 new dependency AsyncConnection::lock (13) -> MDSDaemon::mds_lock (18) creates a cycle at
 ceph version 10.2.0-2221-ge6c2ee0 (e6c2ee0958e18f048f9ba97a96aa1cb5b528a7ec)
 1: (MDSDaemon::ms_verify_authorizer(Connection*, int, int, ceph::buffer::list&, ceph::buffer::list&, bool&, CryptoKey&)+0x5a) [0x55bae0f0a86a]
 2: (AsyncConnection::handle_connect_msg(ceph_msg_connect&, ceph::buffer::list&, ceph::buffer::list&)+0x394) [0x55bae14aa674]
 3: (AsyncConnection::_process_connection()+0x2491) [0x55bae14b0481]
 4: (AsyncConnection::process()+0x1fb0) [0x55bae14b5630]
 5: (EventCenter::process_events(int)+0xa38) [0x55bae1447618]
 6: (Worker::entry()+0x375) [0x55bae1416125]
 7: (()+0x760a) [0x7fe2e144360a]
 8: (clone()+0x6d) [0x7fe2dfe92a4d]

    -3> 2016-06-10 17:39:06.936593 7fe2d67fc700  0 btw, i am holding these locks:
    -2> 2016-06-10 17:39:06.936596 7fe2d67fc700  0   AsyncConnection::lock (13)
    -1> 2016-06-10 17:39:06.936598 7fe2d67fc700  0 

     0> 2016-06-10 17:39:06.938453 7fe2d67fc700 -1 common/lockdep.cc: In function 'int lockdep_will_lock(const char*, int, bool)' thread 7fe2d67fc700 time 2016-06-10 17:39:06.936601
common/lockdep.cc: 330: FAILED assert(0)

 ceph version 10.2.0-2221-ge6c2ee0 (e6c2ee0958e18f048f9ba97a96aa1cb5b528a7ec)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x80) [0x55bae132b330]
 2: (lockdep_will_lock(char const*, int, bool)+0x11f5) [0x55bae13b4135]
 3: (Mutex::Lock(bool)+0x15f) [0x55bae12de49f]
 4: (MDSDaemon::ms_verify_authorizer(Connection*, int, int, ceph::buffer::list&, ceph::buffer::list&, bool&, CryptoKey&)+0x5a) [0x55bae0f0a86a]
 5: (AsyncConnection::handle_connect_msg(ceph_msg_connect&, ceph::buffer::list&, ceph::buffer::list&)+0x394) [0x55bae14aa674]
 6: (AsyncConnection::_process_connection()+0x2491) [0x55bae14b0481]
 7: (AsyncConnection::process()+0x1fb0) [0x55bae14b5630]
 8: (EventCenter::process_events(int)+0xa38) [0x55bae1447618]
 9: (Worker::entry()+0x375) [0x55bae1416125]
 10: (()+0x760a) [0x7fe2e144360a]
 11: (clone()+0x6d) [0x7fe2dfe92a4d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.


triggered by ./test/cephtool-test-mds.sh on master, v10.2.0-2221-ge6c2ee0

Related issues 1 (0 open1 closed)

Copied to Ceph - Backport #16374: jewel: AsyncConnection::lockmsg/async lockdep cycle: AsyncMessenger::lock, MDSDaemon::mds_lock, AsyncConnection::lockResolvedLoïc DacharyActions
Actions #1

Updated by Haomai Wang almost 8 years ago

  • Category set to msgr
  • Status changed from New to Fix Under Review
  • Assignee set to Haomai Wang
Actions #2

Updated by Kefu Chai almost 8 years ago

  • Status changed from Fix Under Review to Resolved
Actions #3

Updated by Haomai Wang almost 8 years ago

  • Status changed from Resolved to Pending Backport
  • Backport set to Jewel
Actions #4

Updated by Nathan Cutler almost 8 years ago

  • Backport changed from Jewel to jewel
Actions #5

Updated by Nathan Cutler almost 8 years ago

  • Copied to Backport #16374: jewel: AsyncConnection::lockmsg/async lockdep cycle: AsyncMessenger::lock, MDSDaemon::mds_lock, AsyncConnection::lock added
Actions #6

Updated by Loïc Dachary over 7 years ago

  • Status changed from Pending Backport to Resolved
Actions #7

Updated by Greg Farnum about 5 years ago

  • Project changed from Ceph to Messengers
  • Category deleted (msgr)
Actions

Also available in: Atom PDF