Actions
Bug #2153
closedosd: locking cycle on pg::_lock. op_tp::lock, OSD::map_lock
Source:
Development
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
------------------------------------ existing dependency PG::_lock (50) -> OSD::op_tp::lock (24) at: ceph version 0.43-138-g97b2e74 (commit:97b2e741423a8a9801ead843141a490043f7788b) 2012-03-08 09:16:18.983547 1: (Mutex::Lock(bool)+0x41) [0xb00eb5] 2012-03-08 09:16:18.983559 2: (ThreadPool::WorkQueue<PG>::queue(PG*)+0x29) [0xb18a11] 2012-03-08 09:16:18.983567 3: (OSD::enqueue_op(PG*, OpRequest*)+0x28f) [0xba6033] 2012-03-08 09:16:18.983574 4: (OSD::handle_op(OpRequest*)+0xeca) [0xba3f3e] 2012-03-08 09:16:18.983581 5: (OSD::dispatch_op(OpRequest*)+0x146) [0xb8b014] 2012-03-08 09:16:18.983587 6: (OSD::_dispatch(Message*)+0x887) [0xb8b8c9] 2012-03-08 09:16:18.983594 7: (OSD::ms_dispatch(Message*)+0x165) [0xb8a13d] 2012-03-08 09:16:18.983600 8: (Messenger::ms_deliver_dispatch(Message*)+0x89) [0xcc25ff] 2012-03-08 09:16:18.983607 9: (SimpleMessenger::dispatch_entry()+0x768) [0xcac3d0] 2012-03-08 09:16:18.983614 10: (SimpleMessenger::DispatchThread::entry()+0x1c) [0xaae85c] 2012-03-08 09:16:18.983621 11: (Thread::_entry_func(void*)+0x23) [0xccaea5] 2012-03-08 09:16:18.983634 12: (()+0x6b50) [0x7f425377cb50] 2012-03-08 09:16:18.983640 13: (clone()+0x6d) [0x7f4251db090d] 2012-03-08 09:16:18.983645 2012-03-08 09:16:18.983654 7f4246aa8700 existing intermediate dependency OSD::map_lock (32) -> PG::_lock (50) at: ceph version 0.43-138-g97b2e74 (commit:97b2e741423a8a9801ead843141a490043f7788b) 2012-03-08 09:16:18.983661 1: (Mutex::Lock(bool)+0x41) [0xb00eb5] 2012-03-08 09:16:18.983667 2: (PG::lock_with_map_lock_held(bool)+0x27) [0xd80617] 2012-03-08 09:16:18.983675 3: (OSD::advance_map(ObjectStore::Transaction&, C_Contexts*)+0x1324) [0xb92f9e] 2012-03-08 09:16:18.983682 4: (OSD::handle_osd_map(MOSDMap*)+0x20f2) [0xb8ffee] 2012-03-08 09:16:18.983688 5: (OSD::_dispatch(Message*)+0x4be) [0xb8b500] 2012-03-08 09:16:18.983695 6: (OSD::ms_dispatch(Message*)+0x165) [0xb8a13d] 2012-03-08 09:16:18.983701 7: (Messenger::ms_deliver_dispatch(Message*)+0x89) [0xcc25ff] 2012-03-08 09:16:18.983707 8: (SimpleMessenger::dispatch_entry()+0x768) [0xcac3d0] 2012-03-08 09:16:18.983714 9: (SimpleMessenger::DispatchThread::entry()+0x1c) [0xaae85c] 2012-03-08 09:16:18.983728 10: (Thread::_entry_func(void*)+0x23) [0xccaea5] 2012-03-08 09:16:18.983735 11: (()+0x6b50) [0x7f425377cb50] 2012-03-08 09:16:18.983741 12: (clone()+0x6d) [0x7f4251db090d] 2012-03-08 09:16:18.983745 2012-03-08 09:16:18.983752 7f4246aa8700 new dependency OSD::op_tp::lock (24) -> OSD::map_lock (32) creates a cycle at ceph version 0.43-138-g97b2e74 (commit:97b2e741423a8a9801ead843141a490043f7788b) 2012-03-08 09:16:18.983760 1: (PG::lock(bool)+0x28) [0xd80562] 2012-03-08 09:16:18.983766 2: (OSD::handle_osd_map(MOSDMap*)+0xbe3) [0xb8eadf] 2012-03-08 09:16:18.983772 3: (OSD::_dispatch(Message*)+0x4be) [0xb8b500] 2012-03-08 09:16:18.983778 4: (OSD::ms_dispatch(Message*)+0x165) [0xb8a13d] 2012-03-08 09:16:18.983784 5: (Messenger::ms_deliver_dispatch(Message*)+0x89) [0xcc25ff] 2012-03-08 09:16:18.983790 6: (SimpleMessenger::dispatch_entry()+0x768) [0xcac3d0] 2012-03-08 09:16:18.983797 7: (SimpleMessenger::DispatchThread::entry()+0x1c) [0xaae85c] 2012-03-08 09:16:18.983803 8: (Thread::_entry_func(void*)+0x23) [0xccaea5] 2012-03-08 09:16:18.983809 9: (()+0x6b50) [0x7f425377cb50] 2012-03-08 09:16:18.983815 10: (clone()+0x6d) [0x7f4251db090d] 2012-03-08 09:16:18.983819 2012-03-08 09:16:18.983823 7f4246aa8700 btw, i am holding these locks: 2012-03-08 09:16:18.983829 7f4246aa8700 OSD::osd_lock (21) 2012-03-08 09:16:18.983834 7f4246aa8700 OSD::op_tp::lock (24) 2012-03-08 09:16:18.983839 7f4246aa8700 common/lockdep.cc: In function 'int lockdep_will_lock(const char*, int)' thread 7f4246aa8700 time 2012-03-08 09:16:18.983846 common/lockdep.cc: 201: FAILED assert(0) ceph version 0.43-138-g97b2e74 (commit:97b2e741423a8a9801ead843141a490043f7788b) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xae) [0xcd5fba] 2: (lockdep_will_lock(char const*, int)+0x8f8) [0xd4f37f] 3: (RWLock::get_read()+0x32) [0xbac714] 4: (PG::lock(bool)+0x28) [0xd80562] 5: (OSD::handle_osd_map(MOSDMap*)+0xbe3) [0xb8eadf] 6: (OSD::_dispatch(Message*)+0x4be) [0xb8b500] 7: (OSD::ms_dispatch(Message*)+0x165) [0xb8a13d] 8: (Messenger::ms_deliver_dispatch(Message*)+0x89) [0xcc25ff] 9: (SimpleMessenger::dispatch_entry()+0x768) [0xcac3d0] 10: (SimpleMessenger::DispatchThread::entry()+0x1c) [0xaae85c] 11: (Thread::_entry_func(void*)+0x23) [0xccaea5] 12: (()+0x6b50) [0x7f425377cb50] 13: (clone()+0x6d) [0x7f4251db090d]
Updated by Sage Weil about 12 years ago
- Status changed from 12 to Resolved
- Assignee set to Sage Weil
Actions