Project

General

Profile

Bug #2322

osd/ReplicatedPG.cc: 3832: FAILED assert(!object_contexts.size())

Added by Sage Weil almost 9 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
OSD
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

--- begin dump of recent events ---
2012-04-20 02:48:11.435759 7f9b2ab327a0  0 ceph version 0.45-245-gf2d4574 (commit:f2d4574bd604e71deefbdb853ae19684a4443f31), process ceph-osd, pid 3896
2012-04-20 02:48:11.437379 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount FIEMAP ioctl is NOT supported
2012-04-20 02:48:11.437410 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount did NOT detect btrfs
2012-04-20 02:48:11.437545 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount found snaps <>
2012-04-20 02:48:11.494460 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount: enabling WRITEAHEAD journal mode: btrfs not detected
2012-04-20 02:48:11.494765 7f9b2ab327a0  1 journal _open /tmp/cephtest/data/osd.1.journal fd 21: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2012-04-20 02:48:11.505540 7f9b2ab327a0  1 journal _open /tmp/cephtest/data/osd.1.journal fd 21: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2012-04-20 02:48:11.506637 7f9b2ab327a0  1 journal close /tmp/cephtest/data/osd.1.journal
2012-04-20 02:48:11.508634 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount FIEMAP ioctl is NOT supported
2012-04-20 02:48:11.508652 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount did NOT detect btrfs
2012-04-20 02:48:11.508796 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount found snaps <>
2012-04-20 02:48:11.561050 7f9b2ab327a0  0 filestore(/tmp/cephtest/data/osd.1.data) mount: enabling WRITEAHEAD journal mode: btrfs not detected
2012-04-20 02:48:11.561224 7f9b2ab327a0  1 journal _open /tmp/cephtest/data/osd.1.journal fd 30: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2012-04-20 02:48:11.561288 7f9b2ab327a0  1 journal _open /tmp/cephtest/data/osd.1.journal fd 30: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2012-04-20 02:48:14.531006 7f9b16f1c700  1 CephxAuthorizeHandler::verify_authorizer isvalid=1
2012-04-20 02:48:19.179499 7f9b19f22700  0 log [INF] : 0.2 scrub ok
2012-04-20 02:48:20.140447 7f9b15502700  1 CephxAuthorizeHandler::verify_authorizer isvalid=1
2012-04-20 02:48:20.140486 7f9b15502700  0 data -> (7.0)
2012-04-20 02:48:20.140492 7f9b15502700  0 rbd -> (7.0)
2012-04-20 02:48:24.180650 7f9b19f22700  0 log [INF] : 0.7 scrub ok
2012-04-20 02:48:26.128418 7f9b21832700  1 journal check_for_full at 34869248 : JOURNAL FULL 34869248 >= 3751935 (max_size 104857600 start 38621184)
2012-04-20 02:48:28.521661 7f9b16512700  1 CephxAuthorizeHandler::verify_authorizer isvalid=1
2012-04-20 02:48:36.415016 7f9b19721700  0 log [INF] : 0.11 scrub ok
2012-04-20 02:48:37.182632 7f9b19721700  0 log [INF] : 1.1 scrub ok
2012-04-20 02:48:39.183421 7f9b19721700  0 log [INF] : 1.6 scrub ok
2012-04-20 02:48:39.872152 7f9b1c828700  0 osd.1 12 pg[0.e( v 11'14 (0'0,11'14] n=14 ec=1 les/c 4/4 12/12/3) [1,0] r=0 lpr=12 lcod 11'13 mlcod 0'0 inactive] Object f904ebfe/plana52_32460_object274/head still has ref count of 1
2012-04-20 02:48:39.876632 7f9b1c828700 -1 osd/ReplicatedPG.cc: In function 'void ReplicatedPG::context_registry_on_change()' thread 7f9b1c828700 time 2012-04-20 02:48:39.872232
osd/ReplicatedPG.cc: 3832: FAILED assert(!object_contexts.size())

 ceph version 0.45-245-gf2d4574 (commit:f2d4574bd604e71deefbdb853ae19684a4443f31)
 1: (ReplicatedPG::context_registry_on_change()+0x256) [0x5391b6]
 2: (ReplicatedPG::on_change()+0x194) [0x553c44]
 3: (PG::start_peering_interval(std::tr1::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&)+0x821) [0x85a4d1]
 4: (PG::RecoveryState::Reset::react(PG::RecoveryState::AdvMap const&)+0x4c1) [0x85c121]
 5: (boost::statechart::simple_state<PG::RecoveryState::Reset, PG::RecoveryState::RecoveryMachine, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x1c6) [0x88df66]
 6: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0x17b) [0x5c1deb]
 7: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x14c) [0x8811cc]
 8: (PG::RecoveryState::handle_advance_map(std::tr1::shared_ptr<OSDMap const>, std::tr1::shared_ptr<OSDMap const>, std::vector<int, std::allocator<int> >&, std::vector<int, std::allocator<int> >&, PG::RecoveryCtx*)+0x3fd) [0x8401ed]
 9: (OSD::advance_map(ObjectStore::Transaction&, C_Contexts*)+0x39ea) [0x64b05a]
 10: (OSD::handle_osd_map(MOSDMap*)+0x3577) [0x64fa17]
 11: (OSD::_dispatch(Message*)+0x5d3) [0x653443]
 12: (OSD::ms_dispatch(Message*)+0x1e3) [0x653fd3]
 13: (SimpleMessenger::dispatch_entry()+0xc33) [0x747b43]
 14: (SimpleMessenger::DispatchThread::entry()+0x15) [0x6f6cb5]
 15: (Thread::_entry_func(void*)+0x12) [0x765372]
 16: (()+0x7efc) [0x7f9b2a50aefc]
 17: (clone()+0x6d) [0x7f9b28b3b89d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- end dump of recent events ---

History

#1 Updated by Sage Weil almost 9 years ago

  • Target version changed from v0.46 to v0.47

#2 Updated by Sage Weil over 8 years ago

  • Status changed from 12 to Need More Info

also going to wait until the threading refactor is complete before diving into this further.

at this point asserting is the safest route. users can safely restarting the osd and be fine.

#3 Updated by Sage Weil over 8 years ago

  • Assignee set to Samuel Just

#4 Updated by Sage Weil over 8 years ago

  • Status changed from Need More Info to Resolved

Also available in: Atom PDF