Project

General

Profile

Bug #3208

osd/PG.cc: 1346: FAILED assert(info.last_complete >= log.tail)

Added by Sage Weil over 11 years ago. Updated over 11 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
OSD
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2012-09-24 12:57:37.271503 7f668dffb700 -1 osd/PG.cc: In function 'void PG::activate(ObjectStore::Transaction&, epoch_t, std::list<Context*>&, std::map<int, std::map<pg_t, pg_query_t> >&, std::map<int, std::vector<std::pair<pg_notify_t, std::map<unsigned int, pg_interval_t> > > >*)' thread 7f668dffb700 time 2012-09
-24 12:57:37.237738
osd/PG.cc: 1346: FAILED assert(info.last_complete >= log.tail)

 ceph version 0.51-699-g7782ccb (commit:7782ccb5f69dbe0f935ed872a688caa0f022ac32)
 1: (PG::activate(ObjectStore::Transaction&, unsigned int, std::list<Context*, std::allocator<Context*> >&, std::map<int, std::map<pg_t, pg_query_t, std::less<pg_t>, std::allocator<std::pair<pg_t const, pg_query_t> > >, std::less<int>, std::allocator<std::pair<int const, std::map<pg_t, pg_query_t, std::less<pg_t>, 
std::allocator<std::pair<pg_t const, pg_query_t> > > > > >&, std::map<int, std::vector<std::pair<pg_notify_t, std::map<unsigned int, pg_interval_t, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > > >, std::allocator<std::pair<pg_notify_t, std::map<unsigned int, pg_interval_t, 
std::less<unsigned int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > > > > >, std::less<int>, std::allocator<std::pair<int const, std::vector<std::pair<pg_notify_t, std::map<unsigned int, pg_interval_t, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > > >, std
::allocator<std::pair<pg_notify_t, std::map<unsigned int, pg_interval_t, std::less<unsigned int>, std::allocator<std::pair<unsigned int const, pg_interval_t> > > > > > > > >*)+0x172) [0x65fc12]
 2: (PG::RecoveryState::Active::Active(boost::statechart::state<PG::RecoveryState::Active, PG::RecoveryState::Primary, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, 
mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::my_context)+0x279) [0x662e79]
 3: (boost::statechart::state<PG::RecoveryState::Active, PG::RecoveryState::Primary, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::state
chart::history_mode)0>::shallow_construct(boost::intrusive_ptr<PG::RecoveryState::Primary> const&, boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>&)+0x4e) [0x6810ae]
 4: (boost::statechart::detail::safe_reaction_result boost::statechart::simple_state<PG::RecoveryState::Peering, PG::RecoveryState::Primary, PG::RecoveryState::GetInfo, (boost::statechart::history_mode)0>::transit_impl<PG::RecoveryState::Active, PG::RecoveryState::RecoveryMachine, boost::statechart::detail::no_tran
sition_function>(boost::statechart::detail::no_transition_function const&)+0x9e) [0x6812de]
 5: (boost::statechart::simple_state<PG::RecoveryState::Peering, PG::RecoveryState::Primary, PG::RecoveryState::GetInfo, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x182) [0x681552]
 6: (boost::statechart::simple_state<PG::RecoveryState::WaitFlushedPeering, PG::RecoveryState::Peering, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_:
:na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0x88) [0x67ed78]
 7: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0xfb) [0x6743bb]
 8: (boost::statechart::state_machine<PG::RecoveryState::RecoveryMachine, PG::RecoveryState::Initial, std::allocator<void>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x1e) [0x67451e]
 9: (PG::handle_peering_event(std::tr1::shared_ptr<PG::CephPeeringEvt>, PG::RecoveryCtx*)+0x347) [0x638cd7]
 10: (OSD::process_peering_events(std::list<PG*, std::allocator<PG*> > const&)+0x263) [0x5e9043]
 11: (OSD::PeeringWQ::_process(std::list<PG*, std::allocator<PG*> > const&)+0x10) [0x618fd0]
 12: (ThreadPool::worker(ThreadPool::WorkThread*)+0x517) [0x7cb6d7]
 13: (ThreadPool::WorkThread::entry()+0x10) [0x7cd620]
 14: (()+0x7e9a) [0x7f66c077ae9a]
 15: (clone()+0x6d) [0x7f66bed8b4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

full logs on plana61 osd.1
teuthology log at ubuntu@teuthology:/a/sage-q2/28596

History

#1 Updated by Samuel Just over 11 years ago

  • Assignee set to Samuel Just

#2 Updated by Samuel Just over 11 years ago

wip_last_complete_assert has a fix/explanation.

#3213 also needs to be handled in this case.

#3 Updated by Samuel Just over 11 years ago

  • Status changed from 12 to Resolved

Also available in: Atom PDF