Project

General

Profile

Actions

Bug #20198

closed

mgr: tcmalloc segv in DaemonServer ctor

Added by Sage Weil almost 7 years ago. Updated almost 7 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2017-06-06T00:40:47.776 INFO:tasks.ceph.mgr.x.smithi092.stderr:*** Caught signal (Segmentation fault) **
2017-06-06T00:40:47.776 INFO:tasks.ceph.mgr.x.smithi092.stderr: in thread 7fe1c1ccb700 thread_name:ms_dispatch
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: ceph version  12.0.2-2293-g4f85f10 (4f85f10b3e0cc0e200b6c96c298fa68a171ddb96) luminous (dev)
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 1: (()+0x393d52) [0x562e86a78d52]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 2: (()+0x113e0) [0x7fe1c71873e0]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 3: (tcmalloc::CentralFreeList::FetchFromOneSpans(int, void**, void**)+0x33) [0x7fe1c7d5bbb3]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 4: (tcmalloc::CentralFreeList::FetchFromOneSpansSafe(int, void**, void**)+0x1a) [0x7fe1c7d5be8a]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 5: (tcmalloc::CentralFreeList::RemoveRange(void**, void**, int)+0x6f) [0x7fe1c7d5bf3f]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 6: (tcmalloc::ThreadCache::FetchFromCentralCache(unsigned long, unsigned long)+0x6a) [0x7fe1c7d5ef2a]
2017-06-06T00:40:47.780 INFO:tasks.ceph.mgr.x.smithi092.stderr: 7: (operator new[](unsigned long)+0x398) [0x7fe1c7d712c8]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 8: (std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, PerfCounters::perf_counter_data_any_d*> > std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<
char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, PerfCounters::perf_counter_data_any_d*>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, PerfCounters::perf_counter_data_any_d*> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, PerfCounters::perf_counter_data_any_d*> > >::_M_emplace_hint_unique<std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>, std::tuple<> >(std::_Rb_tree_const_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, PerfCounters::perf_counter_data_any_d*> >, std::piecewise_construct_t const&, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>&&, std::tuple<>&&)+0x1f) [0x562e86a982af]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 9: (PerfCountersCollection::add(PerfCounters*)+0x6de) [0x562e86a96ace]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 10: (Throttle::Throttle(CephContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long, bool)+0x38d) [0x562e86aaf58d]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 11: (DaemonServer::DaemonServer(MonClient*, Finisher&, DaemonStateIndex&, ClusterState&, PyModules&, std::shared_ptr<LogChannel>, std::shared_ptr<LogChannel>)+0x22d) [0x562e86945cbd]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 12: (Mgr::Mgr(MonClient*, Messenger*, Objecter*, Client*, std::shared_ptr<LogChannel>, std::shared_ptr<LogChannel>)+0x6af) [0x562e8698476f]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 13: (MgrStandby::handle_mgr_map(MMgrMap*)+0x8d9) [0x562e869760d9]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 14: (MgrStandby::ms_dispatch(Message*)+0x113) [0x562e869764b3]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 15: (DispatchQueue::entry()+0xf4a) [0x562e86d0f99a]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 16: (DispatchQueue::DispatchThread::entry()+0xd) [0x562e86b679dd]
2017-06-06T00:40:47.781 INFO:tasks.ceph.mgr.x.smithi092.stderr: 17: (()+0x770a) [0x7fe1c717d70a]
2017-06-06T00:40:47.782 INFO:tasks.ceph.mgr.x.smithi092.stderr: 18: (clone()+0x6d) [0x7fe1c61e982d]
2017-06-06T00:40:47.782 INFO:tasks.ceph.mgr.x.smithi092.stderr:2017-06-06 00:40:47.776734 7fe1c1ccb700 -1 *** Caught signal (Segmentation fault) **
2017-06-06T00:40:47.782 INFO:tasks.ceph.mgr.x.smithi092.stderr: in thread 7fe1c1ccb700 thread_name:ms_dispatch

/a/sage-2017-06-05_22:19:51-rados-wip-sage-testing-distro-basic-smithi/1262626
rados/monthrash/{ceph.yaml clusters/3-mons.yaml d-require-luminous/at-end.yaml mon_kv_backend/rocksdb.yaml msgr-failures/mon-delay.yaml msgr/random.yaml objectstore/filestore-xfs.yaml rados.yaml thrashers/many.yaml workloads/rados_api_tests.yaml}
Actions #1

Updated by Sage Weil almost 7 years ago

  • Status changed from 12 to Resolved

this looks like fallout from the old broken Mgr shutdown; we now respawn.

(in this run we had just shut down and were trying to reactivate)

Actions

Also available in: Atom PDF