Project

General

Profile

Tasks #12118

ceph-mon crashed when starting up

Added by chuanhong wang over 4 years ago. Updated over 4 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Tags:
Reviewed:
Affected Versions:
Pull request ID:

Description

environment: ceph v0.87 + centos7
ceph-mon crashed when start up. Device 9 is not exist but the mon still check the health of it. Threre is the stack info below, and the crush is attatched.

Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7f5b91227700 (LWP 26238)]
0x00007f5b98dbf989 in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007f5b98dbf989 in raise () from /lib64/libc.so.6
#1 0x00007f5b98dc1098 in abort () from /lib64/libc.so.6
#2 0x00007f5b996c39d5 in gnu_cxx::__verbose_terminate_handler() () from /lib64/libstdc++.so.6
#3 0x00007f5b996c1946 in ?? () from /lib64/libstdc++.so.6
#4 0x00007f5b996c1973 in std::terminate() () from /lib64/libstdc++.so.6
#5 0x00007f5b996c1b9f in _cxa_throw () from /lib64/libstdc++.so.6
#6 0x00000000007d77ca in ceph::
_ceph_assert_fail (assertion=assertion@entry=0x9867a7 "exists(osd)", file=file@entry=0x98678c "./osd/OSDMap.h", line=line@entry=442,
func=func@entry=0x98cfa0 <OSDMap::get_addr(int) const::__PRETTY_FUNCTION
> "const entity_addr_t& OSDMap::get_addr(int) const") at common/assert.cc:77
#7 0x000000000055e662 in OSDMap::get_addr (osd=9, this=0x4080088) at osd/OSDMap.h:442
#8 0x0000000000604a11 in get_addr (osd=9, this=0x4080088) at osd/OSDMap.h:406
Python Exception <type 'exceptions.IndexError'> list index out of range:
#9 OSDMonitor::get_health (this=0x4080000, summary=std::list, detail=0x7f5b912252e0) at mon/OSDMonitor.cc:2119
#10 0x00000000005a9e81 in Monitor::get_health (this=0x4070000, status="", detailbl=0x7f5b91225bd0, f=0x0) at mon/Monitor.cc:2099
#11 0x00000000005bec14 in Monitor::handle_command (this=0x4070000, m=0x3e72400) at mon/Monitor.cc:2588
#12 0x00000000005c0cbb in Monitor::dispatch (this=0x4070000, s=0x3e69a00, m=0x3e72400, src_is_mon=<optimized out>) at mon/Monitor.cc:3240
#13 0x00000000005c173d in Monitor::_ms_dispatch (this=this@entry=0x4070000, m=m@entry=0x3e72400) at mon/Monitor.cc:3156
#14 0x00000000005e2343 in Monitor::ms_dispatch (this=0x4070000, m=0x3e72400) at mon/Monitor.h:785
#15 0x0000000000918d2a in ms_deliver_dispatch (m=0x3e72400, this=0x4060000) at msg/Messenger.h:532
#16 DispatchQueue::entry (this=0x40601b8) at msg/DispatchQueue.cc:185
#17 0x00000000007bbe7d in DispatchQueue::DispatchThread::entry (this=<optimized out>) at msg/DispatchQueue.h:104
#18 0x00007f5b9a39ddf3 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f5b98e803dd in clone () from /lib64/libc.so.6
(gdb) f 6
#6 0x00000000007d77ca in ceph::__ceph_assert_fail (assertion=assertion@entry=0x9867a7 "exists(osd)", file=file@entry=0x98678c "./osd/OSDMap.h", line=line@entry=442,
func=func@entry=0x98cfa0 <OSDMap::get_addr(int) const::__PRETTY_FUNCTION__> "const entity_addr_t& OSDMap::get_addr(int) const") at common/assert.cc:77
77 throw FailedAssertion(bt);
(gdb) f 7
#7 0x000000000055e662 in OSDMap::get_addr (osd=9, this=0x4080088) at osd/OSDMap.h:442
442 assert(exists(osd));
(gdb) p osd
$1 = 9
(gdb)

crush.txt View (1.54 KB) chuanhong wang, 06/23/2015 07:22 AM

History

#1 Updated by chuanhong wang over 4 years ago

Sorry, I have make a mistake. Some guys in my commpany changed the code of function "OSDMonitor::get_health" and result in this issue.

#2 Updated by Nathan Cutler over 4 years ago

  • Project changed from Stable releases to Ceph

#3 Updated by Nathan Cutler over 4 years ago

  • Status changed from New to Rejected

Also available in: Atom PDF