Project

General

Profile

Bug #11814 » ceph-mon.2.log

Part of the log when mon is crashing - Herve Rousseau, 05/29/2015 02:36 PM

 
in thread 7ffff7d7a7a0
ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
1: /usr/bin/ceph-mon() [0x8fc1f5]
2: /lib64/libpthread.so.0() [0x3d02e0f710]
3: /usr/bin/ceph-mon() [0x79e48a]
4: (crush_do_rule()+0x4d1) [0x79ec61]
5: (CrushWrapper::do_rule(int, int, std::vector<int, std::allocator<int> >&, int, std::vector<unsigned int, std::allocator<unsigned int> > const&) const+0xa5) [0x80cdc5]
6: (OSDMap::_pg_to_osds(pg_pool_t const&, pg_t, std::vector<int, std::allocator<int> >*, int*, unsigned int*) const+0x96) [0x836c06]
7: (OSDMap::_pg_to_up_acting_osds(pg_t const&, std::vector<int, std::allocator<int> >*, int*, std::vector<int, std::allocator<int> >*, int*) const+0x124) [0x837044]
8: (PGMonitor::map_pg_creates()+0x2d4) [0x65c154]
9: (PGMonitor::post_paxos_update()+0x2d) [0x65cd7d]
10: (Monitor::refresh_from_paxos(bool*)+0x171) [0x583431]
11: (Monitor::init_paxos()+0xd5) [0x5836d5]
12: (Monitor::preinit()+0xaf1) [0x59a411]
13: (main()+0x39ba) [0x55519a]
14: (__libc_start_main()+0xfd) [0x3d02a1ed5d]
15: /usr/bin/ceph-mon() [0x54eb49]
2015-05-29 16:27:43.767499 7ffff7d7a7a0 -1 *** Caught signal (Segmentation fault) **
in thread 7ffff7d7a7a0

ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
1: /usr/bin/ceph-mon() [0x8fc1f5]
2: /lib64/libpthread.so.0() [0x3d02e0f710]
3: /usr/bin/ceph-mon() [0x79e48a]
4: (crush_do_rule()+0x4d1) [0x79ec61]
5: (CrushWrapper::do_rule(int, int, std::vector<int, std::allocator<int> >&, int, std::vector<unsigned int, std::allocator<unsigned int> > const&) const+0xa5) [0x80cdc5]
6: (OSDMap::_pg_to_osds(pg_pool_t const&, pg_t, std::vector<int, std::allocator<int> >*, int*, unsigned int*) const+0x96) [0x836c06]
7: (OSDMap::_pg_to_up_acting_osds(pg_t const&, std::vector<int, std::allocator<int> >*, int*, std::vector<int, std::allocator<int> >*, int*) const+0x124) [0x837044]
8: (PGMonitor::map_pg_creates()+0x2d4) [0x65c154]
9: (PGMonitor::post_paxos_update()+0x2d) [0x65cd7d]
10: (Monitor::refresh_from_paxos(bool*)+0x171) [0x583431]
11: (Monitor::init_paxos()+0xd5) [0x5836d5]
12: (Monitor::preinit()+0xaf1) [0x59a411]
13: (main()+0x39ba) [0x55519a]
14: (__libc_start_main()+0xfd) [0x3d02a1ed5d]
15: /usr/bin/ceph-mon() [0x54eb49]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events --- [92/5813]
-40> 2015-05-29 16:27:29.918865 7ffff7d7a7a0 5 asok(0x36a8000) register_command perfcounters_dump hook 0x3600050
-39> 2015-05-29 16:27:29.918906 7ffff7d7a7a0 5 asok(0x36a8000) register_command 1 hook 0x3600050
-38> 2015-05-29 16:27:29.918914 7ffff7d7a7a0 5 asok(0x36a8000) register_command perf dump hook 0x3600050
-37> 2015-05-29 16:27:29.918921 7ffff7d7a7a0 5 asok(0x36a8000) register_command perfcounters_schema hook 0x3600050
-36> 2015-05-29 16:27:29.918925 7ffff7d7a7a0 5 asok(0x36a8000) register_command 2 hook 0x3600050
-35> 2015-05-29 16:27:29.918929 7ffff7d7a7a0 5 asok(0x36a8000) register_command perf schema hook 0x3600050
-34> 2015-05-29 16:27:29.918932 7ffff7d7a7a0 5 asok(0x36a8000) register_command perf reset hook 0x3600050
-33> 2015-05-29 16:27:29.918936 7ffff7d7a7a0 5 asok(0x36a8000) register_command config show hook 0x3600050
-32> 2015-05-29 16:27:29.918941 7ffff7d7a7a0 5 asok(0x36a8000) register_command config set hook 0x3600050
-31> 2015-05-29 16:27:29.918945 7ffff7d7a7a0 5 asok(0x36a8000) register_command config get hook 0x3600050
-30> 2015-05-29 16:27:29.918948 7ffff7d7a7a0 5 asok(0x36a8000) register_command config diff hook 0x3600050
-29> 2015-05-29 16:27:29.918952 7ffff7d7a7a0 5 asok(0x36a8000) register_command log flush hook 0x3600050
-28> 2015-05-29 16:27:29.918956 7ffff7d7a7a0 5 asok(0x36a8000) register_command log dump hook 0x3600050
-27> 2015-05-29 16:27:29.918960 7ffff7d7a7a0 5 asok(0x36a8000) register_command log reopen hook 0x3600050
-26> 2015-05-29 16:27:29.922039 7ffff7d7a7a0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 3848139
-25> 2015-05-29 16:27:29.965524 7ffff7d7a7a0 5 asok(0x36a8000) init /var/run/ceph/ceph-mon.2.asok
-24> 2015-05-29 16:27:29.965544 7ffff7d7a7a0 5 asok(0x36a8000) bind_and_listen /var/run/ceph/ceph-mon.2.asok
-23> 2015-05-29 16:27:29.966418 7ffff7d7a7a0 5 asok(0x36a8000) register_command 0 hook 0x35f80c0
-22> 2015-05-29 16:27:29.966427 7ffff7d7a7a0 5 asok(0x36a8000) register_command version hook 0x35f80c0
-21> 2015-05-29 16:27:29.966430 7ffff7d7a7a0 5 asok(0x36a8000) register_command git_version hook 0x35f80c0
-20> 2015-05-29 16:27:29.966432 7ffff7d7a7a0 5 asok(0x36a8000) register_command help hook 0x36000b0
-19> 2015-05-29 16:27:29.966435 7ffff7d7a7a0 5 asok(0x36a8000) register_command get_command_descriptions hook 0x3600150
-18> 2015-05-29 16:27:29.967371 7ffff621c700 5 asok(0x36a8000) entry start
-17> 2015-05-29 16:27:30.074809 7ffff7d7a7a0 0 starting mon.2 rank 2 at 128.142.39.144:6790/0 mon_data /var/lib/ceph/mon/mon.2 fsid dd535a7e-4647-4bee-853d-f34112615f81
-16> 2015-05-29 16:27:30.075331 7ffff7d7a7a0 1 -- 128.142.39.144:6790/0 learned my addr 128.142.39.144:6790/0
-15> 2015-05-29 16:27:30.075343 7ffff7d7a7a0 1 accepter.accepter.bind my_inst.addr is 128.142.39.144:6790/0 need_addr=0
-14> 2015-05-29 16:27:30.075388 7ffff7d7a7a0 5 adding auth protocol: cephx
-13> 2015-05-29 16:27:30.075391 7ffff7d7a7a0 5 adding auth protocol: cephx
-12> 2015-05-29 16:27:30.075408 7ffff7d7a7a0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info)
-11> 2015-05-29 16:27:30.075411 7ffff7d7a7a0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info)
-10> 2015-05-29 16:27:30.075495 7ffff7d7a7a0 1 mon.2@-1(probing) e21 preinit fsid dd535a7e-4647-4bee-853d-f34112615f81
-9> 2015-05-29 16:27:30.075828 7ffff7d7a7a0 1 mon.2@-1(probing).paxosservice(pgmap 9708174..9708823) refresh upgraded, format 0 -> 1
-8> 2015-05-29 16:27:30.075838 7ffff7d7a7a0 1 mon.2@-1(probing).pg v0 on_upgrade discarding in-core PGMap
-7> 2015-05-29 16:27:30.223717 7ffff7d7a7a0 4 mon.2@-1(probing).mds e118 new map
-6> 2015-05-29 16:27:30.223730 7ffff7d7a7a0 0 mon.2@-1(probing).mds e118 print_map

epoch 118 [56/5813]
flags 0
created 2015-03-09 17:12:53.736792
modified 2015-05-08 11:13:49.983466
tableserver 0
root 0
session_timeout 60
session_autoclose 300
max_file_size 1099511627776
last_failure 0
last_failure_osd_epoch 91271
compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
max_mds 1
in 0
up {0=76159330}
failed
stopped
data_pools 0
metadata_pool 1
inline_data disabled
76159330: 188.184.164.127:6800/23339 '0' mds.0.5 up:active seq 5

-5> 2015-05-29 16:27:30.225760 7ffff7d7a7a0 0 mon.2@-1(probing).osd e91275 crush map has features 2543728197632, adjusting msgr requires
-4> 2015-05-29 16:27:30.225773 7ffff7d7a7a0 0 mon.2@-1(probing).osd e91275 crush map has features 2543728197632, adjusting msgr requires
-3> 2015-05-29 16:27:30.225779 7ffff7d7a7a0 0 mon.2@-1(probing).osd e91275 crush map has features 20135914242048, adjusting msgr requires
-2> 2015-05-29 16:27:30.225783 7ffff7d7a7a0 0 mon.2@-1(probing).osd e91275 crush map has features 2268850290688, adjusting msgr requires
-1> 2015-05-29 16:27:30.227707 7ffff7d7a7a0 1 mon.2@-1(probing).paxosservice(auth 508701..508827) refresh upgraded, format 0 -> 1
0> 2015-05-29 16:27:43.767499 7ffff7d7a7a0 -1 *** Caught signal (Segmentation fault) **
in thread 7ffff7d7a7a0

ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff)
1: /usr/bin/ceph-mon() [0x8fc1f5]
2: /lib64/libpthread.so.0() [0x3d02e0f710]
3: /usr/bin/ceph-mon() [0x79e48a]
4: (crush_do_rule()+0x4d1) [0x79ec61]
5: (CrushWrapper::do_rule(int, int, std::vector<int, std::allocator<int> >&, int, std::vector<unsigned int, std::allocator<unsigned int> > const&) const+0xa5) [0x80cdc5]
6: (OSDMap::_pg_to_osds(pg_pool_t const&, pg_t, std::vector<int, std::allocator<int> >*, int*, unsigned int*) const+0x96) [0x836c06]
7: (OSDMap::_pg_to_up_acting_osds(pg_t const&, std::vector<int, std::allocator<int> >*, int*, std::vector<int, std::allocator<int> >*, int*) const+0x124) [0x837044]
8: (PGMonitor::map_pg_creates()+0x2d4) [0x65c154]
9: (PGMonitor::post_paxos_update()+0x2d) [0x65cd7d]
10: (Monitor::refresh_from_paxos(bool*)+0x171) [0x583431]
11: (Monitor::init_paxos()+0xd5) [0x5836d5]
12: (Monitor::preinit()+0xaf1) [0x59a411]
13: (main()+0x39ba) [0x55519a]
14: (__libc_start_main()+0xfd) [0x3d02a1ed5d]
15: /usr/bin/ceph-mon() [0x54eb49]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

(2-2/2)