Project

General

Profile

Bug #22342 ยป ceph-mon_error.txt

Marcus Strasser, 12/07/2017 08:35 AM

 
2017-12-07 08:29:07.902151 7fc965f96880 0 set uid:gid to 64045:64045
2017-12-07 08:29:07.911378 7fc965f96880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 493
2017-12-07 08:29:54.794338 7fc965f96880 -1 *** Caught signal (Aborted) **
in thread 7fc965f96880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7fc966699755]
2: (()+0xf8d0) [0x7fc96502a8d0]
3: (gsignal()+0x37) [0x7fc9634ce107]
4: (abort()+0x148) [0x7fc9634cf4e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7fc963db9b3d]
6: (()+0x5ebb6) [0x7fc963db7bb6]
7: (()+0x5ec01) [0x7fc963db7c01]
8: (()+0x5ee19) [0x7fc963db7e19]
9: (()+0x4a9f6) [0x7fc965b109f6]
10: (tc_new()+0x1d) [0x7fc965b428b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7fc963e17e99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7fc963e17f14]

--- begin dump of recent events ---
-24> 2017-12-07 08:29:07.891454 7fc965f96880 5 asok(0x7fc969cb2000) register_command perfcounters_dump hook 0x7fc969c16030
-23> 2017-12-07 08:29:07.892716 7fc965f96880 5 asok(0x7fc969cb2000) register_command 1 hook 0x7fc969c16030
-22> 2017-12-07 08:29:07.892774 7fc965f96880 5 asok(0x7fc969cb2000) register_command perf dump hook 0x7fc969c16030
-21> 2017-12-07 08:29:07.892789 7fc965f96880 5 asok(0x7fc969cb2000) register_command perfcounters_schema hook 0x7fc969c16030
-20> 2017-12-07 08:29:07.892792 7fc965f96880 5 asok(0x7fc969cb2000) register_command 2 hook 0x7fc969c16030
-19> 2017-12-07 08:29:07.892796 7fc965f96880 5 asok(0x7fc969cb2000) register_command perf schema hook 0x7fc969c16030
-18> 2017-12-07 08:29:07.892801 7fc965f96880 5 asok(0x7fc969cb2000) register_command perf reset hook 0x7fc969c16030
-17> 2017-12-07 08:29:07.892808 7fc965f96880 5 asok(0x7fc969cb2000) register_command config show hook 0x7fc969c16030
-16> 2017-12-07 08:29:07.892819 7fc965f96880 5 asok(0x7fc969cb2000) register_command config set hook 0x7fc969c16030
-15> 2017-12-07 08:29:07.892828 7fc965f96880 5 asok(0x7fc969cb2000) register_command config get hook 0x7fc969c16030
-14> 2017-12-07 08:29:07.892837 7fc965f96880 5 asok(0x7fc969cb2000) register_command config diff hook 0x7fc969c16030
-13> 2017-12-07 08:29:07.892846 7fc965f96880 5 asok(0x7fc969cb2000) register_command log flush hook 0x7fc969c16030
-12> 2017-12-07 08:29:07.892853 7fc965f96880 5 asok(0x7fc969cb2000) register_command log dump hook 0x7fc969c16030
-11> 2017-12-07 08:29:07.892857 7fc965f96880 5 asok(0x7fc969cb2000) register_command log reopen hook 0x7fc969c16030
-10> 2017-12-07 08:29:07.902151 7fc965f96880 0 set uid:gid to 64045:64045
-9> 2017-12-07 08:29:07.911378 7fc965f96880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 493
-8> 2017-12-07 08:29:07.994250 7fc965f96880 5 asok(0x7fc969cb2000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 08:29:07.994300 7fc965f96880 5 asok(0x7fc969cb2000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 08:29:07.994364 7fc965f96880 5 asok(0x7fc969cb2000) register_command 0 hook 0x7fc969c120a8
-5> 2017-12-07 08:29:07.994374 7fc965f96880 5 asok(0x7fc969cb2000) register_command version hook 0x7fc969c120a8
-4> 2017-12-07 08:29:07.994378 7fc965f96880 5 asok(0x7fc969cb2000) register_command git_version hook 0x7fc969c120a8
-3> 2017-12-07 08:29:07.994384 7fc965f96880 5 asok(0x7fc969cb2000) register_command help hook 0x7fc969c161d0
-2> 2017-12-07 08:29:07.994388 7fc965f96880 5 asok(0x7fc969cb2000) register_command get_command_descriptions hook 0x7fc969c161e0
-1> 2017-12-07 08:29:07.994530 7fc960a5e700 5 asok(0x7fc969cb2000) entry start
0> 2017-12-07 08:29:54.794338 7fc965f96880 -1 *** Caught signal (Aborted) **
in thread 7fc965f96880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7fc966699755]
2: (()+0xf8d0) [0x7fc96502a8d0]
3: (gsignal()+0x37) [0x7fc9634ce107]
4: (abort()+0x148) [0x7fc9634cf4e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7fc963db9b3d]
6: (()+0x5ebb6) [0x7fc963db7bb6]
7: (()+0x5ec01) [0x7fc963db7c01]
8: (()+0x5ee19) [0x7fc963db7e19]
9: (()+0x4a9f6) [0x7fc965b109f6]
10: (tc_new()+0x1d) [0x7fc965b428b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7fc963e17e99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7fc963e17f14]

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 08:32:04.973395 7f52066c8880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 928
2017-12-07 08:33:16.339580 7f4038c5f880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 971
2017-12-07 08:33:51.718280 7f4038c5f880 -1 *** Caught signal (Aborted) **
in thread 7f4038c5f880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f4039362755]
2: (()+0xf8d0) [0x7f4037cf38d0]
3: (gsignal()+0x37) [0x7f4036197107]
4: (abort()+0x148) [0x7f40361984e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f4036a82b3d]
6: (()+0x5ebb6) [0x7f4036a80bb6]
7: (()+0x5ec01) [0x7f4036a80c01]
8: (()+0x5ee19) [0x7f4036a80e19]
9: (()+0x4a9f6) [0x7f40387d99f6]
10: (tc_new()+0x1d) [0x7f403880b8b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7f4036ae0e99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7f4036ae0f14]
13: (std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long)+0x1e) [0x7f4036ae152e]
14: (()+0x2ac1c) [0x7f4038131c1c]
15: (leveldb::VersionEdit::DecodeFrom(leveldb::Slice const&)+0x4f8) [0x7f4038132a88]
16: (leveldb::VersionSet::Recover()+0x957) [0x7f4038136117]
17: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f4038124f0f]
18: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f403812580e]
19: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f40391d963b]
20: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f4038e7aa08]
21: (main()+0x80d) [0x7f4038e3887d]
22: (__libc_start_main()+0xf5) [0x7f4036183b45]
23: (()+0x1ffa17) [0x7f4038e76a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-24> 2017-12-07 08:33:16.321584 7f4038c5f880 5 asok(0x7f403d89a000) register_command perfcounters_dump hook 0x7f403d7fe030
-23> 2017-12-07 08:33:16.322284 7f4038c5f880 5 asok(0x7f403d89a000) register_command 1 hook 0x7f403d7fe030
-22> 2017-12-07 08:33:16.322338 7f4038c5f880 5 asok(0x7f403d89a000) register_command perf dump hook 0x7f403d7fe030
-21> 2017-12-07 08:33:16.322350 7f4038c5f880 5 asok(0x7f403d89a000) register_command perfcounters_schema hook 0x7f403d7fe030
-20> 2017-12-07 08:33:16.322353 7f4038c5f880 5 asok(0x7f403d89a000) register_command 2 hook 0x7f403d7fe030
-19> 2017-12-07 08:33:16.322357 7f4038c5f880 5 asok(0x7f403d89a000) register_command perf schema hook 0x7f403d7fe030
-18> 2017-12-07 08:33:16.322364 7f4038c5f880 5 asok(0x7f403d89a000) register_command perf reset hook 0x7f403d7fe030
-17> 2017-12-07 08:33:16.322368 7f4038c5f880 5 asok(0x7f403d89a000) register_command config show hook 0x7f403d7fe030
-16> 2017-12-07 08:33:16.322372 7f4038c5f880 5 asok(0x7f403d89a000) register_command config set hook 0x7f403d7fe030
-15> 2017-12-07 08:33:16.322378 7f4038c5f880 5 asok(0x7f403d89a000) register_command config get hook 0x7f403d7fe030
-14> 2017-12-07 08:33:16.322381 7f4038c5f880 5 asok(0x7f403d89a000) register_command config diff hook 0x7f403d7fe030
-13> 2017-12-07 08:33:16.322385 7f4038c5f880 5 asok(0x7f403d89a000) register_command log flush hook 0x7f403d7fe030
-12> 2017-12-07 08:33:16.322391 7f4038c5f880 5 asok(0x7f403d89a000) register_command log dump hook 0x7f403d7fe030
-11> 2017-12-07 08:33:16.322394 7f4038c5f880 5 asok(0x7f403d89a000) register_command log reopen hook 0x7f403d7fe030
-10> 2017-12-07 08:33:16.339413 7f4038c5f880 0 set uid:gid to 64045:64045
-9> 2017-12-07 08:33:16.339580 7f4038c5f880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 971
-8> 2017-12-07 08:33:16.367578 7f4038c5f880 5 asok(0x7f403d89a000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 08:33:16.367603 7f4038c5f880 5 asok(0x7f403d89a000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 08:33:16.367732 7f4038c5f880 5 asok(0x7f403d89a000) register_command 0 hook 0x7f403d7fa0a8
-5> 2017-12-07 08:33:16.367750 7f4038c5f880 5 asok(0x7f403d89a000) register_command version hook 0x7f403d7fa0a8
-4> 2017-12-07 08:33:16.367754 7f4038c5f880 5 asok(0x7f403d89a000) register_command git_version hook 0x7f403d7fa0a8
-3> 2017-12-07 08:33:16.367760 7f4038c5f880 5 asok(0x7f403d89a000) register_command help hook 0x7f403d7fe1d0
-2> 2017-12-07 08:33:16.367767 7f4038c5f880 5 asok(0x7f403d89a000) register_command get_command_descriptions hook 0x7f403d7fe1e0
-1> 2017-12-07 08:33:16.367833 7f4033727700 5 asok(0x7f403d89a000) entry start
0> 2017-12-07 08:33:51.718280 7f4038c5f880 -1 *** Caught signal (Aborted) **
in thread 7f4038c5f880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f4039362755]
2: (()+0xf8d0) [0x7f4037cf38d0]
3: (gsignal()+0x37) [0x7f4036197107]
4: (abort()+0x148) [0x7f40361984e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f4036a82b3d]
6: (()+0x5ebb6) [0x7f4036a80bb6]
7: (()+0x5ec01) [0x7f4036a80c01]
8: (()+0x5ee19) [0x7f4036a80e19]
9: (()+0x4a9f6) [0x7f40387d99f6]
10: (tc_new()+0x1d) [0x7f403880b8b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7f4036ae0e99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7f4036ae0f14]
13: (std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long)+0x1e) [0x7f4036ae152e]
14: (()+0x2ac1c) [0x7f4038131c1c]
15: (leveldb::VersionEdit::DecodeFrom(leveldb::Slice const&)+0x4f8) [0x7f4038132a88]
16: (leveldb::VersionSet::Recover()+0x957) [0x7f4038136117]
17: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f4038124f0f]
18: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f403812580e]
19: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f40391d963b]
20: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f4038e7aa08]
21: (main()+0x80d) [0x7f4038e3887d]
22: (__libc_start_main()+0xf5) [0x7f4036183b45]
23: (()+0x1ffa17) [0x7f4038e76a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 08:36:06.126924 7f1c1cdbf880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 1113
2017-12-07 08:36:41.404629 7f1c1cdbf880 -1 *** Caught signal (Aborted) **
in thread 7f1c1cdbf880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-24> 2017-12-07 08:36:06.117997 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command perfcounters_dump hook 0x7f1c2176e030
-23> 2017-12-07 08:36:06.118045 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command 1 hook 0x7f1c2176e030
-22> 2017-12-07 08:36:06.118091 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command perf dump hook 0x7f1c2176e030
-21> 2017-12-07 08:36:06.118103 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command perfcounters_schema hook 0x7f1c2176e030
-20> 2017-12-07 08:36:06.118106 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command 2 hook 0x7f1c2176e030
-19> 2017-12-07 08:36:06.118116 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command perf schema hook 0x7f1c2176e030
-18> 2017-12-07 08:36:06.118119 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command perf reset hook 0x7f1c2176e030
-17> 2017-12-07 08:36:06.118125 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command config show hook 0x7f1c2176e030
-16> 2017-12-07 08:36:06.118135 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command config set hook 0x7f1c2176e030
-15> 2017-12-07 08:36:06.118139 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command config get hook 0x7f1c2176e030
-14> 2017-12-07 08:36:06.118148 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command config diff hook 0x7f1c2176e030
-13> 2017-12-07 08:36:06.118151 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command log flush hook 0x7f1c2176e030
-12> 2017-12-07 08:36:06.118155 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command log dump hook 0x7f1c2176e030
-11> 2017-12-07 08:36:06.118164 7f1c1cdbf880 5 asok(0x7f1c2180a000) register_command log reopen hook 0x7f1c2176e030
2017-12-07 08:37:28.662698 7f1787502880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 497
2017-12-07 08:43:34.950606 7f897f382880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 495
2017-12-07 08:53:46.973018 7f40a85c9880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 1475
2017-12-07 08:54:27.948065 7f40a85c9880 -1 *** Caught signal (Aborted) **
in thread 7f40a85c9880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f40a8ccc755]
2: (()+0xf8d0) [0x7f40a765d8d0]
3: (gsignal()+0x37) [0x7f40a5b01107]
4: (abort()+0x148) [0x7f40a5b024e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f40a63ecb3d]
6: (()+0x5ebb6) [0x7f40a63eabb6]
7: (()+0x5ec01) [0x7f40a63eac01]
8: (()+0x5ee19) [0x7f40a63eae19]
9: (()+0x4a9f6) [0x7f40a81439f6]
10: (tc_new()+0x1d) [0x7f40a81758b5]
11: (std::pair<std::_Rb_tree_iterator<unsigned long>, bool> std::_Rb_tree<unsigned long, unsigned long, std::_Identity<unsigned long>, std::less<unsigned long>, std::allocator<unsigned long> >::_M_insert_unique<unsigned long const&>(unsigned long const&)+0xc0) [0x7f40a8836d80]
12: (leveldb::VersionSet::Recover()+0xd59) [0x7f40a7aa0519]
13: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f40a7a8ef0f]
14: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f40a7a8f80e]
15: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f40a8b4363b]
16: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f40a87e4a08]
17: (main()+0x80d) [0x7f40a87a287d]
18: (__libc_start_main()+0xf5) [0x7f40a5aedb45]
19: (()+0x1ffa17) [0x7f40a87e0a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is nee
--- begin dump of recent events ---
-24> 2017-12-07 08:53:46.949480 7f40a85c9880 5 asok(0x7f40ab9da000) register_command perfcounters_dump hook 0x7f40ab93e030
-23> 2017-12-07 08:53:46.950148 7f40a85c9880 5 asok(0x7f40ab9da000) register_command 1 hook 0x7f40ab93e030
-22> 2017-12-07 08:53:46.950182 7f40a85c9880 5 asok(0x7f40ab9da000) register_command perf dump hook 0x7f40ab93e030
-21> 2017-12-07 08:53:46.950226 7f40a85c9880 5 asok(0x7f40ab9da000) register_command perfcounters_schema hook 0x7f40ab93e030
-20> 2017-12-07 08:53:46.950238 7f40a85c9880 5 asok(0x7f40ab9da000) register_command 2 hook 0x7f40ab93e030
-19> 2017-12-07 08:53:46.950244 7f40a85c9880 5 asok(0x7f40ab9da000) register_command perf schema hook 0x7f40ab93e030
-18> 2017-12-07 08:53:46.950247 7f40a85c9880 5 asok(0x7f40ab9da000) register_command perf reset hook 0x7f40ab93e030
-17> 2017-12-07 08:53:46.950253 7f40a85c9880 5 asok(0x7f40ab9da000) register_command config show hook 0x7f40ab93e030
-16> 2017-12-07 08:53:46.950258 7f40a85c9880 5 asok(0x7f40ab9da000) register_command config set hook 0x7f40ab93e030
-15> 2017-12-07 08:53:46.950264 7f40a85c9880 5 asok(0x7f40ab9da000) register_command config get hook 0x7f40ab93e030
-14> 2017-12-07 08:53:46.950267 7f40a85c9880 5 asok(0x7f40ab9da000) register_command config diff hook 0x7f40ab93e030
-13> 2017-12-07 08:53:46.950272 7f40a85c9880 5 asok(0x7f40ab9da000) register_command log flush hook 0x7f40ab93e030
-12> 2017-12-07 08:53:46.950278 7f40a85c9880 5 asok(0x7f40ab9da000) register_command log dump hook 0x7f40ab93e030
-11> 2017-12-07 08:53:46.950281 7f40a85c9880 5 asok(0x7f40ab9da000) register_command log reopen hook 0x7f40ab93e030
-10> 2017-12-07 08:53:46.972813 7f40a85c9880 0 set uid:gid to 64045:64045
-9> 2017-12-07 08:53:46.973018 7f40a85c9880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 1475
-8> 2017-12-07 08:53:46.998105 7f40a85c9880 5 asok(0x7f40ab9da000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 08:53:46.998137 7f40a85c9880 5 asok(0x7f40ab9da000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 08:53:46.998353 7f40a85c9880 5 asok(0x7f40ab9da000) register_command 0 hook 0x7f40ab93a0a8
-5> 2017-12-07 08:53:46.998430 7f40a85c9880 5 asok(0x7f40ab9da000) register_command version hook 0x7f40ab93a0a8
-4> 2017-12-07 08:53:46.998445 7f40a85c9880 5 asok(0x7f40ab9da000) register_command git_version hook 0x7f40ab93a0a8
-3> 2017-12-07 08:53:46.998458 7f40a85c9880 5 asok(0x7f40ab9da000) register_command help hook 0x7f40ab93e1d0
-2> 2017-12-07 08:53:46.998465 7f40a85c9880 5 asok(0x7f40ab9da000) register_command get_command_descriptions hook 0x7f40ab93e1e0
-1> 2017-12-07 08:53:46.998577 7f40a3091700 5 asok(0x7f40ab9da000) entry start
0> 2017-12-07 08:54:27.948065 7f40a85c9880 -1 *** Caught signal (Aborted) **
in thread 7f40a85c9880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f40a8ccc755]
2: (()+0xf8d0) [0x7f40a765d8d0]
3: (gsignal()+0x37) [0x7f40a5b01107]
4: (abort()+0x148) [0x7f40a5b024e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f40a63ecb3d]
6: (()+0x5ebb6) [0x7f40a63eabb6]
7: (()+0x5ec01) [0x7f40a63eac01]
8: (()+0x5ee19) [0x7f40a63eae19]
9: (()+0x4a9f6) [0x7f40a81439f6]
10: (tc_new()+0x1d) [0x7f40a81758b5]
11: (std::pair<std::_Rb_tree_iterator<unsigned long>, bool> std::_Rb_tree<unsigned long, unsigned long, std::_Identity<unsigned long>, std::less<unsigned long>, std::allocator<unsigned long> >::_M_insert_unique<unsigned long const&>(unsigned long const&)+0xc0) [0x7f40a8836d80]
12: (leveldb::VersionSet::Recover()+0xd59) [0x7f40a7aa0519]
13: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f40a7a8ef0f]
14: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f40a7a8f80e]
15: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f40a8b4363b]
16: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f40a87e4a08]
17: (main()+0x80d) [0x7f40a87a287d]
18: (__libc_start_main()+0xf5) [0x7f40a5aedb45]
19: (()+0x1ffa17) [0x7f40a87e0a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is nee
--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 08:56:30.467187 7fb2a3182880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 494
2017-12-07 08:57:18.972368 7fb2a3182880 -1 *** Caught signal (Aborted) **
in thread 7fb2a3182880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7fb2a3885755]
2: (()+0xf8d0) [0x7fb2a22168d0]
3: (gsignal()+0x37) [0x7fb2a06ba107]
4: (abort()+0x148) [0x7fb2a06bb4e8]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-24> 2017-12-07 08:56:30.446476 7fb2a3182880 5 asok(0x7fb2a77da000) register_command perfcounters_dump hook 0x7fb2a773e030
-23> 2017-12-07 08:56:30.447102 7fb2a3182880 5 asok(0x7fb2a77da000) register_command 1 hook 0x7fb2a773e030
-22> 2017-12-07 08:56:30.447115 7fb2a3182880 5 asok(0x7fb2a77da000) register_command perf dump hook 0x7fb2a773e030
-21> 2017-12-07 08:56:30.447119 7fb2a3182880 5 asok(0x7fb2a77da000) register_command perfcounters_schema hook 0x7fb2a773e030
-20> 2017-12-07 08:56:30.447122 7fb2a3182880 5 asok(0x7fb2a77da000) register_command 2 hook 0x7fb2a773e030
-19> 2017-12-07 08:56:30.447126 7fb2a3182880 5 asok(0x7fb2a77da000) register_command perf schema hook 0x7fb2a773e030
-18> 2017-12-07 08:56:30.447129 7fb2a3182880 5 asok(0x7fb2a77da000) register_command perf reset hook 0x7fb2a773e030
-17> 2017-12-07 08:56:30.447132 7fb2a3182880 5 asok(0x7fb2a77da000) register_command config show hook 0x7fb2a773e030
-16> 2017-12-07 08:56:30.447136 7fb2a3182880 5 asok(0x7fb2a77da000) register_command config set hook 0x7fb2a773e030
-15> 2017-12-07 08:56:30.447139 7fb2a3182880 5 asok(0x7fb2a77da000) register_command config get hook 0x7fb2a773e030
-14> 2017-12-07 08:56:30.447142 7fb2a3182880 5 asok(0x7fb2a77da000) register_command config diff hook 0x7fb2a773e030
-13> 2017-12-07 08:56:30.447145 7fb2a3182880 5 asok(0x7fb2a77da000) register_command log flush hook 0x7fb2a773e030
-12> 2017-12-07 08:56:30.447148 7fb2a3182880 5 asok(0x7fb2a77da000) register_command log dump hook 0x7fb2a773e030
-11> 2017-12-07 08:56:30.447151 7fb2a3182880 5 asok(0x7fb2a77da000) register_command log reopen hook 0x7fb2a773e030
-10> 2017-12-07 08:56:30.456164 7fb2a3182880 0 set uid:gid to 64045:64045
-9> 2017-12-07 08:56:30.467187 7fb2a3182880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 494
-8> 2017-12-07 08:56:30.591225 7fb2a3182880 5 asok(0x7fb2a77da000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 08:56:30.591250 7fb2a3182880 5 asok(0x7fb2a77da000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 08:56:30.591294 7fb2a3182880 5 asok(0x7fb2a77da000) register_command 0 hook 0x7fb2a773a0a8
-5> 2017-12-07 08:56:30.591309 7fb2a3182880 5 asok(0x7fb2a77da000) register_command version hook 0x7fb2a773a0a8
-4> 2017-12-07 08:56:30.591314 7fb2a3182880 5 asok(0x7fb2a77da000) register_command git_version hook 0x7fb2a773a0a8
-3> 2017-12-07 08:56:30.591322 7fb2a3182880 5 asok(0x7fb2a77da000) register_command help hook 0x7fb2a773e1d0
-2> 2017-12-07 08:56:30.591328 7fb2a3182880 5 asok(0x7fb2a77da000) register_command get_command_descriptions hook 0x7fb2a773e1e0
-1> 2017-12-07 08:56:30.591401 7fb29dc4a700 5 asok(0x7fb2a77da000) entry start
0> 2017-12-07 08:57:18.972368 7fb2a3182880 -1 *** Caught signal (Aborted) **
in thread 7fb2a3182880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7fb2a3885755]
2: (()+0xf8d0) [0x7fb2a22168d0]
3: (gsignal()+0x37) [0x7fb2a06ba107]
4: (abort()+0x148) [0x7fb2a06bb4e8]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 08:57:35.360532 7f6f1826a880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 902
2017-12-07 08:58:14.433259 7f6f1826a880 -1 *** Caught signal (Aborted) **
in thread 7f6f1826a880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f6f1896d755]
2: (()+0xf8d0) [0x7f6f172fe8d0]
3: (gsignal()+0x37) [0x7f6f157a2107]
4: (abort()+0x148) [0x7f6f157a34e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f6f1608db3d]
6: (()+0x5ebb6) [0x7f6f1608bbb6]
7: (()+0x5ec01) [0x7f6f1608bc01]
8: (()+0x5ee19) [0x7f6f1608be19]
9: (()+0x4a9f6) [0x7f6f17de49f6]
10: (tc_new()+0x1d) [0x7f6f17e168b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7f6f160ebe99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7f6f160ebf14]
13: (std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long)+0x1e) [0x7f6f160ec52e]
14: (()+0x2ac1c) [0x7f6f1773cc1c]
15: (leveldb::VersionEdit::DecodeFrom(leveldb::Slice const&)+0x4e4) [0x7f6f1773da74]
16: (leveldb::VersionSet::Recover()+0x957) [0x7f6f17741117]
17: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f6f1772ff0f]
18: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f6f1773080e]
19: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f6f187e463b]
20: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f6f18485a08]
21: (main()+0x80d) [0x7f6f1844387d]
22: (__libc_start_main()+0xf5) [0x7f6f1578eb45]
23: (()+0x1ffa17) [0x7f6f18481a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-24> 2017-12-07 08:57:35.345655 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command perfcounters_dump hook 0x7f6f1b8de030
-23> 2017-12-07 08:57:35.346203 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command 1 hook 0x7f6f1b8de030
-22> 2017-12-07 08:57:35.346258 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command perf dump hook 0x7f6f1b8de030
-21> 2017-12-07 08:57:35.346272 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command perfcounters_schema hook 0x7f6f1b8de030
-20> 2017-12-07 08:57:35.346276 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command 2 hook 0x7f6f1b8de030
-19> 2017-12-07 08:57:35.346280 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command perf schema hook 0x7f6f1b8de030
-18> 2017-12-07 08:57:35.346283 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command perf reset hook 0x7f6f1b8de030
-17> 2017-12-07 08:57:35.346286 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command config show hook 0x7f6f1b8de030
-16> 2017-12-07 08:57:35.346294 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command config set hook 0x7f6f1b8de030
-15> 2017-12-07 08:57:35.346305 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command config get hook 0x7f6f1b8de030
-14> 2017-12-07 08:57:35.346308 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command config diff hook 0x7f6f1b8de030
-13> 2017-12-07 08:57:35.346311 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command log flush hook 0x7f6f1b8de030
-12> 2017-12-07 08:57:35.346314 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command log dump hook 0x7f6f1b8de030
-11> 2017-12-07 08:57:35.346318 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command log reopen hook 0x7f6f1b8de030
-10> 2017-12-07 08:57:35.360399 7f6f1826a880 0 set uid:gid to 64045:64045
-9> 2017-12-07 08:57:35.360532 7f6f1826a880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 902
-8> 2017-12-07 08:57:35.552319 7f6f1826a880 5 asok(0x7f6f1b97a000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 08:57:35.552355 7f6f1826a880 5 asok(0x7f6f1b97a000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 08:57:35.552506 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command 0 hook 0x7f6f1b8da0a8
-5> 2017-12-07 08:57:35.552525 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command version hook 0x7f6f1b8da0a8
-4> 2017-12-07 08:57:35.552530 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command git_version hook 0x7f6f1b8da0a8
-3> 2017-12-07 08:57:35.552536 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command help hook 0x7f6f1b8de1d0
-2> 2017-12-07 08:57:35.552540 7f6f1826a880 5 asok(0x7f6f1b97a000) register_command get_command_descriptions hook 0x7f6f1b8de1e0
-1> 2017-12-07 08:57:35.552618 7f6f12d32700 5 asok(0x7f6f1b97a000) entry start
0> 2017-12-07 08:58:14.433259 7f6f1826a880 -1 *** Caught signal (Aborted) **
in thread 7f6f1826a880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f6f1896d755]
2: (()+0xf8d0) [0x7f6f172fe8d0]
3: (gsignal()+0x37) [0x7f6f157a2107]
4: (abort()+0x148) [0x7f6f157a34e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f6f1608db3d]
6: (()+0x5ebb6) [0x7f6f1608bbb6]
7: (()+0x5ec01) [0x7f6f1608bc01]
8: (()+0x5ee19) [0x7f6f1608be19]
9: (()+0x4a9f6) [0x7f6f17de49f6]
10: (tc_new()+0x1d) [0x7f6f17e168b5]
11: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x7f6f160ebe99]
12: (std::string::_M_mutate(unsigned long, unsigned long, unsigned long)+0x54) [0x7f6f160ebf14]
13: (std::string::_M_replace_safe(unsigned long, unsigned long, char const*, unsigned long)+0x1e) [0x7f6f160ec52e]
14: (()+0x2ac1c) [0x7f6f1773cc1c]
15: (leveldb::VersionEdit::DecodeFrom(leveldb::Slice const&)+0x4e4) [0x7f6f1773da74]
16: (leveldb::VersionSet::Recover()+0x957) [0x7f6f17741117]
17: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f6f1772ff0f]
18: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f6f1773080e]
19: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f6f187e463b]
20: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f6f18485a08]
21: (main()+0x80d) [0x7f6f1844387d]
22: (__libc_start_main()+0xf5) [0x7f6f1578eb45]
23: (()+0x1ffa17) [0x7f6f18481a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 09:09:14.604896 7f227767a880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 493
2017-12-07 09:10:15.612016 7f227767a880 -1 *** Caught signal (Aborted) **
in thread 7f227767a880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f2277d7d755]
2: (()+0xf8d0) [0x7f227670e8d0]
3: (gsignal()+0x37) [0x7f2274bb2107]
4: (abort()+0x148) [0x7f2274bb34e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f227549db3d]
6: (()+0x5ebb6) [0x7f227549bbb6]
7: (()+0x5ec01) [0x7f227549bc01]
8: (()+0x5ee19) [0x7f227549be19]
9: (()+0x4a9f6) [0x7f22771f49f6]
10: (tc_new()+0x1d) [0x7f22772268b5]
11: (leveldb::VersionSet::Recover()+0xdce) [0x7f2276b5158e]
12: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f2276b3ff0f]
13: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f2276b4080e]
14: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f2277bf463b]
15: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f2277895a08]
16: (main()+0x80d) [0x7f227785387d]
17: (__libc_start_main()+0xf5) [0x7f2274b9eb45]
18: (()+0x1ffa17) [0x7f2277891a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-24> 2017-12-07 09:09:14.594456 7f227767a880 5 asok(0x7f227adf0000) register_command perfcounters_dump hook 0x7f227ad54030
-23> 2017-12-07 09:09:14.595132 7f227767a880 5 asok(0x7f227adf0000) register_command 1 hook 0x7f227ad54030
-22> 2017-12-07 09:09:14.595148 7f227767a880 5 asok(0x7f227adf0000) register_command perf dump hook 0x7f227ad54030
-21> 2017-12-07 09:09:14.595152 7f227767a880 5 asok(0x7f227adf0000) register_command perfcounters_schema hook 0x7f227ad54030
-20> 2017-12-07 09:09:14.595155 7f227767a880 5 asok(0x7f227adf0000) register_command 2 hook 0x7f227ad54030
-19> 2017-12-07 09:09:14.595164 7f227767a880 5 asok(0x7f227adf0000) register_command perf schema hook 0x7f227ad54030
-18> 2017-12-07 09:09:14.595167 7f227767a880 5 asok(0x7f227adf0000) register_command perf reset hook 0x7f227ad54030
-17> 2017-12-07 09:09:14.595169 7f227767a880 5 asok(0x7f227adf0000) register_command config show hook 0x7f227ad54030
-16> 2017-12-07 09:09:14.595174 7f227767a880 5 asok(0x7f227adf0000) register_command config set hook 0x7f227ad54030
-15> 2017-12-07 09:09:14.595184 7f227767a880 5 asok(0x7f227adf0000) register_command config get hook 0x7f227ad54030
-14> 2017-12-07 09:09:14.595187 7f227767a880 5 asok(0x7f227adf0000) register_command config diff hook 0x7f227ad54030
-13> 2017-12-07 09:09:14.595189 7f227767a880 5 asok(0x7f227adf0000) register_command log flush hook 0x7f227ad54030
-12> 2017-12-07 09:09:14.595193 7f227767a880 5 asok(0x7f227adf0000) register_command log dump hook 0x7f227ad54030
-11> 2017-12-07 09:09:14.595196 7f227767a880 5 asok(0x7f227adf0000) register_command log reopen hook 0x7f227ad54030
-10> 2017-12-07 09:09:14.603965 7f227767a880 0 set uid:gid to 64045:64045
-9> 2017-12-07 09:09:14.604896 7f227767a880 0 ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299), process ceph-mon, pid 493
-8> 2017-12-07 09:09:14.839801 7f227767a880 5 asok(0x7f227adf0000) init /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-7> 2017-12-07 09:09:14.839821 7f227767a880 5 asok(0x7f227adf0000) bind_and_listen /var/run/ceph/ceph-mon.rmisp-cosd3.asok
-6> 2017-12-07 09:09:14.839849 7f227767a880 5 asok(0x7f227adf0000) register_command 0 hook 0x7f227ad500a8
-5> 2017-12-07 09:09:14.839862 7f227767a880 5 asok(0x7f227adf0000) register_command version hook 0x7f227ad500a8
-4> 2017-12-07 09:09:14.839869 7f227767a880 5 asok(0x7f227adf0000) register_command git_version hook 0x7f227ad500a8
-3> 2017-12-07 09:09:14.839878 7f227767a880 5 asok(0x7f227adf0000) register_command help hook 0x7f227ad541d0
-2> 2017-12-07 09:09:14.839886 7f227767a880 5 asok(0x7f227adf0000) register_command get_command_descriptions hook 0x7f227ad541e0
-1> 2017-12-07 09:09:14.839985 7f2272142700 5 asok(0x7f227adf0000) entry start
0> 2017-12-07 09:10:15.612016 7f227767a880 -1 *** Caught signal (Aborted) **
in thread 7f227767a880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f2277d7d755]
2: (()+0xf8d0) [0x7f227670e8d0]
3: (gsignal()+0x37) [0x7f2274bb2107]
4: (abort()+0x148) [0x7f2274bb34e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f227549db3d]
6: (()+0x5ebb6) [0x7f227549bbb6]
7: (()+0x5ec01) [0x7f227549bc01]
8: (()+0x5ee19) [0x7f227549be19]
9: (()+0x4a9f6) [0x7f22771f49f6]
10: (tc_new()+0x1d) [0x7f22772268b5]
11: (leveldb::VersionSet::Recover()+0xdce) [0x7f2276b5158e]
12: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f2276b3ff0f]
13: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f2276b4080e]
14: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f2277bf463b]
15: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f2277895a08]
16: (main()+0x80d) [0x7f227785387d]
17: (__libc_start_main()+0xf5) [0x7f2274b9eb45]
18: (()+0x1ffa17) [0x7f2277891a17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
2017-12-07 09:18:11.022194 7f1c45f24880 -1 *** Caught signal (Aborted) **
in thread 7f1c45f24880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f1c46627755]
2: (()+0xf8d0) [0x7f1c44fb88d0]
3: (gsignal()+0x37) [0x7f1c4345c107]
4: (abort()+0x148) [0x7f1c4345d4e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f1c43d47b3d]
6: (()+0x5ebb6) [0x7f1c43d45bb6]
7: (()+0x5ec01) [0x7f1c43d45c01]
8: (()+0x5ee19) [0x7f1c43d45e19]
9: (()+0x4a9f6) [0x7f1c45a9e9f6]
10: (tc_new()+0x1d) [0x7f1c45ad08b5]
11: (std::vector<leveldb::FileMetaData*, std::allocator<leveldb::FileMetaData*> >::reserve(unsigned long)+0x6f) [0x7f1c454005ff]
12: (leveldb::VersionSet::Builder::SaveTo(leveldb::Version*)+0x8b) [0x7f1c454006fb]
13: (leveldb::VersionSet::Recover()+0x1353) [0x7f1c453fbb13]
14: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f1c453e9f0f]
15: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f1c453ea80e]
16: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f1c4649e63b]
17: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f1c4613fa08]
18: (main()+0x80d) [0x7f1c460fd87d]
19: (__libc_start_main()+0xf5) [0x7f1c43448b45]
20: (()+0x1ffa17) [0x7f1c4613ba17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-14> 2017-12-07 09:17:26.707184 7f1c45f24880 5 asok(0x7f1c4b050000) register_command perfcounters_dump hook 0x7f1c4afb4030
-13> 2017-12-07 09:17:26.707687 7f1c45f24880 5 asok(0x7f1c4b050000) register_command 1 hook 0x7f1c4afb4030
-12> 2017-12-07 09:17:26.707704 7f1c45f24880 5 asok(0x7f1c4b050000) register_command perf dump hook 0x7f1c4afb4030
-11> 2017-12-07 09:17:26.707707 7f1c45f24880 5 asok(0x7f1c4b050000) register_command perfcounters_schema hook 0x7f1c4afb4030
-10> 2017-12-07 09:17:26.707710 7f1c45f24880 5 asok(0x7f1c4b050000) register_command 2 hook 0x7f1c4afb4030
-9> 2017-12-07 09:17:26.707715 7f1c45f24880 5 asok(0x7f1c4b050000) register_command perf schema hook 0x7f1c4afb4030
-8> 2017-12-07 09:17:26.707718 7f1c45f24880 5 asok(0x7f1c4b050000) register_command perf reset hook 0x7f1c4afb4030
-7> 2017-12-07 09:17:26.707721 7f1c45f24880 5 asok(0x7f1c4b050000) register_command config show hook 0x7f1c4afb4030
-6> 2017-12-07 09:17:26.707726 7f1c45f24880 5 asok(0x7f1c4b050000) register_command config set hook 0x7f1c4afb4030
-5> 2017-12-07 09:17:26.707730 7f1c45f24880 5 asok(0x7f1c4b050000) register_command config get hook 0x7f1c4afb4030
-4> 2017-12-07 09:17:26.707733 7f1c45f24880 5 asok(0x7f1c4b050000) register_command config diff hook 0x7f1c4afb4030
-3> 2017-12-07 09:17:26.707737 7f1c45f24880 5 asok(0x7f1c4b050000) register_command log flush hook 0x7f1c4afb4030
-2> 2017-12-07 09:17:26.707757 7f1c45f24880 5 asok(0x7f1c4b050000) register_command log dump hook 0x7f1c4afb4030
-1> 2017-12-07 09:17:26.707762 7f1c45f24880 5 asok(0x7f1c4b050000) register_command log reopen hook 0x7f1c4afb4030
0> 2017-12-07 09:18:11.022194 7f1c45f24880 -1 *** Caught signal (Aborted) **
in thread 7f1c45f24880

ceph version 9.2.0 (bb2ecea240f3a1d525bcb35670cb07bd1f0ca299)
1: (()+0x6eb755) [0x7f1c46627755]
2: (()+0xf8d0) [0x7f1c44fb88d0]
3: (gsignal()+0x37) [0x7f1c4345c107]
4: (abort()+0x148) [0x7f1c4345d4e8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x15d) [0x7f1c43d47b3d]
6: (()+0x5ebb6) [0x7f1c43d45bb6]
7: (()+0x5ec01) [0x7f1c43d45c01]
8: (()+0x5ee19) [0x7f1c43d45e19]
9: (()+0x4a9f6) [0x7f1c45a9e9f6]
10: (tc_new()+0x1d) [0x7f1c45ad08b5]
11: (std::vector<leveldb::FileMetaData*, std::allocator<leveldb::FileMetaData*> >::reserve(unsigned long)+0x6f) [0x7f1c454005ff]
12: (leveldb::VersionSet::Builder::SaveTo(leveldb::Version*)+0x8b) [0x7f1c454006fb]
13: (leveldb::VersionSet::Recover()+0x1353) [0x7f1c453fbb13]
14: (leveldb::DBImpl::Recover(leveldb::VersionEdit*)+0x19f) [0x7f1c453e9f0f]
15: (leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)+0xfe) [0x7f1c453ea80e]
16: (LevelDBStore::do_open(std::ostream&, bool)+0xfb) [0x7f1c4649e63b]
17: (MonitorDBStore::open(std::ostream&)+0xd8) [0x7f1c4613fa08]
18: (main()+0x80d) [0x7f1c460fd87d]
19: (__libc_start_main()+0xf5) [0x7f1c43448b45]
20: (()+0x1ffa17) [0x7f1c4613ba17]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 newstore
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.rmisp-cosd3.log
--- end dump of recent events ---
    (1-1/1)