2015-09-09 10:40:53.648579 7f45a36a58c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2331 2015-09-09 10:40:55.084393 7f45a36a58c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:40:55.111838 7f45a36a58c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:40:55.813563 7f45a36a58c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:40:55.856374 7f45a36a58c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:40:56.563736 7f45a36a58c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:40:59.419167 7f45a36a58c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f45a36a58c0 time 2015-09-09 10:40:59.405454 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f45a0c33ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -54> 2015-09-09 10:40:53.605674 7f45a36a58c0 5 asok(0x4560000) register_command perfcounters_dump hook 0x44e8050 -53> 2015-09-09 10:40:53.632877 7f45a36a58c0 5 asok(0x4560000) register_command 1 hook 0x44e8050 -52> 2015-09-09 10:40:53.632893 7f45a36a58c0 5 asok(0x4560000) register_command perf dump hook 0x44e8050 -51> 2015-09-09 10:40:53.632900 7f45a36a58c0 5 asok(0x4560000) register_command perfcounters_schema hook 0x44e8050 -50> 2015-09-09 10:40:53.632903 7f45a36a58c0 5 asok(0x4560000) register_command 2 hook 0x44e8050 -49> 2015-09-09 10:40:53.632905 7f45a36a58c0 5 asok(0x4560000) register_command perf schema hook 0x44e8050 -48> 2015-09-09 10:40:53.632907 7f45a36a58c0 5 asok(0x4560000) register_command perf reset hook 0x44e8050 -47> 2015-09-09 10:40:53.632911 7f45a36a58c0 5 asok(0x4560000) register_command config show hook 0x44e8050 -46> 2015-09-09 10:40:53.632917 7f45a36a58c0 5 asok(0x4560000) register_command config set hook 0x44e8050 -45> 2015-09-09 10:40:53.632919 7f45a36a58c0 5 asok(0x4560000) register_command config get hook 0x44e8050 -44> 2015-09-09 10:40:53.632923 7f45a36a58c0 5 asok(0x4560000) register_command config diff hook 0x44e8050 -43> 2015-09-09 10:40:53.632925 7f45a36a58c0 5 asok(0x4560000) register_command log flush hook 0x44e8050 -42> 2015-09-09 10:40:53.632929 7f45a36a58c0 5 asok(0x4560000) register_command log dump hook 0x44e8050 -41> 2015-09-09 10:40:53.632931 7f45a36a58c0 5 asok(0x4560000) register_command log reopen hook 0x44e8050 -40> 2015-09-09 10:40:53.648579 7f45a36a58c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2331 -39> 2015-09-09 10:40:53.649853 7f45a36a58c0 5 asok(0x4560000) init /var/run/ceph/ceph-mon.vm14.asok -38> 2015-09-09 10:40:53.649868 7f45a36a58c0 5 asok(0x4560000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -37> 2015-09-09 10:40:53.649899 7f45a36a58c0 5 asok(0x4560000) register_command 0 hook 0x44e40b8 -36> 2015-09-09 10:40:53.649908 7f45a36a58c0 5 asok(0x4560000) register_command version hook 0x44e40b8 -35> 2015-09-09 10:40:53.649911 7f45a36a58c0 5 asok(0x4560000) register_command git_version hook 0x44e40b8 -34> 2015-09-09 10:40:53.649917 7f45a36a58c0 5 asok(0x4560000) register_command help hook 0x44e80b0 -33> 2015-09-09 10:40:53.649919 7f45a36a58c0 5 asok(0x4560000) register_command get_command_descriptions hook 0x44e8150 -32> 2015-09-09 10:40:53.649953 7f459ea25700 5 asok(0x4560000) entry start -31> 2015-09-09 10:40:54.129644 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -30> 2015-09-09 10:40:55.084393 7f45a36a58c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -29> 2015-09-09 10:40:55.111647 7f45a36a58c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -28> 2015-09-09 10:40:55.111663 7f45a36a58c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -27> 2015-09-09 10:40:55.111713 7f45a36a58c0 5 adding auth protocol: cephx -26> 2015-09-09 10:40:55.111722 7f45a36a58c0 5 adding auth protocol: cephx -25> 2015-09-09 10:40:55.111740 7f45a36a58c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -24> 2015-09-09 10:40:55.111747 7f45a36a58c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -23> 2015-09-09 10:40:55.111838 7f45a36a58c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -22> 2015-09-09 10:40:55.181303 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -21> 2015-09-09 10:40:55.813563 7f45a36a58c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -20> 2015-09-09 10:40:55.856374 7f45a36a58c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -19> 2015-09-09 10:40:56.231730 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -18> 2015-09-09 10:40:56.563712 7f45a36a58c0 4 mon.vm14@-1(probing).mds e1 new map -17> 2015-09-09 10:40:56.563736 7f45a36a58c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled -16> 2015-09-09 10:40:57.283472 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -15> 2015-09-09 10:40:58.338499 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -14> 2015-09-09 10:40:58.623397 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -13> 2015-09-09 10:40:58.627197 7f459ea25700 1 do_command 'config get' 'var:fsid -12> 2015-09-09 10:40:58.627235 7f459ea25700 1 do_command 'config get' 'var:fsid result is 55 bytes -11> 2015-09-09 10:40:58.627242 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'config get' '' to 0x44e8050 returned 55 bytes -10> 2015-09-09 10:40:58.672274 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -9> 2015-09-09 10:40:58.676385 7f459ea25700 1 do_command 'perf dump' ' -8> 2015-09-09 10:40:58.676458 7f459ea25700 1 do_command 'perf dump' 'result is 3474 bytes -7> 2015-09-09 10:40:58.676465 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'perf dump' '' to 0x44e8050 returned 3474 bytes -6> 2015-09-09 10:40:58.719178 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -5> 2015-09-09 10:40:58.722788 7f459ea25700 1 do_command 'perf schema' ' -4> 2015-09-09 10:40:58.722868 7f459ea25700 1 do_command 'perf schema' 'result is 4842 bytes -3> 2015-09-09 10:40:58.722875 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'perf schema' '' to 0x44e8050 returned 4842 bytes -2> 2015-09-09 10:40:58.925232 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes -1> 2015-09-09 10:40:59.388820 7f459ea25700 5 asok(0x4560000) AdminSocket: request 'get_command_descriptions' '' to 0x44e8150 returned 1496 bytes 0> 2015-09-09 10:40:59.419167 7f45a36a58c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f45a36a58c0 time 2015-09-09 10:40:59.405454 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f45a0c33ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:40:59.437215 7f45a36a58c0 -1 *** Caught signal (Aborted) ** in thread 7f45a36a58c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f45a27a9340] 3: (gsignal()+0x39) [0x7f45a0c48cc9] 4: (abort()+0x148) [0x7f45a0c4c0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f45a1553535] 6: (()+0x5e6d6) [0x7f45a15516d6] 7: (()+0x5e703) [0x7f45a1551703] 8: (()+0x5e922) [0x7f45a1551922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f45a0c33ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:40:59.437215 7f45a36a58c0 -1 *** Caught signal (Aborted) ** in thread 7f45a36a58c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f45a27a9340] 3: (gsignal()+0x39) [0x7f45a0c48cc9] 4: (abort()+0x148) [0x7f45a0c4c0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f45a1553535] 6: (()+0x5e6d6) [0x7f45a15516d6] 7: (()+0x5e703) [0x7f45a1551703] 8: (()+0x5e922) [0x7f45a1551922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f45a0c33ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.289574 7fa85de918c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2661 2015-09-09 10:41:01.305348 7fa85de918c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.305579 7fa85de918c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.305912 7fa85de918c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:41:01.305926 7fa85de918c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:41:01.307257 7fa85de918c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:41:01.308828 7fa85de918c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fa85de918c0 time 2015-09-09 10:41:01.307619 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fa85b41fec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:41:01.287553 7fa85de918c0 5 asok(0x49a8000) register_command perfcounters_dump hook 0x4930050 -34> 2015-09-09 10:41:01.287579 7fa85de918c0 5 asok(0x49a8000) register_command 1 hook 0x4930050 -33> 2015-09-09 10:41:01.287582 7fa85de918c0 5 asok(0x49a8000) register_command perf dump hook 0x4930050 -32> 2015-09-09 10:41:01.287586 7fa85de918c0 5 asok(0x49a8000) register_command perfcounters_schema hook 0x4930050 -31> 2015-09-09 10:41:01.287589 7fa85de918c0 5 asok(0x49a8000) register_command 2 hook 0x4930050 -30> 2015-09-09 10:41:01.287590 7fa85de918c0 5 asok(0x49a8000) register_command perf schema hook 0x4930050 -29> 2015-09-09 10:41:01.287592 7fa85de918c0 5 asok(0x49a8000) register_command perf reset hook 0x4930050 -28> 2015-09-09 10:41:01.287594 7fa85de918c0 5 asok(0x49a8000) register_command config show hook 0x4930050 -27> 2015-09-09 10:41:01.287596 7fa85de918c0 5 asok(0x49a8000) register_command config set hook 0x4930050 -26> 2015-09-09 10:41:01.287598 7fa85de918c0 5 asok(0x49a8000) register_command config get hook 0x4930050 -25> 2015-09-09 10:41:01.287600 7fa85de918c0 5 asok(0x49a8000) register_command config diff hook 0x4930050 -24> 2015-09-09 10:41:01.287601 7fa85de918c0 5 asok(0x49a8000) register_command log flush hook 0x4930050 -23> 2015-09-09 10:41:01.287603 7fa85de918c0 5 asok(0x49a8000) register_command log dump hook 0x4930050 -22> 2015-09-09 10:41:01.287605 7fa85de918c0 5 asok(0x49a8000) register_command log reopen hook 0x4930050 -21> 2015-09-09 10:41:01.289574 7fa85de918c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2661 -20> 2015-09-09 10:41:01.290729 7fa85de918c0 5 asok(0x49a8000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:41:01.290741 7fa85de918c0 5 asok(0x49a8000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:41:01.290763 7fa85de918c0 5 asok(0x49a8000) register_command 0 hook 0x492c0b8 -17> 2015-09-09 10:41:01.290767 7fa85de918c0 5 asok(0x49a8000) register_command version hook 0x492c0b8 -16> 2015-09-09 10:41:01.290769 7fa85de918c0 5 asok(0x49a8000) register_command git_version hook 0x492c0b8 -15> 2015-09-09 10:41:01.290772 7fa85de918c0 5 asok(0x49a8000) register_command help hook 0x49300b0 -14> 2015-09-09 10:41:01.290774 7fa85de918c0 5 asok(0x49a8000) register_command get_command_descriptions hook 0x4930150 -13> 2015-09-09 10:41:01.290806 7fa859211700 5 asok(0x49a8000) entry start -12> 2015-09-09 10:41:01.305348 7fa85de918c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:41:01.305409 7fa85de918c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:41:01.305414 7fa85de918c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:41:01.305485 7fa85de918c0 5 adding auth protocol: cephx -8> 2015-09-09 10:41:01.305493 7fa85de918c0 5 adding auth protocol: cephx -7> 2015-09-09 10:41:01.305512 7fa85de918c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:41:01.305514 7fa85de918c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:41:01.305579 7fa85de918c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:41:01.305912 7fa85de918c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:41:01.305926 7fa85de918c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:41:01.307242 7fa85de918c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:41:01.307257 7fa85de918c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:41:01.308828 7fa85de918c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fa85de918c0 time 2015-09-09 10:41:01.307619 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fa85b41fec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.310802 7fa85de918c0 -1 *** Caught signal (Aborted) ** in thread 7fa85de918c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fa85cf95340] 3: (gsignal()+0x39) [0x7fa85b434cc9] 4: (abort()+0x148) [0x7fa85b4380d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa85bd3f535] 6: (()+0x5e6d6) [0x7fa85bd3d6d6] 7: (()+0x5e703) [0x7fa85bd3d703] 8: (()+0x5e922) [0x7fa85bd3d922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fa85b41fec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:41:01.310802 7fa85de918c0 -1 *** Caught signal (Aborted) ** in thread 7fa85de918c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fa85cf95340] 3: (gsignal()+0x39) [0x7fa85b434cc9] 4: (abort()+0x148) [0x7fa85b4380d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa85bd3f535] 6: (()+0x5e6d6) [0x7fa85bd3d6d6] 7: (()+0x5e703) [0x7fa85bd3d703] 8: (()+0x5e922) [0x7fa85bd3d922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fa85b41fec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.430979 7fb09f08e8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2679 2015-09-09 10:41:01.442572 7fb09f08e8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.442788 7fb09f08e8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.443123 7fb09f08e8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:41:01.443136 7fb09f08e8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:41:01.444452 7fb09f08e8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:41:01.446041 7fb09f08e8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fb09f08e8c0 time 2015-09-09 10:41:01.444846 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fb09c61cec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:41:01.429014 7fb09f08e8c0 5 asok(0x4df4000) register_command perfcounters_dump hook 0x4d7c050 -34> 2015-09-09 10:41:01.429040 7fb09f08e8c0 5 asok(0x4df4000) register_command 1 hook 0x4d7c050 -33> 2015-09-09 10:41:01.429043 7fb09f08e8c0 5 asok(0x4df4000) register_command perf dump hook 0x4d7c050 -32> 2015-09-09 10:41:01.429048 7fb09f08e8c0 5 asok(0x4df4000) register_command perfcounters_schema hook 0x4d7c050 -31> 2015-09-09 10:41:01.429050 7fb09f08e8c0 5 asok(0x4df4000) register_command 2 hook 0x4d7c050 -30> 2015-09-09 10:41:01.429052 7fb09f08e8c0 5 asok(0x4df4000) register_command perf schema hook 0x4d7c050 -29> 2015-09-09 10:41:01.429055 7fb09f08e8c0 5 asok(0x4df4000) register_command perf reset hook 0x4d7c050 -28> 2015-09-09 10:41:01.429058 7fb09f08e8c0 5 asok(0x4df4000) register_command config show hook 0x4d7c050 -27> 2015-09-09 10:41:01.429061 7fb09f08e8c0 5 asok(0x4df4000) register_command config set hook 0x4d7c050 -26> 2015-09-09 10:41:01.429063 7fb09f08e8c0 5 asok(0x4df4000) register_command config get hook 0x4d7c050 -25> 2015-09-09 10:41:01.429066 7fb09f08e8c0 5 asok(0x4df4000) register_command config diff hook 0x4d7c050 -24> 2015-09-09 10:41:01.429068 7fb09f08e8c0 5 asok(0x4df4000) register_command log flush hook 0x4d7c050 -23> 2015-09-09 10:41:01.429069 7fb09f08e8c0 5 asok(0x4df4000) register_command log dump hook 0x4d7c050 -22> 2015-09-09 10:41:01.429071 7fb09f08e8c0 5 asok(0x4df4000) register_command log reopen hook 0x4d7c050 -21> 2015-09-09 10:41:01.430979 7fb09f08e8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2679 -20> 2015-09-09 10:41:01.432314 7fb09f08e8c0 5 asok(0x4df4000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:41:01.432325 7fb09f08e8c0 5 asok(0x4df4000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:41:01.432350 7fb09f08e8c0 5 asok(0x4df4000) register_command 0 hook 0x4d780b8 -17> 2015-09-09 10:41:01.432353 7fb09f08e8c0 5 asok(0x4df4000) register_command version hook 0x4d780b8 -16> 2015-09-09 10:41:01.432358 7fb09f08e8c0 5 asok(0x4df4000) register_command git_version hook 0x4d780b8 -15> 2015-09-09 10:41:01.432363 7fb09f08e8c0 5 asok(0x4df4000) register_command help hook 0x4d7c0b0 -14> 2015-09-09 10:41:01.432365 7fb09f08e8c0 5 asok(0x4df4000) register_command get_command_descriptions hook 0x4d7c150 -13> 2015-09-09 10:41:01.432400 7fb09a40e700 5 asok(0x4df4000) entry start -12> 2015-09-09 10:41:01.442572 7fb09f08e8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:41:01.442628 7fb09f08e8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:41:01.442636 7fb09f08e8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:41:01.442691 7fb09f08e8c0 5 adding auth protocol: cephx -8> 2015-09-09 10:41:01.442698 7fb09f08e8c0 5 adding auth protocol: cephx -7> 2015-09-09 10:41:01.442716 7fb09f08e8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:41:01.442721 7fb09f08e8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:41:01.442788 7fb09f08e8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:41:01.443123 7fb09f08e8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:41:01.443136 7fb09f08e8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:41:01.444439 7fb09f08e8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:41:01.444452 7fb09f08e8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:41:01.446041 7fb09f08e8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fb09f08e8c0 time 2015-09-09 10:41:01.444846 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fb09c61cec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.448020 7fb09f08e8c0 -1 *** Caught signal (Aborted) ** in thread 7fb09f08e8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fb09e192340] 3: (gsignal()+0x39) [0x7fb09c631cc9] 4: (abort()+0x148) [0x7fb09c6350d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fb09cf3c535] 6: (()+0x5e6d6) [0x7fb09cf3a6d6] 7: (()+0x5e703) [0x7fb09cf3a703] 8: (()+0x5e922) [0x7fb09cf3a922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fb09c61cec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:41:01.448020 7fb09f08e8c0 -1 *** Caught signal (Aborted) ** in thread 7fb09f08e8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fb09e192340] 3: (gsignal()+0x39) [0x7fb09c631cc9] 4: (abort()+0x148) [0x7fb09c6350d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fb09cf3c535] 6: (()+0x5e6d6) [0x7fb09cf3a6d6] 7: (()+0x5e703) [0x7fb09cf3a703] 8: (()+0x5e922) [0x7fb09cf3a922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fb09c61cec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.568696 7f586c4f88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2727 2015-09-09 10:41:01.581781 7f586c4f88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.582068 7f586c4f88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.582572 7f586c4f88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:41:01.582583 7f586c4f88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:41:01.584258 7f586c4f88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:41:01.586144 7f586c4f88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f586c4f88c0 time 2015-09-09 10:41:01.584700 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f5869a86ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:41:01.566095 7f586c4f88c0 5 asok(0x3644000) register_command perfcounters_dump hook 0x35cc050 -34> 2015-09-09 10:41:01.566133 7f586c4f88c0 5 asok(0x3644000) register_command 1 hook 0x35cc050 -33> 2015-09-09 10:41:01.566142 7f586c4f88c0 5 asok(0x3644000) register_command perf dump hook 0x35cc050 -32> 2015-09-09 10:41:01.566148 7f586c4f88c0 5 asok(0x3644000) register_command perfcounters_schema hook 0x35cc050 -31> 2015-09-09 10:41:01.566159 7f586c4f88c0 5 asok(0x3644000) register_command 2 hook 0x35cc050 -30> 2015-09-09 10:41:01.566162 7f586c4f88c0 5 asok(0x3644000) register_command perf schema hook 0x35cc050 -29> 2015-09-09 10:41:01.566172 7f586c4f88c0 5 asok(0x3644000) register_command perf reset hook 0x35cc050 -28> 2015-09-09 10:41:01.566174 7f586c4f88c0 5 asok(0x3644000) register_command config show hook 0x35cc050 -27> 2015-09-09 10:41:01.566177 7f586c4f88c0 5 asok(0x3644000) register_command config set hook 0x35cc050 -26> 2015-09-09 10:41:01.566185 7f586c4f88c0 5 asok(0x3644000) register_command config get hook 0x35cc050 -25> 2015-09-09 10:41:01.566187 7f586c4f88c0 5 asok(0x3644000) register_command config diff hook 0x35cc050 -24> 2015-09-09 10:41:01.566191 7f586c4f88c0 5 asok(0x3644000) register_command log flush hook 0x35cc050 -23> 2015-09-09 10:41:01.566198 7f586c4f88c0 5 asok(0x3644000) register_command log dump hook 0x35cc050 -22> 2015-09-09 10:41:01.566201 7f586c4f88c0 5 asok(0x3644000) register_command log reopen hook 0x35cc050 -21> 2015-09-09 10:41:01.568696 7f586c4f88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2727 -20> 2015-09-09 10:41:01.570138 7f586c4f88c0 5 asok(0x3644000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:41:01.570157 7f586c4f88c0 5 asok(0x3644000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:41:01.570186 7f586c4f88c0 5 asok(0x3644000) register_command 0 hook 0x35c80b8 -17> 2015-09-09 10:41:01.570198 7f586c4f88c0 5 asok(0x3644000) register_command version hook 0x35c80b8 -16> 2015-09-09 10:41:01.570201 7f586c4f88c0 5 asok(0x3644000) register_command git_version hook 0x35c80b8 -15> 2015-09-09 10:41:01.570206 7f586c4f88c0 5 asok(0x3644000) register_command help hook 0x35cc0b0 -14> 2015-09-09 10:41:01.570215 7f586c4f88c0 5 asok(0x3644000) register_command get_command_descriptions hook 0x35cc150 -13> 2015-09-09 10:41:01.570252 7f5867878700 5 asok(0x3644000) entry start -12> 2015-09-09 10:41:01.581781 7f586c4f88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:41:01.581847 7f586c4f88c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:41:01.581852 7f586c4f88c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:41:01.581960 7f586c4f88c0 5 adding auth protocol: cephx -8> 2015-09-09 10:41:01.581969 7f586c4f88c0 5 adding auth protocol: cephx -7> 2015-09-09 10:41:01.581994 7f586c4f88c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:41:01.581997 7f586c4f88c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:41:01.582068 7f586c4f88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:41:01.582572 7f586c4f88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:41:01.582583 7f586c4f88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:41:01.584251 7f586c4f88c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:41:01.584258 7f586c4f88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:41:01.586144 7f586c4f88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f586c4f88c0 time 2015-09-09 10:41:01.584700 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f5869a86ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.589123 7f586c4f88c0 -1 *** Caught signal (Aborted) ** in thread 7f586c4f88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f586b5fc340] 3: (gsignal()+0x39) [0x7f5869a9bcc9] 4: (abort()+0x148) [0x7f5869a9f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f586a3a6535] 6: (()+0x5e6d6) [0x7f586a3a46d6] 7: (()+0x5e703) [0x7f586a3a4703] 8: (()+0x5e922) [0x7f586a3a4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f5869a86ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:41:01.589123 7f586c4f88c0 -1 *** Caught signal (Aborted) ** in thread 7f586c4f88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f586b5fc340] 3: (gsignal()+0x39) [0x7f5869a9bcc9] 4: (abort()+0x148) [0x7f5869a9f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f586a3a6535] 6: (()+0x5e6d6) [0x7f586a3a46d6] 7: (()+0x5e703) [0x7f586a3a4703] 8: (()+0x5e922) [0x7f586a3a4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f5869a86ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.706769 7f6b3a0068c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2745 2015-09-09 10:41:01.718337 7f6b3a0068c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.718550 7f6b3a0068c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.718889 7f6b3a0068c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:41:01.718904 7f6b3a0068c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:41:01.720220 7f6b3a0068c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:41:01.721799 7f6b3a0068c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f6b3a0068c0 time 2015-09-09 10:41:01.720591 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f6b37594ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:41:01.704726 7f6b3a0068c0 5 asok(0x50ce000) register_command perfcounters_dump hook 0x5056050 -34> 2015-09-09 10:41:01.704751 7f6b3a0068c0 5 asok(0x50ce000) register_command 1 hook 0x5056050 -33> 2015-09-09 10:41:01.704755 7f6b3a0068c0 5 asok(0x50ce000) register_command perf dump hook 0x5056050 -32> 2015-09-09 10:41:01.704759 7f6b3a0068c0 5 asok(0x50ce000) register_command perfcounters_schema hook 0x5056050 -31> 2015-09-09 10:41:01.704761 7f6b3a0068c0 5 asok(0x50ce000) register_command 2 hook 0x5056050 -30> 2015-09-09 10:41:01.704763 7f6b3a0068c0 5 asok(0x50ce000) register_command perf schema hook 0x5056050 -29> 2015-09-09 10:41:01.704765 7f6b3a0068c0 5 asok(0x50ce000) register_command perf reset hook 0x5056050 -28> 2015-09-09 10:41:01.704767 7f6b3a0068c0 5 asok(0x50ce000) register_command config show hook 0x5056050 -27> 2015-09-09 10:41:01.704769 7f6b3a0068c0 5 asok(0x50ce000) register_command config set hook 0x5056050 -26> 2015-09-09 10:41:01.704771 7f6b3a0068c0 5 asok(0x50ce000) register_command config get hook 0x5056050 -25> 2015-09-09 10:41:01.704772 7f6b3a0068c0 5 asok(0x50ce000) register_command config diff hook 0x5056050 -24> 2015-09-09 10:41:01.704774 7f6b3a0068c0 5 asok(0x50ce000) register_command log flush hook 0x5056050 -23> 2015-09-09 10:41:01.704775 7f6b3a0068c0 5 asok(0x50ce000) register_command log dump hook 0x5056050 -22> 2015-09-09 10:41:01.704777 7f6b3a0068c0 5 asok(0x50ce000) register_command log reopen hook 0x5056050 -21> 2015-09-09 10:41:01.706769 7f6b3a0068c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2745 -20> 2015-09-09 10:41:01.707916 7f6b3a0068c0 5 asok(0x50ce000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:41:01.707929 7f6b3a0068c0 5 asok(0x50ce000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:41:01.707952 7f6b3a0068c0 5 asok(0x50ce000) register_command 0 hook 0x50520b8 -17> 2015-09-09 10:41:01.707955 7f6b3a0068c0 5 asok(0x50ce000) register_command version hook 0x50520b8 -16> 2015-09-09 10:41:01.707958 7f6b3a0068c0 5 asok(0x50ce000) register_command git_version hook 0x50520b8 -15> 2015-09-09 10:41:01.707960 7f6b3a0068c0 5 asok(0x50ce000) register_command help hook 0x50560b0 -14> 2015-09-09 10:41:01.707962 7f6b3a0068c0 5 asok(0x50ce000) register_command get_command_descriptions hook 0x5056150 -13> 2015-09-09 10:41:01.707996 7f6b35386700 5 asok(0x50ce000) entry start -12> 2015-09-09 10:41:01.718337 7f6b3a0068c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:41:01.718396 7f6b3a0068c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:41:01.718400 7f6b3a0068c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:41:01.718457 7f6b3a0068c0 5 adding auth protocol: cephx -8> 2015-09-09 10:41:01.718464 7f6b3a0068c0 5 adding auth protocol: cephx -7> 2015-09-09 10:41:01.718483 7f6b3a0068c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:41:01.718486 7f6b3a0068c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:41:01.718550 7f6b3a0068c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:41:01.718889 7f6b3a0068c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:41:01.718904 7f6b3a0068c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:41:01.720206 7f6b3a0068c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:41:01.720220 7f6b3a0068c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:41:01.721799 7f6b3a0068c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f6b3a0068c0 time 2015-09-09 10:41:01.720591 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f6b37594ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.723806 7f6b3a0068c0 -1 *** Caught signal (Aborted) ** in thread 7f6b3a0068c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f6b3910a340] 3: (gsignal()+0x39) [0x7f6b375a9cc9] 4: (abort()+0x148) [0x7f6b375ad0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f6b37eb4535] 6: (()+0x5e6d6) [0x7f6b37eb26d6] 7: (()+0x5e703) [0x7f6b37eb2703] 8: (()+0x5e922) [0x7f6b37eb2922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f6b37594ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:41:01.723806 7f6b3a0068c0 -1 *** Caught signal (Aborted) ** in thread 7f6b3a0068c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f6b3910a340] 3: (gsignal()+0x39) [0x7f6b375a9cc9] 4: (abort()+0x148) [0x7f6b375ad0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f6b37eb4535] 6: (()+0x5e6d6) [0x7f6b37eb26d6] 7: (()+0x5e703) [0x7f6b37eb2703] 8: (()+0x5e922) [0x7f6b37eb2922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f6b37594ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.844002 7f7ed679f8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2763 2015-09-09 10:41:01.863998 7f7ed679f8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.864281 7f7ed679f8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:41:01.864627 7f7ed679f8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:41:01.864660 7f7ed679f8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:41:01.866097 7f7ed679f8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:41:01.867736 7f7ed679f8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f7ed679f8c0 time 2015-09-09 10:41:01.866537 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f7ed3d2dec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:41:01.841926 7f7ed679f8c0 5 asok(0x3802000) register_command perfcounters_dump hook 0x378a050 -34> 2015-09-09 10:41:01.841955 7f7ed679f8c0 5 asok(0x3802000) register_command 1 hook 0x378a050 -33> 2015-09-09 10:41:01.841959 7f7ed679f8c0 5 asok(0x3802000) register_command perf dump hook 0x378a050 -32> 2015-09-09 10:41:01.841965 7f7ed679f8c0 5 asok(0x3802000) register_command perfcounters_schema hook 0x378a050 -31> 2015-09-09 10:41:01.841968 7f7ed679f8c0 5 asok(0x3802000) register_command 2 hook 0x378a050 -30> 2015-09-09 10:41:01.841971 7f7ed679f8c0 5 asok(0x3802000) register_command perf schema hook 0x378a050 -29> 2015-09-09 10:41:01.841974 7f7ed679f8c0 5 asok(0x3802000) register_command perf reset hook 0x378a050 -28> 2015-09-09 10:41:01.841976 7f7ed679f8c0 5 asok(0x3802000) register_command config show hook 0x378a050 -27> 2015-09-09 10:41:01.841979 7f7ed679f8c0 5 asok(0x3802000) register_command config set hook 0x378a050 -26> 2015-09-09 10:41:01.841982 7f7ed679f8c0 5 asok(0x3802000) register_command config get hook 0x378a050 -25> 2015-09-09 10:41:01.841985 7f7ed679f8c0 5 asok(0x3802000) register_command config diff hook 0x378a050 -24> 2015-09-09 10:41:01.841987 7f7ed679f8c0 5 asok(0x3802000) register_command log flush hook 0x378a050 -23> 2015-09-09 10:41:01.841990 7f7ed679f8c0 5 asok(0x3802000) register_command log dump hook 0x378a050 -22> 2015-09-09 10:41:01.841992 7f7ed679f8c0 5 asok(0x3802000) register_command log reopen hook 0x378a050 -21> 2015-09-09 10:41:01.844002 7f7ed679f8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 2763 -20> 2015-09-09 10:41:01.845202 7f7ed679f8c0 5 asok(0x3802000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:41:01.845214 7f7ed679f8c0 5 asok(0x3802000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:41:01.845237 7f7ed679f8c0 5 asok(0x3802000) register_command 0 hook 0x37860b8 -17> 2015-09-09 10:41:01.845241 7f7ed679f8c0 5 asok(0x3802000) register_command version hook 0x37860b8 -16> 2015-09-09 10:41:01.845244 7f7ed679f8c0 5 asok(0x3802000) register_command git_version hook 0x37860b8 -15> 2015-09-09 10:41:01.845250 7f7ed679f8c0 5 asok(0x3802000) register_command help hook 0x378a0b0 -14> 2015-09-09 10:41:01.845252 7f7ed679f8c0 5 asok(0x3802000) register_command get_command_descriptions hook 0x378a150 -13> 2015-09-09 10:41:01.845285 7f7ed1b1f700 5 asok(0x3802000) entry start -12> 2015-09-09 10:41:01.863998 7f7ed679f8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:41:01.864084 7f7ed679f8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:41:01.864097 7f7ed679f8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:41:01.864163 7f7ed679f8c0 5 adding auth protocol: cephx -8> 2015-09-09 10:41:01.864176 7f7ed679f8c0 5 adding auth protocol: cephx -7> 2015-09-09 10:41:01.864201 7f7ed679f8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:41:01.864211 7f7ed679f8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:41:01.864281 7f7ed679f8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:41:01.864627 7f7ed679f8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:41:01.864660 7f7ed679f8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:41:01.866078 7f7ed679f8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:41:01.866097 7f7ed679f8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:41:01.867736 7f7ed679f8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f7ed679f8c0 time 2015-09-09 10:41:01.866537 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f7ed3d2dec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:41:01.869855 7f7ed679f8c0 -1 *** Caught signal (Aborted) ** in thread 7f7ed679f8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f7ed58a3340] 3: (gsignal()+0x39) [0x7f7ed3d42cc9] 4: (abort()+0x148) [0x7f7ed3d460d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f7ed464d535] 6: (()+0x5e6d6) [0x7f7ed464b6d6] 7: (()+0x5e703) [0x7f7ed464b703] 8: (()+0x5e922) [0x7f7ed464b922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f7ed3d2dec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:41:01.869855 7f7ed679f8c0 -1 *** Caught signal (Aborted) ** in thread 7f7ed679f8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f7ed58a3340] 3: (gsignal()+0x39) [0x7f7ed3d42cc9] 4: (abort()+0x148) [0x7f7ed3d460d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f7ed464d535] 6: (()+0x5e6d6) [0x7f7ed464b6d6] 7: (()+0x5e703) [0x7f7ed464b703] 8: (()+0x5e922) [0x7f7ed464b922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f7ed3d2dec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.137688 7f3f869c48c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4300 2015-09-09 10:48:53.156998 7f3f869c48c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.157277 7f3f869c48c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.157644 7f3f869c48c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.157653 7f3f869c48c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.159028 7f3f869c48c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.160711 7f3f869c48c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f3f869c48c0 time 2015-09-09 10:48:53.159436 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f3f83f52ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.135634 7f3f869c48c0 5 asok(0x5172000) register_command perfcounters_dump hook 0x50fa050 -34> 2015-09-09 10:48:53.135663 7f3f869c48c0 5 asok(0x5172000) register_command 1 hook 0x50fa050 -33> 2015-09-09 10:48:53.135666 7f3f869c48c0 5 asok(0x5172000) register_command perf dump hook 0x50fa050 -32> 2015-09-09 10:48:53.135671 7f3f869c48c0 5 asok(0x5172000) register_command perfcounters_schema hook 0x50fa050 -31> 2015-09-09 10:48:53.135673 7f3f869c48c0 5 asok(0x5172000) register_command 2 hook 0x50fa050 -30> 2015-09-09 10:48:53.135675 7f3f869c48c0 5 asok(0x5172000) register_command perf schema hook 0x50fa050 -29> 2015-09-09 10:48:53.135677 7f3f869c48c0 5 asok(0x5172000) register_command perf reset hook 0x50fa050 -28> 2015-09-09 10:48:53.135679 7f3f869c48c0 5 asok(0x5172000) register_command config show hook 0x50fa050 -27> 2015-09-09 10:48:53.135681 7f3f869c48c0 5 asok(0x5172000) register_command config set hook 0x50fa050 -26> 2015-09-09 10:48:53.135683 7f3f869c48c0 5 asok(0x5172000) register_command config get hook 0x50fa050 -25> 2015-09-09 10:48:53.135685 7f3f869c48c0 5 asok(0x5172000) register_command config diff hook 0x50fa050 -24> 2015-09-09 10:48:53.135686 7f3f869c48c0 5 asok(0x5172000) register_command log flush hook 0x50fa050 -23> 2015-09-09 10:48:53.135688 7f3f869c48c0 5 asok(0x5172000) register_command log dump hook 0x50fa050 -22> 2015-09-09 10:48:53.135690 7f3f869c48c0 5 asok(0x5172000) register_command log reopen hook 0x50fa050 -21> 2015-09-09 10:48:53.137688 7f3f869c48c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4300 -20> 2015-09-09 10:48:53.139019 7f3f869c48c0 5 asok(0x5172000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.139028 7f3f869c48c0 5 asok(0x5172000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.139055 7f3f869c48c0 5 asok(0x5172000) register_command 0 hook 0x50f60b8 -17> 2015-09-09 10:48:53.139060 7f3f869c48c0 5 asok(0x5172000) register_command version hook 0x50f60b8 -16> 2015-09-09 10:48:53.139063 7f3f869c48c0 5 asok(0x5172000) register_command git_version hook 0x50f60b8 -15> 2015-09-09 10:48:53.139065 7f3f869c48c0 5 asok(0x5172000) register_command help hook 0x50fa0a0 -14> 2015-09-09 10:48:53.139067 7f3f869c48c0 5 asok(0x5172000) register_command get_command_descriptions hook 0x50fa140 -13> 2015-09-09 10:48:53.139100 7f3f81d44700 5 asok(0x5172000) entry start -12> 2015-09-09 10:48:53.156998 7f3f869c48c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.157088 7f3f869c48c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.157097 7f3f869c48c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.157166 7f3f869c48c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.157172 7f3f869c48c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.157198 7f3f869c48c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.157202 7f3f869c48c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.157277 7f3f869c48c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.157644 7f3f869c48c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.157653 7f3f869c48c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.159007 7f3f869c48c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.159028 7f3f869c48c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.160711 7f3f869c48c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f3f869c48c0 time 2015-09-09 10:48:53.159436 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f3f83f52ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.162743 7f3f869c48c0 -1 *** Caught signal (Aborted) ** in thread 7f3f869c48c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f3f85ac8340] 3: (gsignal()+0x39) [0x7f3f83f67cc9] 4: (abort()+0x148) [0x7f3f83f6b0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f3f84872535] 6: (()+0x5e6d6) [0x7f3f848706d6] 7: (()+0x5e703) [0x7f3f84870703] 8: (()+0x5e922) [0x7f3f84870922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f3f83f52ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.162743 7f3f869c48c0 -1 *** Caught signal (Aborted) ** in thread 7f3f869c48c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f3f85ac8340] 3: (gsignal()+0x39) [0x7f3f83f67cc9] 4: (abort()+0x148) [0x7f3f83f6b0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f3f84872535] 6: (()+0x5e6d6) [0x7f3f848706d6] 7: (()+0x5e703) [0x7f3f84870703] 8: (()+0x5e922) [0x7f3f84870922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f3f83f52ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.296352 7f8bda30c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4374 2015-09-09 10:48:53.313376 7f8bda30c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.313750 7f8bda30c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.314126 7f8bda30c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.314136 7f8bda30c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.316369 7f8bda30c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.319913 7f8bda30c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8bda30c8c0 time 2015-09-09 10:48:53.318481 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8bd789aec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.294381 7f8bda30c8c0 5 asok(0x42b6000) register_command perfcounters_dump hook 0x423e050 -34> 2015-09-09 10:48:53.294411 7f8bda30c8c0 5 asok(0x42b6000) register_command 1 hook 0x423e050 -33> 2015-09-09 10:48:53.294413 7f8bda30c8c0 5 asok(0x42b6000) register_command perf dump hook 0x423e050 -32> 2015-09-09 10:48:53.294418 7f8bda30c8c0 5 asok(0x42b6000) register_command perfcounters_schema hook 0x423e050 -31> 2015-09-09 10:48:53.294420 7f8bda30c8c0 5 asok(0x42b6000) register_command 2 hook 0x423e050 -30> 2015-09-09 10:48:53.294422 7f8bda30c8c0 5 asok(0x42b6000) register_command perf schema hook 0x423e050 -29> 2015-09-09 10:48:53.294424 7f8bda30c8c0 5 asok(0x42b6000) register_command perf reset hook 0x423e050 -28> 2015-09-09 10:48:53.294426 7f8bda30c8c0 5 asok(0x42b6000) register_command config show hook 0x423e050 -27> 2015-09-09 10:48:53.294428 7f8bda30c8c0 5 asok(0x42b6000) register_command config set hook 0x423e050 -26> 2015-09-09 10:48:53.294430 7f8bda30c8c0 5 asok(0x42b6000) register_command config get hook 0x423e050 -25> 2015-09-09 10:48:53.294432 7f8bda30c8c0 5 asok(0x42b6000) register_command config diff hook 0x423e050 -24> 2015-09-09 10:48:53.294433 7f8bda30c8c0 5 asok(0x42b6000) register_command log flush hook 0x423e050 -23> 2015-09-09 10:48:53.294435 7f8bda30c8c0 5 asok(0x42b6000) register_command log dump hook 0x423e050 -22> 2015-09-09 10:48:53.294437 7f8bda30c8c0 5 asok(0x42b6000) register_command log reopen hook 0x423e050 -21> 2015-09-09 10:48:53.296352 7f8bda30c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4374 -20> 2015-09-09 10:48:53.298757 7f8bda30c8c0 5 asok(0x42b6000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.298776 7f8bda30c8c0 5 asok(0x42b6000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.298804 7f8bda30c8c0 5 asok(0x42b6000) register_command 0 hook 0x423a0b8 -17> 2015-09-09 10:48:53.298817 7f8bda30c8c0 5 asok(0x42b6000) register_command version hook 0x423a0b8 -16> 2015-09-09 10:48:53.298820 7f8bda30c8c0 5 asok(0x42b6000) register_command git_version hook 0x423a0b8 -15> 2015-09-09 10:48:53.298822 7f8bda30c8c0 5 asok(0x42b6000) register_command help hook 0x423e0b0 -14> 2015-09-09 10:48:53.298824 7f8bda30c8c0 5 asok(0x42b6000) register_command get_command_descriptions hook 0x423e150 -13> 2015-09-09 10:48:53.298853 7f8bd568c700 5 asok(0x42b6000) entry start -12> 2015-09-09 10:48:53.313376 7f8bda30c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.313435 7f8bda30c8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.313439 7f8bda30c8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.313652 7f8bda30c8c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.313659 7f8bda30c8c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.313678 7f8bda30c8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.313680 7f8bda30c8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.313750 7f8bda30c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.314126 7f8bda30c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.314136 7f8bda30c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.316346 7f8bda30c8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.316369 7f8bda30c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.319913 7f8bda30c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8bda30c8c0 time 2015-09-09 10:48:53.318481 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8bd789aec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.329352 7f8bda30c8c0 -1 *** Caught signal (Aborted) ** in thread 7f8bda30c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8bd9410340] 3: (gsignal()+0x39) [0x7f8bd78afcc9] 4: (abort()+0x148) [0x7f8bd78b30d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8bd81ba535] 6: (()+0x5e6d6) [0x7f8bd81b86d6] 7: (()+0x5e703) [0x7f8bd81b8703] 8: (()+0x5e922) [0x7f8bd81b8922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8bd789aec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.329352 7f8bda30c8c0 -1 *** Caught signal (Aborted) ** in thread 7f8bda30c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8bd9410340] 3: (gsignal()+0x39) [0x7f8bd78afcc9] 4: (abort()+0x148) [0x7f8bd78b30d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8bd81ba535] 6: (()+0x5e6d6) [0x7f8bd81b86d6] 7: (()+0x5e703) [0x7f8bd81b8703] 8: (()+0x5e922) [0x7f8bd81b8922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8bd789aec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.457336 7fee8625c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4471 2015-09-09 10:48:53.469863 7fee8625c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.470111 7fee8625c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.470498 7fee8625c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.470507 7fee8625c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.471805 7fee8625c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.473632 7fee8625c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fee8625c8c0 time 2015-09-09 10:48:53.472181 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fee837eaec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.455347 7fee8625c8c0 5 asok(0x3956000) register_command perfcounters_dump hook 0x38de050 -34> 2015-09-09 10:48:53.455385 7fee8625c8c0 5 asok(0x3956000) register_command 1 hook 0x38de050 -33> 2015-09-09 10:48:53.455389 7fee8625c8c0 5 asok(0x3956000) register_command perf dump hook 0x38de050 -32> 2015-09-09 10:48:53.455393 7fee8625c8c0 5 asok(0x3956000) register_command perfcounters_schema hook 0x38de050 -31> 2015-09-09 10:48:53.455395 7fee8625c8c0 5 asok(0x3956000) register_command 2 hook 0x38de050 -30> 2015-09-09 10:48:53.455397 7fee8625c8c0 5 asok(0x3956000) register_command perf schema hook 0x38de050 -29> 2015-09-09 10:48:53.455399 7fee8625c8c0 5 asok(0x3956000) register_command perf reset hook 0x38de050 -28> 2015-09-09 10:48:53.455401 7fee8625c8c0 5 asok(0x3956000) register_command config show hook 0x38de050 -27> 2015-09-09 10:48:53.455403 7fee8625c8c0 5 asok(0x3956000) register_command config set hook 0x38de050 -26> 2015-09-09 10:48:53.455405 7fee8625c8c0 5 asok(0x3956000) register_command config get hook 0x38de050 -25> 2015-09-09 10:48:53.455407 7fee8625c8c0 5 asok(0x3956000) register_command config diff hook 0x38de050 -24> 2015-09-09 10:48:53.455408 7fee8625c8c0 5 asok(0x3956000) register_command log flush hook 0x38de050 -23> 2015-09-09 10:48:53.455410 7fee8625c8c0 5 asok(0x3956000) register_command log dump hook 0x38de050 -22> 2015-09-09 10:48:53.455412 7fee8625c8c0 5 asok(0x3956000) register_command log reopen hook 0x38de050 -21> 2015-09-09 10:48:53.457336 7fee8625c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4471 -20> 2015-09-09 10:48:53.458630 7fee8625c8c0 5 asok(0x3956000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.458645 7fee8625c8c0 5 asok(0x3956000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.458676 7fee8625c8c0 5 asok(0x3956000) register_command 0 hook 0x38da0b8 -17> 2015-09-09 10:48:53.458685 7fee8625c8c0 5 asok(0x3956000) register_command version hook 0x38da0b8 -16> 2015-09-09 10:48:53.458687 7fee8625c8c0 5 asok(0x3956000) register_command git_version hook 0x38da0b8 -15> 2015-09-09 10:48:53.458692 7fee8625c8c0 5 asok(0x3956000) register_command help hook 0x38de0b0 -14> 2015-09-09 10:48:53.458694 7fee8625c8c0 5 asok(0x3956000) register_command get_command_descriptions hook 0x38de150 -13> 2015-09-09 10:48:53.458729 7fee815dc700 5 asok(0x3956000) entry start -12> 2015-09-09 10:48:53.469863 7fee8625c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.469922 7fee8625c8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.469926 7fee8625c8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.470016 7fee8625c8c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.470022 7fee8625c8c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.470041 7fee8625c8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.470044 7fee8625c8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.470111 7fee8625c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.470498 7fee8625c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.470507 7fee8625c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.471798 7fee8625c8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.471805 7fee8625c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.473632 7fee8625c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fee8625c8c0 time 2015-09-09 10:48:53.472181 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fee837eaec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.475824 7fee8625c8c0 -1 *** Caught signal (Aborted) ** in thread 7fee8625c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fee85360340] 3: (gsignal()+0x39) [0x7fee837ffcc9] 4: (abort()+0x148) [0x7fee838030d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fee8410a535] 6: (()+0x5e6d6) [0x7fee841086d6] 7: (()+0x5e703) [0x7fee84108703] 8: (()+0x5e922) [0x7fee84108922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fee837eaec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.475824 7fee8625c8c0 -1 *** Caught signal (Aborted) ** in thread 7fee8625c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fee85360340] 3: (gsignal()+0x39) [0x7fee837ffcc9] 4: (abort()+0x148) [0x7fee838030d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fee8410a535] 6: (()+0x5e6d6) [0x7fee841086d6] 7: (()+0x5e703) [0x7fee84108703] 8: (()+0x5e922) [0x7fee84108922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fee837eaec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.606653 7f9733a3a8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4489 2015-09-09 10:48:53.618847 7f9733a3a8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.619091 7f9733a3a8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.619488 7f9733a3a8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.619496 7f9733a3a8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.620903 7f9733a3a8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.622638 7f9733a3a8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f9733a3a8c0 time 2015-09-09 10:48:53.621410 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f9730fc8ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.604571 7f9733a3a8c0 5 asok(0x418c000) register_command perfcounters_dump hook 0x4114050 -34> 2015-09-09 10:48:53.604601 7f9733a3a8c0 5 asok(0x418c000) register_command 1 hook 0x4114050 -33> 2015-09-09 10:48:53.604604 7f9733a3a8c0 5 asok(0x418c000) register_command perf dump hook 0x4114050 -32> 2015-09-09 10:48:53.604609 7f9733a3a8c0 5 asok(0x418c000) register_command perfcounters_schema hook 0x4114050 -31> 2015-09-09 10:48:53.604611 7f9733a3a8c0 5 asok(0x418c000) register_command 2 hook 0x4114050 -30> 2015-09-09 10:48:53.604613 7f9733a3a8c0 5 asok(0x418c000) register_command perf schema hook 0x4114050 -29> 2015-09-09 10:48:53.604614 7f9733a3a8c0 5 asok(0x418c000) register_command perf reset hook 0x4114050 -28> 2015-09-09 10:48:53.604616 7f9733a3a8c0 5 asok(0x418c000) register_command config show hook 0x4114050 -27> 2015-09-09 10:48:53.604618 7f9733a3a8c0 5 asok(0x418c000) register_command config set hook 0x4114050 -26> 2015-09-09 10:48:53.604620 7f9733a3a8c0 5 asok(0x418c000) register_command config get hook 0x4114050 -25> 2015-09-09 10:48:53.604622 7f9733a3a8c0 5 asok(0x418c000) register_command config diff hook 0x4114050 -24> 2015-09-09 10:48:53.604623 7f9733a3a8c0 5 asok(0x418c000) register_command log flush hook 0x4114050 -23> 2015-09-09 10:48:53.604625 7f9733a3a8c0 5 asok(0x418c000) register_command log dump hook 0x4114050 -22> 2015-09-09 10:48:53.604627 7f9733a3a8c0 5 asok(0x418c000) register_command log reopen hook 0x4114050 -21> 2015-09-09 10:48:53.606653 7f9733a3a8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4489 -20> 2015-09-09 10:48:53.608015 7f9733a3a8c0 5 asok(0x418c000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.608028 7f9733a3a8c0 5 asok(0x418c000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.608053 7f9733a3a8c0 5 asok(0x418c000) register_command 0 hook 0x41100b8 -17> 2015-09-09 10:48:53.608060 7f9733a3a8c0 5 asok(0x418c000) register_command version hook 0x41100b8 -16> 2015-09-09 10:48:53.608063 7f9733a3a8c0 5 asok(0x418c000) register_command git_version hook 0x41100b8 -15> 2015-09-09 10:48:53.608065 7f9733a3a8c0 5 asok(0x418c000) register_command help hook 0x41140b0 -14> 2015-09-09 10:48:53.608067 7f9733a3a8c0 5 asok(0x418c000) register_command get_command_descriptions hook 0x4114150 -13> 2015-09-09 10:48:53.608101 7f972edba700 5 asok(0x418c000) entry start -12> 2015-09-09 10:48:53.618847 7f9733a3a8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.618908 7f9733a3a8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.618913 7f9733a3a8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.618997 7f9733a3a8c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.619003 7f9733a3a8c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.619023 7f9733a3a8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.619025 7f9733a3a8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.619091 7f9733a3a8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.619488 7f9733a3a8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.619496 7f9733a3a8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.620884 7f9733a3a8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.620903 7f9733a3a8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.622638 7f9733a3a8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f9733a3a8c0 time 2015-09-09 10:48:53.621410 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f9730fc8ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.624624 7f9733a3a8c0 -1 *** Caught signal (Aborted) ** in thread 7f9733a3a8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f9732b3e340] 3: (gsignal()+0x39) [0x7f9730fddcc9] 4: (abort()+0x148) [0x7f9730fe10d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f97318e8535] 6: (()+0x5e6d6) [0x7f97318e66d6] 7: (()+0x5e703) [0x7f97318e6703] 8: (()+0x5e922) [0x7f97318e6922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f9730fc8ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.624624 7f9733a3a8c0 -1 *** Caught signal (Aborted) ** in thread 7f9733a3a8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f9732b3e340] 3: (gsignal()+0x39) [0x7f9730fddcc9] 4: (abort()+0x148) [0x7f9730fe10d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f97318e8535] 6: (()+0x5e6d6) [0x7f97318e66d6] 7: (()+0x5e703) [0x7f97318e6703] 8: (()+0x5e922) [0x7f97318e6922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f9730fc8ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.760404 7ff7a44868c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4507 2015-09-09 10:48:53.774630 7ff7a44868c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.774949 7ff7a44868c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.775516 7ff7a44868c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.775530 7ff7a44868c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.776880 7ff7a44868c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.778700 7ff7a44868c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7ff7a44868c0 time 2015-09-09 10:48:53.777273 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7ff7a1a14ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.758353 7ff7a44868c0 5 asok(0x4996000) register_command perfcounters_dump hook 0x491e050 -34> 2015-09-09 10:48:53.758387 7ff7a44868c0 5 asok(0x4996000) register_command 1 hook 0x491e050 -33> 2015-09-09 10:48:53.758390 7ff7a44868c0 5 asok(0x4996000) register_command perf dump hook 0x491e050 -32> 2015-09-09 10:48:53.758395 7ff7a44868c0 5 asok(0x4996000) register_command perfcounters_schema hook 0x491e050 -31> 2015-09-09 10:48:53.758397 7ff7a44868c0 5 asok(0x4996000) register_command 2 hook 0x491e050 -30> 2015-09-09 10:48:53.758399 7ff7a44868c0 5 asok(0x4996000) register_command perf schema hook 0x491e050 -29> 2015-09-09 10:48:53.758401 7ff7a44868c0 5 asok(0x4996000) register_command perf reset hook 0x491e050 -28> 2015-09-09 10:48:53.758403 7ff7a44868c0 5 asok(0x4996000) register_command config show hook 0x491e050 -27> 2015-09-09 10:48:53.758405 7ff7a44868c0 5 asok(0x4996000) register_command config set hook 0x491e050 -26> 2015-09-09 10:48:53.758407 7ff7a44868c0 5 asok(0x4996000) register_command config get hook 0x491e050 -25> 2015-09-09 10:48:53.758408 7ff7a44868c0 5 asok(0x4996000) register_command config diff hook 0x491e050 -24> 2015-09-09 10:48:53.758410 7ff7a44868c0 5 asok(0x4996000) register_command log flush hook 0x491e050 -23> 2015-09-09 10:48:53.758412 7ff7a44868c0 5 asok(0x4996000) register_command log dump hook 0x491e050 -22> 2015-09-09 10:48:53.758413 7ff7a44868c0 5 asok(0x4996000) register_command log reopen hook 0x491e050 -21> 2015-09-09 10:48:53.760404 7ff7a44868c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4507 -20> 2015-09-09 10:48:53.762047 7ff7a44868c0 5 asok(0x4996000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.762069 7ff7a44868c0 5 asok(0x4996000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.762109 7ff7a44868c0 5 asok(0x4996000) register_command 0 hook 0x491a0b8 -17> 2015-09-09 10:48:53.762122 7ff7a44868c0 5 asok(0x4996000) register_command version hook 0x491a0b8 -16> 2015-09-09 10:48:53.762130 7ff7a44868c0 5 asok(0x4996000) register_command git_version hook 0x491a0b8 -15> 2015-09-09 10:48:53.762138 7ff7a44868c0 5 asok(0x4996000) register_command help hook 0x491e0b0 -14> 2015-09-09 10:48:53.762143 7ff7a44868c0 5 asok(0x4996000) register_command get_command_descriptions hook 0x491e150 -13> 2015-09-09 10:48:53.762190 7ff79f806700 5 asok(0x4996000) entry start -12> 2015-09-09 10:48:53.774630 7ff7a44868c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.774716 7ff7a44868c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.774723 7ff7a44868c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.774829 7ff7a44868c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.774837 7ff7a44868c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.774860 7ff7a44868c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.774864 7ff7a44868c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.774949 7ff7a44868c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.775516 7ff7a44868c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.775530 7ff7a44868c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.776873 7ff7a44868c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.776880 7ff7a44868c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.778700 7ff7a44868c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7ff7a44868c0 time 2015-09-09 10:48:53.777273 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7ff7a1a14ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.781007 7ff7a44868c0 -1 *** Caught signal (Aborted) ** in thread 7ff7a44868c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7ff7a358a340] 3: (gsignal()+0x39) [0x7ff7a1a29cc9] 4: (abort()+0x148) [0x7ff7a1a2d0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff7a2334535] 6: (()+0x5e6d6) [0x7ff7a23326d6] 7: (()+0x5e703) [0x7ff7a2332703] 8: (()+0x5e922) [0x7ff7a2332922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7ff7a1a14ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.781007 7ff7a44868c0 -1 *** Caught signal (Aborted) ** in thread 7ff7a44868c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7ff7a358a340] 3: (gsignal()+0x39) [0x7ff7a1a29cc9] 4: (abort()+0x148) [0x7ff7a1a2d0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff7a2334535] 6: (()+0x5e6d6) [0x7ff7a23326d6] 7: (()+0x5e703) [0x7ff7a2332703] 8: (()+0x5e922) [0x7ff7a2332922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7ff7a1a14ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.915965 7f09b03188c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4525 2015-09-09 10:48:53.928777 7f09b03188c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.929039 7f09b03188c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:48:53.929376 7f09b03188c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:48:53.929383 7f09b03188c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:48:53.930854 7f09b03188c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:48:53.932659 7f09b03188c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f09b03188c0 time 2015-09-09 10:48:53.931246 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f09ad8a6ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:48:53.913947 7f09b03188c0 5 asok(0x3d94000) register_command perfcounters_dump hook 0x3d1c050 -34> 2015-09-09 10:48:53.913981 7f09b03188c0 5 asok(0x3d94000) register_command 1 hook 0x3d1c050 -33> 2015-09-09 10:48:53.913984 7f09b03188c0 5 asok(0x3d94000) register_command perf dump hook 0x3d1c050 -32> 2015-09-09 10:48:53.913988 7f09b03188c0 5 asok(0x3d94000) register_command perfcounters_schema hook 0x3d1c050 -31> 2015-09-09 10:48:53.913991 7f09b03188c0 5 asok(0x3d94000) register_command 2 hook 0x3d1c050 -30> 2015-09-09 10:48:53.913992 7f09b03188c0 5 asok(0x3d94000) register_command perf schema hook 0x3d1c050 -29> 2015-09-09 10:48:53.913994 7f09b03188c0 5 asok(0x3d94000) register_command perf reset hook 0x3d1c050 -28> 2015-09-09 10:48:53.913996 7f09b03188c0 5 asok(0x3d94000) register_command config show hook 0x3d1c050 -27> 2015-09-09 10:48:53.914000 7f09b03188c0 5 asok(0x3d94000) register_command config set hook 0x3d1c050 -26> 2015-09-09 10:48:53.914003 7f09b03188c0 5 asok(0x3d94000) register_command config get hook 0x3d1c050 -25> 2015-09-09 10:48:53.914008 7f09b03188c0 5 asok(0x3d94000) register_command config diff hook 0x3d1c050 -24> 2015-09-09 10:48:53.914010 7f09b03188c0 5 asok(0x3d94000) register_command log flush hook 0x3d1c050 -23> 2015-09-09 10:48:53.914013 7f09b03188c0 5 asok(0x3d94000) register_command log dump hook 0x3d1c050 -22> 2015-09-09 10:48:53.914017 7f09b03188c0 5 asok(0x3d94000) register_command log reopen hook 0x3d1c050 -21> 2015-09-09 10:48:53.915965 7f09b03188c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 4525 -20> 2015-09-09 10:48:53.917343 7f09b03188c0 5 asok(0x3d94000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:48:53.917360 7f09b03188c0 5 asok(0x3d94000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:48:53.917388 7f09b03188c0 5 asok(0x3d94000) register_command 0 hook 0x3d180b8 -17> 2015-09-09 10:48:53.917398 7f09b03188c0 5 asok(0x3d94000) register_command version hook 0x3d180b8 -16> 2015-09-09 10:48:53.917401 7f09b03188c0 5 asok(0x3d94000) register_command git_version hook 0x3d180b8 -15> 2015-09-09 10:48:53.917406 7f09b03188c0 5 asok(0x3d94000) register_command help hook 0x3d1c0b0 -14> 2015-09-09 10:48:53.917408 7f09b03188c0 5 asok(0x3d94000) register_command get_command_descriptions hook 0x3d1c150 -13> 2015-09-09 10:48:53.917442 7f09ab698700 5 asok(0x3d94000) entry start -12> 2015-09-09 10:48:53.928777 7f09b03188c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:48:53.928842 7f09b03188c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:48:53.928846 7f09b03188c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:48:53.928939 7f09b03188c0 5 adding auth protocol: cephx -8> 2015-09-09 10:48:53.928945 7f09b03188c0 5 adding auth protocol: cephx -7> 2015-09-09 10:48:53.928966 7f09b03188c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:48:53.928968 7f09b03188c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:48:53.929039 7f09b03188c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:48:53.929376 7f09b03188c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:48:53.929383 7f09b03188c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:48:53.930847 7f09b03188c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:48:53.930854 7f09b03188c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:48:53.932659 7f09b03188c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f09b03188c0 time 2015-09-09 10:48:53.931246 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f09ad8a6ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:48:53.934696 7f09b03188c0 -1 *** Caught signal (Aborted) ** in thread 7f09b03188c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f09af41c340] 3: (gsignal()+0x39) [0x7f09ad8bbcc9] 4: (abort()+0x148) [0x7f09ad8bf0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f09ae1c6535] 6: (()+0x5e6d6) [0x7f09ae1c46d6] 7: (()+0x5e703) [0x7f09ae1c4703] 8: (()+0x5e922) [0x7f09ae1c4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f09ad8a6ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:48:53.934696 7f09b03188c0 -1 *** Caught signal (Aborted) ** in thread 7f09b03188c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f09af41c340] 3: (gsignal()+0x39) [0x7f09ad8bbcc9] 4: (abort()+0x148) [0x7f09ad8bf0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f09ae1c6535] 6: (()+0x5e6d6) [0x7f09ae1c46d6] 7: (()+0x5e703) [0x7f09ae1c4703] 8: (()+0x5e922) [0x7f09ae1c4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f09ad8a6ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:50:56.910744 7fb1eb0078c0 -1 wrote monmap to monmap 2015-09-09 10:52:55.411469 7f4da74618c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 5289 2015-09-09 10:52:55.430260 7f4da74618c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:52:55.430613 7f4da74618c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 10:52:55.430996 7f4da74618c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 10:52:55.431009 7f4da74618c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 10:52:55.432519 7f4da74618c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 10:52:55.434234 7f4da74618c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f4da74618c0 time 2015-09-09 10:52:55.432949 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f4da49efec5] 9: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 10:52:55.409412 7f4da74618c0 5 asok(0x5148000) register_command perfcounters_dump hook 0x50d0040 -34> 2015-09-09 10:52:55.409459 7f4da74618c0 5 asok(0x5148000) register_command 1 hook 0x50d0040 -33> 2015-09-09 10:52:55.409482 7f4da74618c0 5 asok(0x5148000) register_command perf dump hook 0x50d0040 -32> 2015-09-09 10:52:55.409499 7f4da74618c0 5 asok(0x5148000) register_command perfcounters_schema hook 0x50d0040 -31> 2015-09-09 10:52:55.409502 7f4da74618c0 5 asok(0x5148000) register_command 2 hook 0x50d0040 -30> 2015-09-09 10:52:55.409504 7f4da74618c0 5 asok(0x5148000) register_command perf schema hook 0x50d0040 -29> 2015-09-09 10:52:55.409511 7f4da74618c0 5 asok(0x5148000) register_command perf reset hook 0x50d0040 -28> 2015-09-09 10:52:55.409513 7f4da74618c0 5 asok(0x5148000) register_command config show hook 0x50d0040 -27> 2015-09-09 10:52:55.409515 7f4da74618c0 5 asok(0x5148000) register_command config set hook 0x50d0040 -26> 2015-09-09 10:52:55.409522 7f4da74618c0 5 asok(0x5148000) register_command config get hook 0x50d0040 -25> 2015-09-09 10:52:55.409523 7f4da74618c0 5 asok(0x5148000) register_command config diff hook 0x50d0040 -24> 2015-09-09 10:52:55.409525 7f4da74618c0 5 asok(0x5148000) register_command log flush hook 0x50d0040 -23> 2015-09-09 10:52:55.409527 7f4da74618c0 5 asok(0x5148000) register_command log dump hook 0x50d0040 -22> 2015-09-09 10:52:55.409532 7f4da74618c0 5 asok(0x5148000) register_command log reopen hook 0x50d0040 -21> 2015-09-09 10:52:55.411469 7f4da74618c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 5289 -20> 2015-09-09 10:52:55.413915 7f4da74618c0 5 asok(0x5148000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 10:52:55.413942 7f4da74618c0 5 asok(0x5148000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 10:52:55.413983 7f4da74618c0 5 asok(0x5148000) register_command 0 hook 0x50cc0b8 -17> 2015-09-09 10:52:55.413996 7f4da74618c0 5 asok(0x5148000) register_command version hook 0x50cc0b8 -16> 2015-09-09 10:52:55.414016 7f4da74618c0 5 asok(0x5148000) register_command git_version hook 0x50cc0b8 -15> 2015-09-09 10:52:55.414018 7f4da74618c0 5 asok(0x5148000) register_command help hook 0x50d0130 -14> 2015-09-09 10:52:55.414022 7f4da74618c0 5 asok(0x5148000) register_command get_command_descriptions hook 0x50d0120 -13> 2015-09-09 10:52:55.414063 7f4da27e1700 5 asok(0x5148000) entry start -12> 2015-09-09 10:52:55.430260 7f4da74618c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 10:52:55.430360 7f4da74618c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 10:52:55.430373 7f4da74618c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 10:52:55.430443 7f4da74618c0 5 adding auth protocol: cephx -8> 2015-09-09 10:52:55.430452 7f4da74618c0 5 adding auth protocol: cephx -7> 2015-09-09 10:52:55.430482 7f4da74618c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 10:52:55.430491 7f4da74618c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 10:52:55.430613 7f4da74618c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 10:52:55.430996 7f4da74618c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 10:52:55.431009 7f4da74618c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 10:52:55.432497 7f4da74618c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 10:52:55.432519 7f4da74618c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 10:52:55.434234 7f4da74618c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f4da74618c0 time 2015-09-09 10:52:55.432949 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f4da49efec5] 9: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 10:52:55.436534 7f4da74618c0 -1 *** Caught signal (Aborted) ** in thread 7f4da74618c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f4da6565340] 3: (gsignal()+0x39) [0x7f4da4a04cc9] 4: (abort()+0x148) [0x7f4da4a080d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f4da530f535] 6: (()+0x5e6d6) [0x7f4da530d6d6] 7: (()+0x5e703) [0x7f4da530d703] 8: (()+0x5e922) [0x7f4da530d922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f4da49efec5] 17: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 10:52:55.436534 7f4da74618c0 -1 *** Caught signal (Aborted) ** in thread 7f4da74618c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f4da6565340] 3: (gsignal()+0x39) [0x7f4da4a04cc9] 4: (abort()+0x148) [0x7f4da4a080d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f4da530f535] 6: (()+0x5e6d6) [0x7f4da530d6d6] 7: (()+0x5e703) [0x7f4da530d703] 8: (()+0x5e922) [0x7f4da530d922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f4da49efec5] 17: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.299210 7f8f6bdf88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7508 2015-09-09 11:04:09.313067 7f8f6bdf88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.313337 7f8f6bdf88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.313758 7f8f6bdf88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:09.313766 7f8f6bdf88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:09.315647 7f8f6bdf88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:09.317415 7f8f6bdf88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8f6bdf88c0 time 2015-09-09 11:04:09.316093 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8f69386ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:09.297037 7f8f6bdf88c0 5 asok(0x47a6000) register_command perfcounters_dump hook 0x472e050 -34> 2015-09-09 11:04:09.297066 7f8f6bdf88c0 5 asok(0x47a6000) register_command 1 hook 0x472e050 -33> 2015-09-09 11:04:09.297069 7f8f6bdf88c0 5 asok(0x47a6000) register_command perf dump hook 0x472e050 -32> 2015-09-09 11:04:09.297074 7f8f6bdf88c0 5 asok(0x47a6000) register_command perfcounters_schema hook 0x472e050 -31> 2015-09-09 11:04:09.297076 7f8f6bdf88c0 5 asok(0x47a6000) register_command 2 hook 0x472e050 -30> 2015-09-09 11:04:09.297077 7f8f6bdf88c0 5 asok(0x47a6000) register_command perf schema hook 0x472e050 -29> 2015-09-09 11:04:09.297080 7f8f6bdf88c0 5 asok(0x47a6000) register_command perf reset hook 0x472e050 -28> 2015-09-09 11:04:09.297084 7f8f6bdf88c0 5 asok(0x47a6000) register_command config show hook 0x472e050 -27> 2015-09-09 11:04:09.297093 7f8f6bdf88c0 5 asok(0x47a6000) register_command config set hook 0x472e050 -26> 2015-09-09 11:04:09.297096 7f8f6bdf88c0 5 asok(0x47a6000) register_command config get hook 0x472e050 -25> 2015-09-09 11:04:09.297098 7f8f6bdf88c0 5 asok(0x47a6000) register_command config diff hook 0x472e050 -24> 2015-09-09 11:04:09.297102 7f8f6bdf88c0 5 asok(0x47a6000) register_command log flush hook 0x472e050 -23> 2015-09-09 11:04:09.297103 7f8f6bdf88c0 5 asok(0x47a6000) register_command log dump hook 0x472e050 -22> 2015-09-09 11:04:09.297105 7f8f6bdf88c0 5 asok(0x47a6000) register_command log reopen hook 0x472e050 -21> 2015-09-09 11:04:09.299210 7f8f6bdf88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7508 -20> 2015-09-09 11:04:09.300526 7f8f6bdf88c0 5 asok(0x47a6000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:09.300542 7f8f6bdf88c0 5 asok(0x47a6000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:09.300637 7f8f6bdf88c0 5 asok(0x47a6000) register_command 0 hook 0x472a0b8 -17> 2015-09-09 11:04:09.300649 7f8f6bdf88c0 5 asok(0x47a6000) register_command version hook 0x472a0b8 -16> 2015-09-09 11:04:09.300653 7f8f6bdf88c0 5 asok(0x47a6000) register_command git_version hook 0x472a0b8 -15> 2015-09-09 11:04:09.300658 7f8f6bdf88c0 5 asok(0x47a6000) register_command help hook 0x472e0b0 -14> 2015-09-09 11:04:09.300664 7f8f6bdf88c0 5 asok(0x47a6000) register_command get_command_descriptions hook 0x472e150 -13> 2015-09-09 11:04:09.300699 7f8f67178700 5 asok(0x47a6000) entry start -12> 2015-09-09 11:04:09.313067 7f8f6bdf88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:09.313131 7f8f6bdf88c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:09.313135 7f8f6bdf88c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:09.313224 7f8f6bdf88c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:09.313231 7f8f6bdf88c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:09.313250 7f8f6bdf88c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:09.313252 7f8f6bdf88c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:09.313337 7f8f6bdf88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:09.313758 7f8f6bdf88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:09.313766 7f8f6bdf88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:09.315639 7f8f6bdf88c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:09.315647 7f8f6bdf88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:09.317415 7f8f6bdf88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8f6bdf88c0 time 2015-09-09 11:04:09.316093 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8f69386ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.319456 7f8f6bdf88c0 -1 *** Caught signal (Aborted) ** in thread 7f8f6bdf88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8f6aefc340] 3: (gsignal()+0x39) [0x7f8f6939bcc9] 4: (abort()+0x148) [0x7f8f6939f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8f69ca6535] 6: (()+0x5e6d6) [0x7f8f69ca46d6] 7: (()+0x5e703) [0x7f8f69ca4703] 8: (()+0x5e922) [0x7f8f69ca4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8f69386ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:09.319456 7f8f6bdf88c0 -1 *** Caught signal (Aborted) ** in thread 7f8f6bdf88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8f6aefc340] 3: (gsignal()+0x39) [0x7f8f6939bcc9] 4: (abort()+0x148) [0x7f8f6939f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8f69ca6535] 6: (()+0x5e6d6) [0x7f8f69ca46d6] 7: (()+0x5e703) [0x7f8f69ca4703] 8: (()+0x5e922) [0x7f8f69ca4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8f69386ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.447769 7ff15232c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7526 2015-09-09 11:04:09.461998 7ff15232c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.462251 7ff15232c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.462571 7ff15232c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:09.462580 7ff15232c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:09.463932 7ff15232c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:09.465583 7ff15232c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7ff15232c8c0 time 2015-09-09 11:04:09.464335 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7ff14f8baec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:09.445791 7ff15232c8c0 5 asok(0x4178000) register_command perfcounters_dump hook 0x4100050 -34> 2015-09-09 11:04:09.445833 7ff15232c8c0 5 asok(0x4178000) register_command 1 hook 0x4100050 -33> 2015-09-09 11:04:09.445837 7ff15232c8c0 5 asok(0x4178000) register_command perf dump hook 0x4100050 -32> 2015-09-09 11:04:09.445846 7ff15232c8c0 5 asok(0x4178000) register_command perfcounters_schema hook 0x4100050 -31> 2015-09-09 11:04:09.445852 7ff15232c8c0 5 asok(0x4178000) register_command 2 hook 0x4100050 -30> 2015-09-09 11:04:09.445854 7ff15232c8c0 5 asok(0x4178000) register_command perf schema hook 0x4100050 -29> 2015-09-09 11:04:09.445861 7ff15232c8c0 5 asok(0x4178000) register_command perf reset hook 0x4100050 -28> 2015-09-09 11:04:09.445863 7ff15232c8c0 5 asok(0x4178000) register_command config show hook 0x4100050 -27> 2015-09-09 11:04:09.445865 7ff15232c8c0 5 asok(0x4178000) register_command config set hook 0x4100050 -26> 2015-09-09 11:04:09.445868 7ff15232c8c0 5 asok(0x4178000) register_command config get hook 0x4100050 -25> 2015-09-09 11:04:09.445873 7ff15232c8c0 5 asok(0x4178000) register_command config diff hook 0x4100050 -24> 2015-09-09 11:04:09.445875 7ff15232c8c0 5 asok(0x4178000) register_command log flush hook 0x4100050 -23> 2015-09-09 11:04:09.445877 7ff15232c8c0 5 asok(0x4178000) register_command log dump hook 0x4100050 -22> 2015-09-09 11:04:09.445881 7ff15232c8c0 5 asok(0x4178000) register_command log reopen hook 0x4100050 -21> 2015-09-09 11:04:09.447769 7ff15232c8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7526 -20> 2015-09-09 11:04:09.449117 7ff15232c8c0 5 asok(0x4178000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:09.449133 7ff15232c8c0 5 asok(0x4178000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:09.449164 7ff15232c8c0 5 asok(0x4178000) register_command 0 hook 0x40fc0b8 -17> 2015-09-09 11:04:09.449173 7ff15232c8c0 5 asok(0x4178000) register_command version hook 0x40fc0b8 -16> 2015-09-09 11:04:09.449176 7ff15232c8c0 5 asok(0x4178000) register_command git_version hook 0x40fc0b8 -15> 2015-09-09 11:04:09.449181 7ff15232c8c0 5 asok(0x4178000) register_command help hook 0x41000b0 -14> 2015-09-09 11:04:09.449184 7ff15232c8c0 5 asok(0x4178000) register_command get_command_descriptions hook 0x4100150 -13> 2015-09-09 11:04:09.449220 7ff14d6ac700 5 asok(0x4178000) entry start -12> 2015-09-09 11:04:09.461998 7ff15232c8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:09.462063 7ff15232c8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:09.462068 7ff15232c8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:09.462157 7ff15232c8c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:09.462163 7ff15232c8c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:09.462183 7ff15232c8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:09.462186 7ff15232c8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:09.462251 7ff15232c8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:09.462571 7ff15232c8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:09.462580 7ff15232c8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:09.463925 7ff15232c8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:09.463932 7ff15232c8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:09.465583 7ff15232c8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7ff15232c8c0 time 2015-09-09 11:04:09.464335 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7ff14f8baec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.467573 7ff15232c8c0 -1 *** Caught signal (Aborted) ** in thread 7ff15232c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7ff151430340] 3: (gsignal()+0x39) [0x7ff14f8cfcc9] 4: (abort()+0x148) [0x7ff14f8d30d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff1501da535] 6: (()+0x5e6d6) [0x7ff1501d86d6] 7: (()+0x5e703) [0x7ff1501d8703] 8: (()+0x5e922) [0x7ff1501d8922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7ff14f8baec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:09.467573 7ff15232c8c0 -1 *** Caught signal (Aborted) ** in thread 7ff15232c8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7ff151430340] 3: (gsignal()+0x39) [0x7ff14f8cfcc9] 4: (abort()+0x148) [0x7ff14f8d30d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff1501da535] 6: (()+0x5e6d6) [0x7ff1501d86d6] 7: (()+0x5e703) [0x7ff1501d8703] 8: (()+0x5e922) [0x7ff1501d8922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7ff14f8baec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.596792 7f9c4ccf88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7544 2015-09-09 11:04:09.608942 7f9c4ccf88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.609205 7f9c4ccf88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.609527 7f9c4ccf88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:09.609535 7f9c4ccf88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:09.610890 7f9c4ccf88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:09.612845 7f9c4ccf88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f9c4ccf88c0 time 2015-09-09 11:04:09.611353 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f9c4a286ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:09.594818 7f9c4ccf88c0 5 asok(0x39a2000) register_command perfcounters_dump hook 0x392a050 -34> 2015-09-09 11:04:09.594849 7f9c4ccf88c0 5 asok(0x39a2000) register_command 1 hook 0x392a050 -33> 2015-09-09 11:04:09.594852 7f9c4ccf88c0 5 asok(0x39a2000) register_command perf dump hook 0x392a050 -32> 2015-09-09 11:04:09.594856 7f9c4ccf88c0 5 asok(0x39a2000) register_command perfcounters_schema hook 0x392a050 -31> 2015-09-09 11:04:09.594859 7f9c4ccf88c0 5 asok(0x39a2000) register_command 2 hook 0x392a050 -30> 2015-09-09 11:04:09.594860 7f9c4ccf88c0 5 asok(0x39a2000) register_command perf schema hook 0x392a050 -29> 2015-09-09 11:04:09.594862 7f9c4ccf88c0 5 asok(0x39a2000) register_command perf reset hook 0x392a050 -28> 2015-09-09 11:04:09.594864 7f9c4ccf88c0 5 asok(0x39a2000) register_command config show hook 0x392a050 -27> 2015-09-09 11:04:09.594866 7f9c4ccf88c0 5 asok(0x39a2000) register_command config set hook 0x392a050 -26> 2015-09-09 11:04:09.594868 7f9c4ccf88c0 5 asok(0x39a2000) register_command config get hook 0x392a050 -25> 2015-09-09 11:04:09.594870 7f9c4ccf88c0 5 asok(0x39a2000) register_command config diff hook 0x392a050 -24> 2015-09-09 11:04:09.594871 7f9c4ccf88c0 5 asok(0x39a2000) register_command log flush hook 0x392a050 -23> 2015-09-09 11:04:09.594873 7f9c4ccf88c0 5 asok(0x39a2000) register_command log dump hook 0x392a050 -22> 2015-09-09 11:04:09.594875 7f9c4ccf88c0 5 asok(0x39a2000) register_command log reopen hook 0x392a050 -21> 2015-09-09 11:04:09.596792 7f9c4ccf88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7544 -20> 2015-09-09 11:04:09.597986 7f9c4ccf88c0 5 asok(0x39a2000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:09.597995 7f9c4ccf88c0 5 asok(0x39a2000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:09.598023 7f9c4ccf88c0 5 asok(0x39a2000) register_command 0 hook 0x39260b8 -17> 2015-09-09 11:04:09.598028 7f9c4ccf88c0 5 asok(0x39a2000) register_command version hook 0x39260b8 -16> 2015-09-09 11:04:09.598030 7f9c4ccf88c0 5 asok(0x39a2000) register_command git_version hook 0x39260b8 -15> 2015-09-09 11:04:09.598033 7f9c4ccf88c0 5 asok(0x39a2000) register_command help hook 0x392a0a0 -14> 2015-09-09 11:04:09.598036 7f9c4ccf88c0 5 asok(0x39a2000) register_command get_command_descriptions hook 0x392a140 -13> 2015-09-09 11:04:09.598066 7f9c48078700 5 asok(0x39a2000) entry start -12> 2015-09-09 11:04:09.608942 7f9c4ccf88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:09.609016 7f9c4ccf88c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:09.609021 7f9c4ccf88c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:09.609112 7f9c4ccf88c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:09.609118 7f9c4ccf88c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:09.609137 7f9c4ccf88c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:09.609139 7f9c4ccf88c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:09.609205 7f9c4ccf88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:09.609527 7f9c4ccf88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:09.609535 7f9c4ccf88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:09.610883 7f9c4ccf88c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:09.610890 7f9c4ccf88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:09.612845 7f9c4ccf88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f9c4ccf88c0 time 2015-09-09 11:04:09.611353 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f9c4a286ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.614804 7f9c4ccf88c0 -1 *** Caught signal (Aborted) ** in thread 7f9c4ccf88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f9c4bdfc340] 3: (gsignal()+0x39) [0x7f9c4a29bcc9] 4: (abort()+0x148) [0x7f9c4a29f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f9c4aba6535] 6: (()+0x5e6d6) [0x7f9c4aba46d6] 7: (()+0x5e703) [0x7f9c4aba4703] 8: (()+0x5e922) [0x7f9c4aba4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f9c4a286ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:09.614804 7f9c4ccf88c0 -1 *** Caught signal (Aborted) ** in thread 7f9c4ccf88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f9c4bdfc340] 3: (gsignal()+0x39) [0x7f9c4a29bcc9] 4: (abort()+0x148) [0x7f9c4a29f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f9c4aba6535] 6: (()+0x5e6d6) [0x7f9c4aba46d6] 7: (()+0x5e703) [0x7f9c4aba4703] 8: (()+0x5e922) [0x7f9c4aba4922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f9c4a286ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.746383 7f6eb3fd98c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7562 2015-09-09 11:04:09.760966 7f6eb3fd98c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.761194 7f6eb3fd98c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.761521 7f6eb3fd98c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:09.761536 7f6eb3fd98c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:09.762977 7f6eb3fd98c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:09.764683 7f6eb3fd98c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f6eb3fd98c0 time 2015-09-09 11:04:09.763390 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f6eb1567ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:09.743787 7f6eb3fd98c0 5 asok(0x39c2000) register_command perfcounters_dump hook 0x394a050 -34> 2015-09-09 11:04:09.743821 7f6eb3fd98c0 5 asok(0x39c2000) register_command 1 hook 0x394a050 -33> 2015-09-09 11:04:09.743825 7f6eb3fd98c0 5 asok(0x39c2000) register_command perf dump hook 0x394a050 -32> 2015-09-09 11:04:09.743832 7f6eb3fd98c0 5 asok(0x39c2000) register_command perfcounters_schema hook 0x394a050 -31> 2015-09-09 11:04:09.743835 7f6eb3fd98c0 5 asok(0x39c2000) register_command 2 hook 0x394a050 -30> 2015-09-09 11:04:09.743837 7f6eb3fd98c0 5 asok(0x39c2000) register_command perf schema hook 0x394a050 -29> 2015-09-09 11:04:09.743840 7f6eb3fd98c0 5 asok(0x39c2000) register_command perf reset hook 0x394a050 -28> 2015-09-09 11:04:09.743842 7f6eb3fd98c0 5 asok(0x39c2000) register_command config show hook 0x394a050 -27> 2015-09-09 11:04:09.743845 7f6eb3fd98c0 5 asok(0x39c2000) register_command config set hook 0x394a050 -26> 2015-09-09 11:04:09.743847 7f6eb3fd98c0 5 asok(0x39c2000) register_command config get hook 0x394a050 -25> 2015-09-09 11:04:09.743849 7f6eb3fd98c0 5 asok(0x39c2000) register_command config diff hook 0x394a050 -24> 2015-09-09 11:04:09.743851 7f6eb3fd98c0 5 asok(0x39c2000) register_command log flush hook 0x394a050 -23> 2015-09-09 11:04:09.743853 7f6eb3fd98c0 5 asok(0x39c2000) register_command log dump hook 0x394a050 -22> 2015-09-09 11:04:09.743856 7f6eb3fd98c0 5 asok(0x39c2000) register_command log reopen hook 0x394a050 -21> 2015-09-09 11:04:09.746383 7f6eb3fd98c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7562 -20> 2015-09-09 11:04:09.748393 7f6eb3fd98c0 5 asok(0x39c2000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:09.748409 7f6eb3fd98c0 5 asok(0x39c2000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:09.748440 7f6eb3fd98c0 5 asok(0x39c2000) register_command 0 hook 0x39460b8 -17> 2015-09-09 11:04:09.748451 7f6eb3fd98c0 5 asok(0x39c2000) register_command version hook 0x39460b8 -16> 2015-09-09 11:04:09.748454 7f6eb3fd98c0 5 asok(0x39c2000) register_command git_version hook 0x39460b8 -15> 2015-09-09 11:04:09.748459 7f6eb3fd98c0 5 asok(0x39c2000) register_command help hook 0x394a0b0 -14> 2015-09-09 11:04:09.748462 7f6eb3fd98c0 5 asok(0x39c2000) register_command get_command_descriptions hook 0x394a150 -13> 2015-09-09 11:04:09.748514 7f6eaf359700 5 asok(0x39c2000) entry start -12> 2015-09-09 11:04:09.760966 7f6eb3fd98c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:09.761034 7f6eb3fd98c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:09.761043 7f6eb3fd98c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:09.761102 7f6eb3fd98c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:09.761109 7f6eb3fd98c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:09.761128 7f6eb3fd98c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:09.761131 7f6eb3fd98c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:09.761194 7f6eb3fd98c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:09.761521 7f6eb3fd98c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:09.761536 7f6eb3fd98c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:09.762956 7f6eb3fd98c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:09.762977 7f6eb3fd98c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:09.764683 7f6eb3fd98c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f6eb3fd98c0 time 2015-09-09 11:04:09.763390 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f6eb1567ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.766654 7f6eb3fd98c0 -1 *** Caught signal (Aborted) ** in thread 7f6eb3fd98c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f6eb30dd340] 3: (gsignal()+0x39) [0x7f6eb157ccc9] 4: (abort()+0x148) [0x7f6eb15800d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f6eb1e87535] 6: (()+0x5e6d6) [0x7f6eb1e856d6] 7: (()+0x5e703) [0x7f6eb1e85703] 8: (()+0x5e922) [0x7f6eb1e85922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f6eb1567ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:09.766654 7f6eb3fd98c0 -1 *** Caught signal (Aborted) ** in thread 7f6eb3fd98c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f6eb30dd340] 3: (gsignal()+0x39) [0x7f6eb157ccc9] 4: (abort()+0x148) [0x7f6eb15800d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f6eb1e87535] 6: (()+0x5e6d6) [0x7f6eb1e856d6] 7: (()+0x5e703) [0x7f6eb1e85703] 8: (()+0x5e922) [0x7f6eb1e85922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f6eb1567ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.894840 7f05417d28c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7580 2015-09-09 11:04:09.909001 7f05417d28c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.909243 7f05417d28c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:09.909561 7f05417d28c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:09.909569 7f05417d28c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:09.910947 7f05417d28c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:09.912648 7f05417d28c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f05417d28c0 time 2015-09-09 11:04:09.911342 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f053ed60ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:09.892190 7f05417d28c0 5 asok(0x47c6000) register_command perfcounters_dump hook 0x474e050 -34> 2015-09-09 11:04:09.892235 7f05417d28c0 5 asok(0x47c6000) register_command 1 hook 0x474e050 -33> 2015-09-09 11:04:09.892245 7f05417d28c0 5 asok(0x47c6000) register_command perf dump hook 0x474e050 -32> 2015-09-09 11:04:09.892252 7f05417d28c0 5 asok(0x47c6000) register_command perfcounters_schema hook 0x474e050 -31> 2015-09-09 11:04:09.892261 7f05417d28c0 5 asok(0x47c6000) register_command 2 hook 0x474e050 -30> 2015-09-09 11:04:09.892264 7f05417d28c0 5 asok(0x47c6000) register_command perf schema hook 0x474e050 -29> 2015-09-09 11:04:09.892272 7f05417d28c0 5 asok(0x47c6000) register_command perf reset hook 0x474e050 -28> 2015-09-09 11:04:09.892274 7f05417d28c0 5 asok(0x47c6000) register_command config show hook 0x474e050 -27> 2015-09-09 11:04:09.892282 7f05417d28c0 5 asok(0x47c6000) register_command config set hook 0x474e050 -26> 2015-09-09 11:04:09.892284 7f05417d28c0 5 asok(0x47c6000) register_command config get hook 0x474e050 -25> 2015-09-09 11:04:09.892290 7f05417d28c0 5 asok(0x47c6000) register_command config diff hook 0x474e050 -24> 2015-09-09 11:04:09.892292 7f05417d28c0 5 asok(0x47c6000) register_command log flush hook 0x474e050 -23> 2015-09-09 11:04:09.892295 7f05417d28c0 5 asok(0x47c6000) register_command log dump hook 0x474e050 -22> 2015-09-09 11:04:09.892302 7f05417d28c0 5 asok(0x47c6000) register_command log reopen hook 0x474e050 -21> 2015-09-09 11:04:09.894840 7f05417d28c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7580 -20> 2015-09-09 11:04:09.896422 7f05417d28c0 5 asok(0x47c6000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:09.896443 7f05417d28c0 5 asok(0x47c6000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:09.896497 7f05417d28c0 5 asok(0x47c6000) register_command 0 hook 0x474a0b8 -17> 2015-09-09 11:04:09.896509 7f05417d28c0 5 asok(0x47c6000) register_command version hook 0x474a0b8 -16> 2015-09-09 11:04:09.896517 7f05417d28c0 5 asok(0x47c6000) register_command git_version hook 0x474a0b8 -15> 2015-09-09 11:04:09.896521 7f05417d28c0 5 asok(0x47c6000) register_command help hook 0x474e0b0 -14> 2015-09-09 11:04:09.896532 7f05417d28c0 5 asok(0x47c6000) register_command get_command_descriptions hook 0x474e150 -13> 2015-09-09 11:04:09.896626 7f053cb52700 5 asok(0x47c6000) entry start -12> 2015-09-09 11:04:09.909001 7f05417d28c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:09.909064 7f05417d28c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:09.909068 7f05417d28c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:09.909150 7f05417d28c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:09.909156 7f05417d28c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:09.909176 7f05417d28c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:09.909178 7f05417d28c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:09.909243 7f05417d28c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:09.909561 7f05417d28c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:09.909569 7f05417d28c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:09.910941 7f05417d28c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:09.910947 7f05417d28c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:09.912648 7f05417d28c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f05417d28c0 time 2015-09-09 11:04:09.911342 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f053ed60ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:09.914590 7f05417d28c0 -1 *** Caught signal (Aborted) ** in thread 7f05417d28c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f05408d6340] 3: (gsignal()+0x39) [0x7f053ed75cc9] 4: (abort()+0x148) [0x7f053ed790d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f053f680535] 6: (()+0x5e6d6) [0x7f053f67e6d6] 7: (()+0x5e703) [0x7f053f67e703] 8: (()+0x5e922) [0x7f053f67e922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f053ed60ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:09.914590 7f05417d28c0 -1 *** Caught signal (Aborted) ** in thread 7f05417d28c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f05408d6340] 3: (gsignal()+0x39) [0x7f053ed75cc9] 4: (abort()+0x148) [0x7f053ed790d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f053f680535] 6: (()+0x5e6d6) [0x7f053f67e6d6] 7: (()+0x5e703) [0x7f053f67e703] 8: (()+0x5e922) [0x7f053f67e922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f053ed60ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:10.039651 7f364b0228c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7598 2015-09-09 11:04:10.052033 7f364b0228c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:10.052281 7f364b0228c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 11:04:10.052643 7f364b0228c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 11:04:10.052660 7f364b0228c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 11:04:10.054039 7f364b0228c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 11:04:10.055709 7f364b0228c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f364b0228c0 time 2015-09-09 11:04:10.054451 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f36485b0ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 11:04:10.037651 7f364b0228c0 5 asok(0x3d76000) register_command perfcounters_dump hook 0x3cfe050 -34> 2015-09-09 11:04:10.037691 7f364b0228c0 5 asok(0x3d76000) register_command 1 hook 0x3cfe050 -33> 2015-09-09 11:04:10.037694 7f364b0228c0 5 asok(0x3d76000) register_command perf dump hook 0x3cfe050 -32> 2015-09-09 11:04:10.037699 7f364b0228c0 5 asok(0x3d76000) register_command perfcounters_schema hook 0x3cfe050 -31> 2015-09-09 11:04:10.037701 7f364b0228c0 5 asok(0x3d76000) register_command 2 hook 0x3cfe050 -30> 2015-09-09 11:04:10.037703 7f364b0228c0 5 asok(0x3d76000) register_command perf schema hook 0x3cfe050 -29> 2015-09-09 11:04:10.037705 7f364b0228c0 5 asok(0x3d76000) register_command perf reset hook 0x3cfe050 -28> 2015-09-09 11:04:10.037707 7f364b0228c0 5 asok(0x3d76000) register_command config show hook 0x3cfe050 -27> 2015-09-09 11:04:10.037709 7f364b0228c0 5 asok(0x3d76000) register_command config set hook 0x3cfe050 -26> 2015-09-09 11:04:10.037711 7f364b0228c0 5 asok(0x3d76000) register_command config get hook 0x3cfe050 -25> 2015-09-09 11:04:10.037713 7f364b0228c0 5 asok(0x3d76000) register_command config diff hook 0x3cfe050 -24> 2015-09-09 11:04:10.037732 7f364b0228c0 5 asok(0x3d76000) register_command log flush hook 0x3cfe050 -23> 2015-09-09 11:04:10.037738 7f364b0228c0 5 asok(0x3d76000) register_command log dump hook 0x3cfe050 -22> 2015-09-09 11:04:10.037740 7f364b0228c0 5 asok(0x3d76000) register_command log reopen hook 0x3cfe050 -21> 2015-09-09 11:04:10.039651 7f364b0228c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 7598 -20> 2015-09-09 11:04:10.040954 7f364b0228c0 5 asok(0x3d76000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 11:04:10.040970 7f364b0228c0 5 asok(0x3d76000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 11:04:10.041003 7f364b0228c0 5 asok(0x3d76000) register_command 0 hook 0x3cfa0b8 -17> 2015-09-09 11:04:10.041013 7f364b0228c0 5 asok(0x3d76000) register_command version hook 0x3cfa0b8 -16> 2015-09-09 11:04:10.041018 7f364b0228c0 5 asok(0x3d76000) register_command git_version hook 0x3cfa0b8 -15> 2015-09-09 11:04:10.041025 7f364b0228c0 5 asok(0x3d76000) register_command help hook 0x3cfe0b0 -14> 2015-09-09 11:04:10.041028 7f364b0228c0 5 asok(0x3d76000) register_command get_command_descriptions hook 0x3cfe150 -13> 2015-09-09 11:04:10.041065 7f36463a2700 5 asok(0x3d76000) entry start -12> 2015-09-09 11:04:10.052033 7f364b0228c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 11:04:10.052108 7f364b0228c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 11:04:10.052117 7f364b0228c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 11:04:10.052181 7f364b0228c0 5 adding auth protocol: cephx -8> 2015-09-09 11:04:10.052190 7f364b0228c0 5 adding auth protocol: cephx -7> 2015-09-09 11:04:10.052210 7f364b0228c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 11:04:10.052216 7f364b0228c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 11:04:10.052281 7f364b0228c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 11:04:10.052643 7f364b0228c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 11:04:10.052660 7f364b0228c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 11:04:10.054023 7f364b0228c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 11:04:10.054039 7f364b0228c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 11:04:10.055709 7f364b0228c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f364b0228c0 time 2015-09-09 11:04:10.054451 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f36485b0ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 11:04:10.057685 7f364b0228c0 -1 *** Caught signal (Aborted) ** in thread 7f364b0228c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f364a126340] 3: (gsignal()+0x39) [0x7f36485c5cc9] 4: (abort()+0x148) [0x7f36485c90d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f3648ed0535] 6: (()+0x5e6d6) [0x7f3648ece6d6] 7: (()+0x5e703) [0x7f3648ece703] 8: (()+0x5e922) [0x7f3648ece922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f36485b0ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 11:04:10.057685 7f364b0228c0 -1 *** Caught signal (Aborted) ** in thread 7f364b0228c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f364a126340] 3: (gsignal()+0x39) [0x7f36485c5cc9] 4: (abort()+0x148) [0x7f36485c90d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f3648ed0535] 6: (()+0x5e6d6) [0x7f3648ece6d6] 7: (()+0x5e703) [0x7f3648ece703] 8: (()+0x5e922) [0x7f3648ece922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f36485b0ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.424202 7f8d49fc78c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1629 2015-09-09 13:45:00.446048 7f8d49fc78c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.446235 7f8d49fc78c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.446564 7f8d49fc78c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:00.446572 7f8d49fc78c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:00.449591 7f8d49fc78c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:00.451542 7f8d49fc78c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8d49fc78c0 time 2015-09-09 13:45:00.450362 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8d47555ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:00.422251 7f8d49fc78c0 5 asok(0x4faa000) register_command perfcounters_dump hook 0x4f32050 -34> 2015-09-09 13:45:00.422280 7f8d49fc78c0 5 asok(0x4faa000) register_command 1 hook 0x4f32050 -33> 2015-09-09 13:45:00.422283 7f8d49fc78c0 5 asok(0x4faa000) register_command perf dump hook 0x4f32050 -32> 2015-09-09 13:45:00.422289 7f8d49fc78c0 5 asok(0x4faa000) register_command perfcounters_schema hook 0x4f32050 -31> 2015-09-09 13:45:00.422291 7f8d49fc78c0 5 asok(0x4faa000) register_command 2 hook 0x4f32050 -30> 2015-09-09 13:45:00.422292 7f8d49fc78c0 5 asok(0x4faa000) register_command perf schema hook 0x4f32050 -29> 2015-09-09 13:45:00.422294 7f8d49fc78c0 5 asok(0x4faa000) register_command perf reset hook 0x4f32050 -28> 2015-09-09 13:45:00.422296 7f8d49fc78c0 5 asok(0x4faa000) register_command config show hook 0x4f32050 -27> 2015-09-09 13:45:00.422298 7f8d49fc78c0 5 asok(0x4faa000) register_command config set hook 0x4f32050 -26> 2015-09-09 13:45:00.422300 7f8d49fc78c0 5 asok(0x4faa000) register_command config get hook 0x4f32050 -25> 2015-09-09 13:45:00.422302 7f8d49fc78c0 5 asok(0x4faa000) register_command config diff hook 0x4f32050 -24> 2015-09-09 13:45:00.422304 7f8d49fc78c0 5 asok(0x4faa000) register_command log flush hook 0x4f32050 -23> 2015-09-09 13:45:00.422305 7f8d49fc78c0 5 asok(0x4faa000) register_command log dump hook 0x4f32050 -22> 2015-09-09 13:45:00.422307 7f8d49fc78c0 5 asok(0x4faa000) register_command log reopen hook 0x4f32050 -21> 2015-09-09 13:45:00.424202 7f8d49fc78c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1629 -20> 2015-09-09 13:45:00.429648 7f8d49fc78c0 5 asok(0x4faa000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:00.429661 7f8d49fc78c0 5 asok(0x4faa000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:00.429692 7f8d49fc78c0 5 asok(0x4faa000) register_command 0 hook 0x4f2e0b8 -17> 2015-09-09 13:45:00.429696 7f8d49fc78c0 5 asok(0x4faa000) register_command version hook 0x4f2e0b8 -16> 2015-09-09 13:45:00.429700 7f8d49fc78c0 5 asok(0x4faa000) register_command git_version hook 0x4f2e0b8 -15> 2015-09-09 13:45:00.429703 7f8d49fc78c0 5 asok(0x4faa000) register_command help hook 0x4f320b0 -14> 2015-09-09 13:45:00.429705 7f8d49fc78c0 5 asok(0x4faa000) register_command get_command_descriptions hook 0x4f32150 -13> 2015-09-09 13:45:00.429754 7f8d45347700 5 asok(0x4faa000) entry start -12> 2015-09-09 13:45:00.446048 7f8d49fc78c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:00.446105 7f8d49fc78c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:00.446110 7f8d49fc78c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:00.446151 7f8d49fc78c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:00.446155 7f8d49fc78c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:00.446172 7f8d49fc78c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:00.446175 7f8d49fc78c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:00.446235 7f8d49fc78c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:00.446564 7f8d49fc78c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:00.446572 7f8d49fc78c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:00.449584 7f8d49fc78c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:00.449591 7f8d49fc78c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:00.451542 7f8d49fc78c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f8d49fc78c0 time 2015-09-09 13:45:00.450362 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f8d47555ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.453490 7f8d49fc78c0 -1 *** Caught signal (Aborted) ** in thread 7f8d49fc78c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8d490cb340] 3: (gsignal()+0x39) [0x7f8d4756acc9] 4: (abort()+0x148) [0x7f8d4756e0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8d47e75535] 6: (()+0x5e6d6) [0x7f8d47e736d6] 7: (()+0x5e703) [0x7f8d47e73703] 8: (()+0x5e922) [0x7f8d47e73922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8d47555ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:00.453490 7f8d49fc78c0 -1 *** Caught signal (Aborted) ** in thread 7f8d49fc78c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f8d490cb340] 3: (gsignal()+0x39) [0x7f8d4756acc9] 4: (abort()+0x148) [0x7f8d4756e0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f8d47e75535] 6: (()+0x5e6d6) [0x7f8d47e736d6] 7: (()+0x5e703) [0x7f8d47e73703] 8: (()+0x5e922) [0x7f8d47e73922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f8d47555ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.624510 7f65f3ae88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1750 2015-09-09 13:45:00.637567 7f65f3ae88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.637789 7f65f3ae88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.638395 7f65f3ae88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:00.638408 7f65f3ae88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:00.639623 7f65f3ae88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:00.641294 7f65f3ae88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f65f3ae88c0 time 2015-09-09 13:45:00.640034 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f65f1076ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:00.622618 7f65f3ae88c0 5 asok(0x5122000) register_command perfcounters_dump hook 0x50aa050 -34> 2015-09-09 13:45:00.622645 7f65f3ae88c0 5 asok(0x5122000) register_command 1 hook 0x50aa050 -33> 2015-09-09 13:45:00.622651 7f65f3ae88c0 5 asok(0x5122000) register_command perf dump hook 0x50aa050 -32> 2015-09-09 13:45:00.622655 7f65f3ae88c0 5 asok(0x5122000) register_command perfcounters_schema hook 0x50aa050 -31> 2015-09-09 13:45:00.622658 7f65f3ae88c0 5 asok(0x5122000) register_command 2 hook 0x50aa050 -30> 2015-09-09 13:45:00.622659 7f65f3ae88c0 5 asok(0x5122000) register_command perf schema hook 0x50aa050 -29> 2015-09-09 13:45:00.622661 7f65f3ae88c0 5 asok(0x5122000) register_command perf reset hook 0x50aa050 -28> 2015-09-09 13:45:00.622663 7f65f3ae88c0 5 asok(0x5122000) register_command config show hook 0x50aa050 -27> 2015-09-09 13:45:00.622665 7f65f3ae88c0 5 asok(0x5122000) register_command config set hook 0x50aa050 -26> 2015-09-09 13:45:00.622667 7f65f3ae88c0 5 asok(0x5122000) register_command config get hook 0x50aa050 -25> 2015-09-09 13:45:00.622669 7f65f3ae88c0 5 asok(0x5122000) register_command config diff hook 0x50aa050 -24> 2015-09-09 13:45:00.622670 7f65f3ae88c0 5 asok(0x5122000) register_command log flush hook 0x50aa050 -23> 2015-09-09 13:45:00.622672 7f65f3ae88c0 5 asok(0x5122000) register_command log dump hook 0x50aa050 -22> 2015-09-09 13:45:00.622674 7f65f3ae88c0 5 asok(0x5122000) register_command log reopen hook 0x50aa050 -21> 2015-09-09 13:45:00.624510 7f65f3ae88c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1750 -20> 2015-09-09 13:45:00.625657 7f65f3ae88c0 5 asok(0x5122000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:00.625665 7f65f3ae88c0 5 asok(0x5122000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:00.625690 7f65f3ae88c0 5 asok(0x5122000) register_command 0 hook 0x50a60b8 -17> 2015-09-09 13:45:00.625694 7f65f3ae88c0 5 asok(0x5122000) register_command version hook 0x50a60b8 -16> 2015-09-09 13:45:00.625696 7f65f3ae88c0 5 asok(0x5122000) register_command git_version hook 0x50a60b8 -15> 2015-09-09 13:45:00.625699 7f65f3ae88c0 5 asok(0x5122000) register_command help hook 0x50aa0a0 -14> 2015-09-09 13:45:00.625701 7f65f3ae88c0 5 asok(0x5122000) register_command get_command_descriptions hook 0x50aa140 -13> 2015-09-09 13:45:00.633971 7f65eee68700 5 asok(0x5122000) entry start -12> 2015-09-09 13:45:00.637567 7f65f3ae88c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:00.637626 7f65f3ae88c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:00.637630 7f65f3ae88c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:00.637686 7f65f3ae88c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:00.637692 7f65f3ae88c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:00.637718 7f65f3ae88c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:00.637723 7f65f3ae88c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:00.637789 7f65f3ae88c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:00.638395 7f65f3ae88c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:00.638408 7f65f3ae88c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:00.639616 7f65f3ae88c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:00.639623 7f65f3ae88c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:00.641294 7f65f3ae88c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f65f3ae88c0 time 2015-09-09 13:45:00.640034 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f65f1076ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.643214 7f65f3ae88c0 -1 *** Caught signal (Aborted) ** in thread 7f65f3ae88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f65f2bec340] 3: (gsignal()+0x39) [0x7f65f108bcc9] 4: (abort()+0x148) [0x7f65f108f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f65f1996535] 6: (()+0x5e6d6) [0x7f65f19946d6] 7: (()+0x5e703) [0x7f65f1994703] 8: (()+0x5e922) [0x7f65f1994922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f65f1076ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:00.643214 7f65f3ae88c0 -1 *** Caught signal (Aborted) ** in thread 7f65f3ae88c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f65f2bec340] 3: (gsignal()+0x39) [0x7f65f108bcc9] 4: (abort()+0x148) [0x7f65f108f0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f65f1996535] 6: (()+0x5e6d6) [0x7f65f19946d6] 7: (()+0x5e703) [0x7f65f1994703] 8: (()+0x5e922) [0x7f65f1994922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f65f1076ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.767420 7fa366b198c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1808 2015-09-09 13:45:00.779023 7fa366b198c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.779254 7fa366b198c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.779577 7fa366b198c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:00.779591 7fa366b198c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:00.780933 7fa366b198c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:00.782514 7fa366b198c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fa366b198c0 time 2015-09-09 13:45:00.781311 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fa3640a7ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:00.765561 7fa366b198c0 5 asok(0x4ea4000) register_command perfcounters_dump hook 0x4e2c050 -34> 2015-09-09 13:45:00.765587 7fa366b198c0 5 asok(0x4ea4000) register_command 1 hook 0x4e2c050 -33> 2015-09-09 13:45:00.765590 7fa366b198c0 5 asok(0x4ea4000) register_command perf dump hook 0x4e2c050 -32> 2015-09-09 13:45:00.765594 7fa366b198c0 5 asok(0x4ea4000) register_command perfcounters_schema hook 0x4e2c050 -31> 2015-09-09 13:45:00.765596 7fa366b198c0 5 asok(0x4ea4000) register_command 2 hook 0x4e2c050 -30> 2015-09-09 13:45:00.765598 7fa366b198c0 5 asok(0x4ea4000) register_command perf schema hook 0x4e2c050 -29> 2015-09-09 13:45:00.765600 7fa366b198c0 5 asok(0x4ea4000) register_command perf reset hook 0x4e2c050 -28> 2015-09-09 13:45:00.765602 7fa366b198c0 5 asok(0x4ea4000) register_command config show hook 0x4e2c050 -27> 2015-09-09 13:45:00.765604 7fa366b198c0 5 asok(0x4ea4000) register_command config set hook 0x4e2c050 -26> 2015-09-09 13:45:00.765606 7fa366b198c0 5 asok(0x4ea4000) register_command config get hook 0x4e2c050 -25> 2015-09-09 13:45:00.765607 7fa366b198c0 5 asok(0x4ea4000) register_command config diff hook 0x4e2c050 -24> 2015-09-09 13:45:00.765609 7fa366b198c0 5 asok(0x4ea4000) register_command log flush hook 0x4e2c050 -23> 2015-09-09 13:45:00.765611 7fa366b198c0 5 asok(0x4ea4000) register_command log dump hook 0x4e2c050 -22> 2015-09-09 13:45:00.765612 7fa366b198c0 5 asok(0x4ea4000) register_command log reopen hook 0x4e2c050 -21> 2015-09-09 13:45:00.767420 7fa366b198c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1808 -20> 2015-09-09 13:45:00.768607 7fa366b198c0 5 asok(0x4ea4000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:00.768620 7fa366b198c0 5 asok(0x4ea4000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:00.768643 7fa366b198c0 5 asok(0x4ea4000) register_command 0 hook 0x4e280b8 -17> 2015-09-09 13:45:00.768646 7fa366b198c0 5 asok(0x4ea4000) register_command version hook 0x4e280b8 -16> 2015-09-09 13:45:00.768649 7fa366b198c0 5 asok(0x4ea4000) register_command git_version hook 0x4e280b8 -15> 2015-09-09 13:45:00.768651 7fa366b198c0 5 asok(0x4ea4000) register_command help hook 0x4e2c0b0 -14> 2015-09-09 13:45:00.768653 7fa366b198c0 5 asok(0x4ea4000) register_command get_command_descriptions hook 0x4e2c150 -13> 2015-09-09 13:45:00.768854 7fa361e99700 5 asok(0x4ea4000) entry start -12> 2015-09-09 13:45:00.779023 7fa366b198c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:00.779081 7fa366b198c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:00.779086 7fa366b198c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:00.779158 7fa366b198c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:00.779167 7fa366b198c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:00.779187 7fa366b198c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:00.779192 7fa366b198c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:00.779254 7fa366b198c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:00.779577 7fa366b198c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:00.779591 7fa366b198c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:00.780918 7fa366b198c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:00.780933 7fa366b198c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:00.782514 7fa366b198c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fa366b198c0 time 2015-09-09 13:45:00.781311 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fa3640a7ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.784434 7fa366b198c0 -1 *** Caught signal (Aborted) ** in thread 7fa366b198c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fa365c1d340] 3: (gsignal()+0x39) [0x7fa3640bccc9] 4: (abort()+0x148) [0x7fa3640c00d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa3649c7535] 6: (()+0x5e6d6) [0x7fa3649c56d6] 7: (()+0x5e703) [0x7fa3649c5703] 8: (()+0x5e922) [0x7fa3649c5922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fa3640a7ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:00.784434 7fa366b198c0 -1 *** Caught signal (Aborted) ** in thread 7fa366b198c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fa365c1d340] 3: (gsignal()+0x39) [0x7fa3640bccc9] 4: (abort()+0x148) [0x7fa3640c00d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fa3649c7535] 6: (()+0x5e6d6) [0x7fa3649c56d6] 7: (()+0x5e703) [0x7fa3649c5703] 8: (()+0x5e922) [0x7fa3649c5922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fa3640a7ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.914217 7f2e52ecb8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1838 2015-09-09 13:45:00.931578 7f2e52ecb8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.931903 7f2e52ecb8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:00.932439 7f2e52ecb8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:00.932452 7f2e52ecb8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:00.934922 7f2e52ecb8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:00.937136 7f2e52ecb8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f2e52ecb8c0 time 2015-09-09 13:45:00.935567 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f2e50459ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:00.910817 7f2e52ecb8c0 5 asok(0x371e000) register_command perfcounters_dump hook 0x36a6050 -34> 2015-09-09 13:45:00.910864 7f2e52ecb8c0 5 asok(0x371e000) register_command 1 hook 0x36a6050 -33> 2015-09-09 13:45:00.910875 7f2e52ecb8c0 5 asok(0x371e000) register_command perf dump hook 0x36a6050 -32> 2015-09-09 13:45:00.910890 7f2e52ecb8c0 5 asok(0x371e000) register_command perfcounters_schema hook 0x36a6050 -31> 2015-09-09 13:45:00.910900 7f2e52ecb8c0 5 asok(0x371e000) register_command 2 hook 0x36a6050 -30> 2015-09-09 13:45:00.910910 7f2e52ecb8c0 5 asok(0x371e000) register_command perf schema hook 0x36a6050 -29> 2015-09-09 13:45:00.910913 7f2e52ecb8c0 5 asok(0x371e000) register_command perf reset hook 0x36a6050 -28> 2015-09-09 13:45:00.910923 7f2e52ecb8c0 5 asok(0x371e000) register_command config show hook 0x36a6050 -27> 2015-09-09 13:45:00.910926 7f2e52ecb8c0 5 asok(0x371e000) register_command config set hook 0x36a6050 -26> 2015-09-09 13:45:00.910930 7f2e52ecb8c0 5 asok(0x371e000) register_command config get hook 0x36a6050 -25> 2015-09-09 13:45:00.910932 7f2e52ecb8c0 5 asok(0x371e000) register_command config diff hook 0x36a6050 -24> 2015-09-09 13:45:00.910935 7f2e52ecb8c0 5 asok(0x371e000) register_command log flush hook 0x36a6050 -23> 2015-09-09 13:45:00.910937 7f2e52ecb8c0 5 asok(0x371e000) register_command log dump hook 0x36a6050 -22> 2015-09-09 13:45:00.910940 7f2e52ecb8c0 5 asok(0x371e000) register_command log reopen hook 0x36a6050 -21> 2015-09-09 13:45:00.914217 7f2e52ecb8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1838 -20> 2015-09-09 13:45:00.915933 7f2e52ecb8c0 5 asok(0x371e000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:00.915954 7f2e52ecb8c0 5 asok(0x371e000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:00.915994 7f2e52ecb8c0 5 asok(0x371e000) register_command 0 hook 0x36a20b8 -17> 2015-09-09 13:45:00.916007 7f2e52ecb8c0 5 asok(0x371e000) register_command version hook 0x36a20b8 -16> 2015-09-09 13:45:00.916018 7f2e52ecb8c0 5 asok(0x371e000) register_command git_version hook 0x36a20b8 -15> 2015-09-09 13:45:00.916023 7f2e52ecb8c0 5 asok(0x371e000) register_command help hook 0x36a60b0 -14> 2015-09-09 13:45:00.916033 7f2e52ecb8c0 5 asok(0x371e000) register_command get_command_descriptions hook 0x36a6150 -13> 2015-09-09 13:45:00.916076 7f2e4e24b700 5 asok(0x371e000) entry start -12> 2015-09-09 13:45:00.931578 7f2e52ecb8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:00.931656 7f2e52ecb8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:00.931663 7f2e52ecb8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:00.931781 7f2e52ecb8c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:00.931790 7f2e52ecb8c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:00.931815 7f2e52ecb8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:00.931819 7f2e52ecb8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:00.931903 7f2e52ecb8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:00.932439 7f2e52ecb8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:00.932452 7f2e52ecb8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:00.934891 7f2e52ecb8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:00.934922 7f2e52ecb8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:00.937136 7f2e52ecb8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f2e52ecb8c0 time 2015-09-09 13:45:00.935567 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f2e50459ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:00.939739 7f2e52ecb8c0 -1 *** Caught signal (Aborted) ** in thread 7f2e52ecb8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f2e51fcf340] 3: (gsignal()+0x39) [0x7f2e5046ecc9] 4: (abort()+0x148) [0x7f2e504720d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f2e50d79535] 6: (()+0x5e6d6) [0x7f2e50d776d6] 7: (()+0x5e703) [0x7f2e50d77703] 8: (()+0x5e922) [0x7f2e50d77922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f2e50459ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:00.939739 7f2e52ecb8c0 -1 *** Caught signal (Aborted) ** in thread 7f2e52ecb8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f2e51fcf340] 3: (gsignal()+0x39) [0x7f2e5046ecc9] 4: (abort()+0x148) [0x7f2e504720d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f2e50d79535] 6: (()+0x5e6d6) [0x7f2e50d776d6] 7: (()+0x5e703) [0x7f2e50d77703] 8: (()+0x5e922) [0x7f2e50d77922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f2e50459ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:01.070353 7f113c6108c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1858 2015-09-09 13:45:01.086130 7f113c6108c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:01.086468 7f113c6108c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:01.087142 7f113c6108c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:01.087192 7f113c6108c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:01.089497 7f113c6108c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:01.091683 7f113c6108c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f113c6108c0 time 2015-09-09 13:45:01.090146 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f1139b9eec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:01.068442 7f113c6108c0 5 asok(0x4e94000) register_command perfcounters_dump hook 0x4e1c050 -34> 2015-09-09 13:45:01.068470 7f113c6108c0 5 asok(0x4e94000) register_command 1 hook 0x4e1c050 -33> 2015-09-09 13:45:01.068482 7f113c6108c0 5 asok(0x4e94000) register_command perf dump hook 0x4e1c050 -32> 2015-09-09 13:45:01.068487 7f113c6108c0 5 asok(0x4e94000) register_command perfcounters_schema hook 0x4e1c050 -31> 2015-09-09 13:45:01.068489 7f113c6108c0 5 asok(0x4e94000) register_command 2 hook 0x4e1c050 -30> 2015-09-09 13:45:01.068493 7f113c6108c0 5 asok(0x4e94000) register_command perf schema hook 0x4e1c050 -29> 2015-09-09 13:45:01.068499 7f113c6108c0 5 asok(0x4e94000) register_command perf reset hook 0x4e1c050 -28> 2015-09-09 13:45:01.068501 7f113c6108c0 5 asok(0x4e94000) register_command config show hook 0x4e1c050 -27> 2015-09-09 13:45:01.068504 7f113c6108c0 5 asok(0x4e94000) register_command config set hook 0x4e1c050 -26> 2015-09-09 13:45:01.068506 7f113c6108c0 5 asok(0x4e94000) register_command config get hook 0x4e1c050 -25> 2015-09-09 13:45:01.068507 7f113c6108c0 5 asok(0x4e94000) register_command config diff hook 0x4e1c050 -24> 2015-09-09 13:45:01.068509 7f113c6108c0 5 asok(0x4e94000) register_command log flush hook 0x4e1c050 -23> 2015-09-09 13:45:01.068511 7f113c6108c0 5 asok(0x4e94000) register_command log dump hook 0x4e1c050 -22> 2015-09-09 13:45:01.068513 7f113c6108c0 5 asok(0x4e94000) register_command log reopen hook 0x4e1c050 -21> 2015-09-09 13:45:01.070353 7f113c6108c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1858 -20> 2015-09-09 13:45:01.071727 7f113c6108c0 5 asok(0x4e94000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:01.071744 7f113c6108c0 5 asok(0x4e94000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:01.071776 7f113c6108c0 5 asok(0x4e94000) register_command 0 hook 0x4e180b8 -17> 2015-09-09 13:45:01.071785 7f113c6108c0 5 asok(0x4e94000) register_command version hook 0x4e180b8 -16> 2015-09-09 13:45:01.071788 7f113c6108c0 5 asok(0x4e94000) register_command git_version hook 0x4e180b8 -15> 2015-09-09 13:45:01.071792 7f113c6108c0 5 asok(0x4e94000) register_command help hook 0x4e1c0b0 -14> 2015-09-09 13:45:01.071796 7f113c6108c0 5 asok(0x4e94000) register_command get_command_descriptions hook 0x4e1c150 -13> 2015-09-09 13:45:01.071845 7f1137990700 5 asok(0x4e94000) entry start -12> 2015-09-09 13:45:01.086130 7f113c6108c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:01.086215 7f113c6108c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:01.086228 7f113c6108c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:01.086316 7f113c6108c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:01.086328 7f113c6108c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:01.086354 7f113c6108c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:01.086374 7f113c6108c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:01.086468 7f113c6108c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:01.087142 7f113c6108c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:01.087192 7f113c6108c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:01.089470 7f113c6108c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:01.089497 7f113c6108c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:01.091683 7f113c6108c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f113c6108c0 time 2015-09-09 13:45:01.090146 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f1139b9eec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:01.094364 7f113c6108c0 -1 *** Caught signal (Aborted) ** in thread 7f113c6108c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f113b714340] 3: (gsignal()+0x39) [0x7f1139bb3cc9] 4: (abort()+0x148) [0x7f1139bb70d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f113a4be535] 6: (()+0x5e6d6) [0x7f113a4bc6d6] 7: (()+0x5e703) [0x7f113a4bc703] 8: (()+0x5e922) [0x7f113a4bc922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f1139b9eec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:01.094364 7f113c6108c0 -1 *** Caught signal (Aborted) ** in thread 7f113c6108c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f113b714340] 3: (gsignal()+0x39) [0x7f1139bb3cc9] 4: (abort()+0x148) [0x7f1139bb70d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f113a4be535] 6: (()+0x5e6d6) [0x7f113a4bc6d6] 7: (()+0x5e703) [0x7f113a4bc703] 8: (()+0x5e922) [0x7f113a4bc922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f1139b9eec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:01.218325 7f4864c368c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1879 2015-09-09 13:45:01.230257 7f4864c368c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:01.230471 7f4864c368c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 13:45:01.230890 7f4864c368c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 13:45:01.230905 7f4864c368c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 13:45:01.232210 7f4864c368c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 13:45:01.233816 7f4864c368c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f4864c368c0 time 2015-09-09 13:45:01.232618 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f48621c4ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 13:45:01.216359 7f4864c368c0 5 asok(0x43bc000) register_command perfcounters_dump hook 0x4344050 -34> 2015-09-09 13:45:01.216392 7f4864c368c0 5 asok(0x43bc000) register_command 1 hook 0x4344050 -33> 2015-09-09 13:45:01.216397 7f4864c368c0 5 asok(0x43bc000) register_command perf dump hook 0x4344050 -32> 2015-09-09 13:45:01.216405 7f4864c368c0 5 asok(0x43bc000) register_command perfcounters_schema hook 0x4344050 -31> 2015-09-09 13:45:01.216408 7f4864c368c0 5 asok(0x43bc000) register_command 2 hook 0x4344050 -30> 2015-09-09 13:45:01.216412 7f4864c368c0 5 asok(0x43bc000) register_command perf schema hook 0x4344050 -29> 2015-09-09 13:45:01.216414 7f4864c368c0 5 asok(0x43bc000) register_command perf reset hook 0x4344050 -28> 2015-09-09 13:45:01.216416 7f4864c368c0 5 asok(0x43bc000) register_command config show hook 0x4344050 -27> 2015-09-09 13:45:01.216419 7f4864c368c0 5 asok(0x43bc000) register_command config set hook 0x4344050 -26> 2015-09-09 13:45:01.216425 7f4864c368c0 5 asok(0x43bc000) register_command config get hook 0x4344050 -25> 2015-09-09 13:45:01.216427 7f4864c368c0 5 asok(0x43bc000) register_command config diff hook 0x4344050 -24> 2015-09-09 13:45:01.216428 7f4864c368c0 5 asok(0x43bc000) register_command log flush hook 0x4344050 -23> 2015-09-09 13:45:01.216431 7f4864c368c0 5 asok(0x43bc000) register_command log dump hook 0x4344050 -22> 2015-09-09 13:45:01.216437 7f4864c368c0 5 asok(0x43bc000) register_command log reopen hook 0x4344050 -21> 2015-09-09 13:45:01.218325 7f4864c368c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 1879 -20> 2015-09-09 13:45:01.219481 7f4864c368c0 5 asok(0x43bc000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 13:45:01.219493 7f4864c368c0 5 asok(0x43bc000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 13:45:01.219516 7f4864c368c0 5 asok(0x43bc000) register_command 0 hook 0x43400b8 -17> 2015-09-09 13:45:01.219523 7f4864c368c0 5 asok(0x43bc000) register_command version hook 0x43400b8 -16> 2015-09-09 13:45:01.219525 7f4864c368c0 5 asok(0x43bc000) register_command git_version hook 0x43400b8 -15> 2015-09-09 13:45:01.219527 7f4864c368c0 5 asok(0x43bc000) register_command help hook 0x43440b0 -14> 2015-09-09 13:45:01.219530 7f4864c368c0 5 asok(0x43bc000) register_command get_command_descriptions hook 0x4344150 -13> 2015-09-09 13:45:01.219561 7f485ffb6700 5 asok(0x43bc000) entry start -12> 2015-09-09 13:45:01.230257 7f4864c368c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 13:45:01.230315 7f4864c368c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 13:45:01.230319 7f4864c368c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 13:45:01.230380 7f4864c368c0 5 adding auth protocol: cephx -8> 2015-09-09 13:45:01.230387 7f4864c368c0 5 adding auth protocol: cephx -7> 2015-09-09 13:45:01.230406 7f4864c368c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 13:45:01.230408 7f4864c368c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 13:45:01.230471 7f4864c368c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 13:45:01.230890 7f4864c368c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 13:45:01.230905 7f4864c368c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 13:45:01.232194 7f4864c368c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 13:45:01.232210 7f4864c368c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 13:45:01.233816 7f4864c368c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7f4864c368c0 time 2015-09-09 13:45:01.232618 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7f48621c4ec5] 9: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 13:45:01.235809 7f4864c368c0 -1 *** Caught signal (Aborted) ** in thread 7f4864c368c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f4863d3a340] 3: (gsignal()+0x39) [0x7f48621d9cc9] 4: (abort()+0x148) [0x7f48621dd0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f4862ae4535] 6: (()+0x5e6d6) [0x7f4862ae26d6] 7: (()+0x5e703) [0x7f4862ae2703] 8: (()+0x5e922) [0x7f4862ae2922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f48621c4ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 13:45:01.235809 7f4864c368c0 -1 *** Caught signal (Aborted) ** in thread 7f4864c368c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: /usr/bin/ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7f4863d3a340] 3: (gsignal()+0x39) [0x7f48621d9cc9] 4: (abort()+0x148) [0x7f48621dd0d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f4862ae4535] 6: (()+0x5e6d6) [0x7f4862ae26d6] 7: (()+0x5e703) [0x7f4862ae2703] 8: (()+0x5e922) [0x7f4862ae2922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7f48621c4ec5] 17: /usr/bin/ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 15:25:25.220906 7fb4d96ed8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 17063 2015-09-09 15:25:25.234773 7fb4d96ed8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 15:25:25.235038 7fb4d96ed8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe 2015-09-09 15:25:25.235418 7fb4d96ed8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 2015-09-09 15:25:25.235432 7fb4d96ed8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap 2015-09-09 15:25:25.237369 7fb4d96ed8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 2015-09-09 15:25:25.239140 7fb4d96ed8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fb4d96ed8c0 time 2015-09-09 15:25:25.237844 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fb4d6c7bec5] 9: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -35> 2015-09-09 15:25:25.218967 7fb4d96ed8c0 5 asok(0x3f18000) register_command perfcounters_dump hook 0x3ea0040 -34> 2015-09-09 15:25:25.219004 7fb4d96ed8c0 5 asok(0x3f18000) register_command 1 hook 0x3ea0040 -33> 2015-09-09 15:25:25.219011 7fb4d96ed8c0 5 asok(0x3f18000) register_command perf dump hook 0x3ea0040 -32> 2015-09-09 15:25:25.219017 7fb4d96ed8c0 5 asok(0x3f18000) register_command perfcounters_schema hook 0x3ea0040 -31> 2015-09-09 15:25:25.219024 7fb4d96ed8c0 5 asok(0x3f18000) register_command 2 hook 0x3ea0040 -30> 2015-09-09 15:25:25.219025 7fb4d96ed8c0 5 asok(0x3f18000) register_command perf schema hook 0x3ea0040 -29> 2015-09-09 15:25:25.219027 7fb4d96ed8c0 5 asok(0x3f18000) register_command perf reset hook 0x3ea0040 -28> 2015-09-09 15:25:25.219032 7fb4d96ed8c0 5 asok(0x3f18000) register_command config show hook 0x3ea0040 -27> 2015-09-09 15:25:25.219034 7fb4d96ed8c0 5 asok(0x3f18000) register_command config set hook 0x3ea0040 -26> 2015-09-09 15:25:25.219041 7fb4d96ed8c0 5 asok(0x3f18000) register_command config get hook 0x3ea0040 -25> 2015-09-09 15:25:25.219043 7fb4d96ed8c0 5 asok(0x3f18000) register_command config diff hook 0x3ea0040 -24> 2015-09-09 15:25:25.219044 7fb4d96ed8c0 5 asok(0x3f18000) register_command log flush hook 0x3ea0040 -23> 2015-09-09 15:25:25.219046 7fb4d96ed8c0 5 asok(0x3f18000) register_command log dump hook 0x3ea0040 -22> 2015-09-09 15:25:25.219049 7fb4d96ed8c0 5 asok(0x3f18000) register_command log reopen hook 0x3ea0040 -21> 2015-09-09 15:25:25.220906 7fb4d96ed8c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 17063 -20> 2015-09-09 15:25:25.223137 7fb4d96ed8c0 5 asok(0x3f18000) init /var/run/ceph/ceph-mon.vm14.asok -19> 2015-09-09 15:25:25.223163 7fb4d96ed8c0 5 asok(0x3f18000) bind_and_listen /var/run/ceph/ceph-mon.vm14.asok -18> 2015-09-09 15:25:25.223205 7fb4d96ed8c0 5 asok(0x3f18000) register_command 0 hook 0x3e9c0b8 -17> 2015-09-09 15:25:25.223217 7fb4d96ed8c0 5 asok(0x3f18000) register_command version hook 0x3e9c0b8 -16> 2015-09-09 15:25:25.223237 7fb4d96ed8c0 5 asok(0x3f18000) register_command git_version hook 0x3e9c0b8 -15> 2015-09-09 15:25:25.223240 7fb4d96ed8c0 5 asok(0x3f18000) register_command help hook 0x3ea0150 -14> 2015-09-09 15:25:25.223245 7fb4d96ed8c0 5 asok(0x3f18000) register_command get_command_descriptions hook 0x3ea0140 -13> 2015-09-09 15:25:25.223281 7fb4d4a6d700 5 asok(0x3f18000) entry start -12> 2015-09-09 15:25:25.234773 7fb4d96ed8c0 0 starting mon.vm14 rank 1 at 172.16.31.213:6789/0 mon_data /var/lib/ceph/mon/ceph-vm14 fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -11> 2015-09-09 15:25:25.234858 7fb4d96ed8c0 1 -- 172.16.31.213:6789/0 learned my addr 172.16.31.213:6789/0 -10> 2015-09-09 15:25:25.234867 7fb4d96ed8c0 1 accepter.accepter.bind my_inst.addr is 172.16.31.213:6789/0 need_addr=0 -9> 2015-09-09 15:25:25.234921 7fb4d96ed8c0 5 adding auth protocol: cephx -8> 2015-09-09 15:25:25.234928 7fb4d96ed8c0 5 adding auth protocol: cephx -7> 2015-09-09 15:25:25.234949 7fb4d96ed8c0 10 log_channel(cluster) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info) -6> 2015-09-09 15:25:25.234957 7fb4d96ed8c0 10 log_channel(audit) update_config to_monitors: true to_syslog: false syslog_facility: local0 prio: info) -5> 2015-09-09 15:25:25.235038 7fb4d96ed8c0 1 mon.vm14@-1(probing) e1 preinit fsid dc4f5635-5454-4d2c-94b7-ec05254596fe -4> 2015-09-09 15:25:25.235418 7fb4d96ed8c0 1 mon.vm14@-1(probing).paxosservice(pgmap 182369..182960) refresh upgraded, format 0 -> 1 -3> 2015-09-09 15:25:25.235432 7fb4d96ed8c0 1 mon.vm14@-1(probing).pg v0 on_upgrade discarding in-core PGMap -2> 2015-09-09 15:25:25.237343 7fb4d96ed8c0 4 mon.vm14@-1(probing).mds e1 new map -1> 2015-09-09 15:25:25.237369 7fb4d96ed8c0 0 mon.vm14@-1(probing).mds e1 print_map epoch 1 flags 0 created 0.000000 modified 2015-07-31 15:48:47.744313 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed stopped data_pools metadata_pool 0 inline_data disabled 0> 2015-09-09 15:25:25.239140 7fb4d96ed8c0 -1 mon/OSDMonitor.cc: In function 'virtual void OSDMonitor::update_from_paxos(bool*)' thread 7fb4d96ed8c0 time 2015-09-09 15:25:25.237844 mon/OSDMonitor.cc: 204: FAILED assert(err == 0) ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7dd01b] 2: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 3: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 4: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 5: (Monitor::init_paxos()+0x85) [0x5af645] 6: (Monitor::preinit()+0x7d7) [0x5b42a7] 7: (main()+0x230c) [0x57640c] 8: (__libc_start_main()+0xf5) [0x7fb4d6c7bec5] 9: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 15:25:25.241405 7fb4d96ed8c0 -1 *** Caught signal (Aborted) ** in thread 7fb4d96ed8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fb4d87f1340] 3: (gsignal()+0x39) [0x7fb4d6c90cc9] 4: (abort()+0x148) [0x7fb4d6c940d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fb4d759b535] 6: (()+0x5e6d6) [0x7fb4d75996d6] 7: (()+0x5e703) [0x7fb4d7599703] 8: (()+0x5e922) [0x7fb4d7599922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fb4d6c7bec5] 17: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- 0> 2015-09-09 15:25:25.241405 7fb4d96ed8c0 -1 *** Caught signal (Aborted) ** in thread 7fb4d96ed8c0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) 1: ceph-mon() [0x9a781a] 2: (()+0x10340) [0x7fb4d87f1340] 3: (gsignal()+0x39) [0x7fb4d6c90cc9] 4: (abort()+0x148) [0x7fb4d6c940d8] 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7fb4d759b535] 6: (()+0x5e6d6) [0x7fb4d75996d6] 7: (()+0x5e703) [0x7fb4d7599703] 8: (()+0x5e922) [0x7fb4d7599922] 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x7dd208] 10: (OSDMonitor::update_from_paxos(bool*)+0x21eb) [0x622d3b] 11: (PaxosService::refresh(bool*)+0x19a) [0x60315a] 12: (Monitor::refresh_from_paxos(bool*)+0x1db) [0x5af2db] 13: (Monitor::init_paxos()+0x85) [0x5af645] 14: (Monitor::preinit()+0x7d7) [0x5b42a7] 15: (main()+0x230c) [0x57640c] 16: (__libc_start_main()+0xf5) [0x7fb4d6c7bec5] 17: ceph-mon() [0x598027] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-mon.vm14.log --- end dump of recent events --- 2015-09-09 15:26:31.484164 7fa82a0588c0 0 ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3), process ceph-mon, pid 17241 2015-09-09 15:26:49.580528 7f91de15d8c0 -1 wrote monmap to here