2013-05-14 17:40:59.458008 7f230f0bb780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 1471
2013-05-14 17:40:59.902657 7f230f0bb780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:40:59.910734 7f230f0bb780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f230f0bb780 time 2013-05-14 17:40:59.910363
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
   -26> 2013-05-14 17:40:59.456679 7f230f0bb780  5 asok(0x223f000) register_command perfcounters_dump hook 0x2232010
   -25> 2013-05-14 17:40:59.456698 7f230f0bb780  5 asok(0x223f000) register_command 1 hook 0x2232010
   -24> 2013-05-14 17:40:59.456700 7f230f0bb780  5 asok(0x223f000) register_command perf dump hook 0x2232010
   -23> 2013-05-14 17:40:59.456705 7f230f0bb780  5 asok(0x223f000) register_command perfcounters_schema hook 0x2232010
   -22> 2013-05-14 17:40:59.456707 7f230f0bb780  5 asok(0x223f000) register_command 2 hook 0x2232010
   -21> 2013-05-14 17:40:59.456708 7f230f0bb780  5 asok(0x223f000) register_command perf schema hook 0x2232010
   -20> 2013-05-14 17:40:59.456710 7f230f0bb780  5 asok(0x223f000) register_command config show hook 0x2232010
   -19> 2013-05-14 17:40:59.456711 7f230f0bb780  5 asok(0x223f000) register_command config set hook 0x2232010
   -18> 2013-05-14 17:40:59.456714 7f230f0bb780  5 asok(0x223f000) register_command log flush hook 0x2232010
   -17> 2013-05-14 17:40:59.456716 7f230f0bb780  5 asok(0x223f000) register_command log dump hook 0x2232010
   -16> 2013-05-14 17:40:59.456719 7f230f0bb780  5 asok(0x223f000) register_command log reopen hook 0x2232010
   -15> 2013-05-14 17:40:59.458008 7f230f0bb780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 1471
   -14> 2013-05-14 17:40:59.458377 7f230f0bb780  1 finished global_init_daemonize
   -13> 2013-05-14 17:40:59.460046 7f230f0bb780  5 asok(0x223f000) init /var/run/ceph/ceph-mon.0.asok
   -12> 2013-05-14 17:40:59.460059 7f230f0bb780  5 asok(0x223f000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
   -11> 2013-05-14 17:40:59.460106 7f230f0bb780  5 asok(0x223f000) register_command 0 hook 0x22310b8
   -10> 2013-05-14 17:40:59.460111 7f230f0bb780  5 asok(0x223f000) register_command version hook 0x22310b8
    -9> 2013-05-14 17:40:59.460115 7f230f0bb780  5 asok(0x223f000) register_command git_version hook 0x22310b8
    -8> 2013-05-14 17:40:59.460118 7f230f0bb780  5 asok(0x223f000) register_command help hook 0x22320b0
    -7> 2013-05-14 17:40:59.460134 7f230d98e700  5 asok(0x223f000) entry start
    -6> 2013-05-14 17:40:59.902603 7f230f0bb780  1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
    -5> 2013-05-14 17:40:59.902611 7f230f0bb780  1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
    -4> 2013-05-14 17:40:59.902622 7f230f0bb780  5 adding auth protocol: cephx
    -3> 2013-05-14 17:40:59.902624 7f230f0bb780  5 adding auth protocol: cephx
    -2> 2013-05-14 17:40:59.902657 7f230f0bb780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
    -1> 2013-05-14 17:40:59.909012 7f230f0bb780  4 mon.0@-1(probing).mds e4428 new map
     0> 2013-05-14 17:40:59.910734 7f230f0bb780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f230f0bb780 time 2013-05-14 17:40:59.910363
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   0/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:40:59.911539 7f230f0bb780 -1 *** Caught signal (Aborted) **
 in thread 7f230f0bb780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7f23110e4ff0]
 3: (gsignal()+0x35) [0x7f230fd361b5]
 4: (abort()+0x180) [0x7f230fd38fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f23105cadc5]
 6: (()+0xcb166) [0x7f23105c9166]
 7: (()+0xcb193) [0x7f23105c9193]
 8: (()+0xcb28e) [0x7f23105c928e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2013-05-14 17:40:59.911539 7f230f0bb780 -1 *** Caught signal (Aborted) **
 in thread 7f230f0bb780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7f23110e4ff0]
 3: (gsignal()+0x35) [0x7f230fd361b5]
 4: (abort()+0x180) [0x7f230fd38fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f23105cadc5]
 6: (()+0xcb166) [0x7f23105c9166]
 7: (()+0xcb193) [0x7f23105c9193]
 8: (()+0xcb28e) [0x7f23105c928e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   0/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:41:17.113420 7f9580bf6780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 2340
2013-05-14 17:41:17.723588 7f9580bf6780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:41:17.730730 7f9580bf6780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f9580bf6780 time 2013-05-14 17:41:17.730342
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7f958185dc8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
   -26> 2013-05-14 17:41:17.111709 7f9580bf6780  5 asok(0x18c8000) register_command perfcounters_dump hook 0x18bc010
   -25> 2013-05-14 17:41:17.111732 7f9580bf6780  5 asok(0x18c8000) register_command 1 hook 0x18bc010
   -24> 2013-05-14 17:41:17.111743 7f9580bf6780  5 asok(0x18c8000) register_command perf dump hook 0x18bc010
   -23> 2013-05-14 17:41:17.111747 7f9580bf6780  5 asok(0x18c8000) register_command perfcounters_schema hook 0x18bc010
   -22> 2013-05-14 17:41:17.111750 7f9580bf6780  5 asok(0x18c8000) register_command 2 hook 0x18bc010
   -21> 2013-05-14 17:41:17.111753 7f9580bf6780  5 asok(0x18c8000) register_command perf schema hook 0x18bc010
   -20> 2013-05-14 17:41:17.111755 7f9580bf6780  5 asok(0x18c8000) register_command config show hook 0x18bc010
   -19> 2013-05-14 17:41:17.111758 7f9580bf6780  5 asok(0x18c8000) register_command config set hook 0x18bc010
   -18> 2013-05-14 17:41:17.111759 7f9580bf6780  5 asok(0x18c8000) register_command log flush hook 0x18bc010
   -17> 2013-05-14 17:41:17.111764 7f9580bf6780  5 asok(0x18c8000) register_command log dump hook 0x18bc010
   -16> 2013-05-14 17:41:17.111766 7f9580bf6780  5 asok(0x18c8000) register_command log reopen hook 0x18bc010
   -15> 2013-05-14 17:41:17.113420 7f9580bf6780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 2340
   -14> 2013-05-14 17:41:17.113749 7f9580bf6780  1 finished global_init_daemonize
   -13> 2013-05-14 17:41:17.115358 7f9580bf6780  5 asok(0x18c8000) init /var/run/ceph/ceph-mon.0.asok
   -12> 2013-05-14 17:41:17.115373 7f9580bf6780  5 asok(0x18c8000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
   -11> 2013-05-14 17:41:17.115407 7f9580bf6780  5 asok(0x18c8000) register_command 0 hook 0x18bb0b8
   -10> 2013-05-14 17:41:17.115413 7f9580bf6780  5 asok(0x18c8000) register_command version hook 0x18bb0b8
    -9> 2013-05-14 17:41:17.115416 7f9580bf6780  5 asok(0x18c8000) register_command git_version hook 0x18bb0b8
    -8> 2013-05-14 17:41:17.115420 7f9580bf6780  5 asok(0x18c8000) register_command help hook 0x18bc0b0
    -7> 2013-05-14 17:41:17.115437 7f957f4c9700  5 asok(0x18c8000) entry start
    -6> 2013-05-14 17:41:17.723528 7f9580bf6780  1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
    -5> 2013-05-14 17:41:17.723536 7f9580bf6780  1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
    -4> 2013-05-14 17:41:17.723549 7f9580bf6780  5 adding auth protocol: cephx
    -3> 2013-05-14 17:41:17.723551 7f9580bf6780  5 adding auth protocol: cephx
    -2> 2013-05-14 17:41:17.723588 7f9580bf6780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
    -1> 2013-05-14 17:41:17.729667 7f9580bf6780  4 mon.0@-1(probing).mds e4428 new map
     0> 2013-05-14 17:41:17.730730 7f9580bf6780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f9580bf6780 time 2013-05-14 17:41:17.730342
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7f958185dc8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   0/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:41:17.731554 7f9580bf6780 -1 *** Caught signal (Aborted) **
 in thread 7f9580bf6780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7f9582c1fff0]
 3: (gsignal()+0x35) [0x7f95818711b5]
 4: (abort()+0x180) [0x7f9581873fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f9582105dc5]
 6: (()+0xcb166) [0x7f9582104166]
 7: (()+0xcb193) [0x7f9582104193]
 8: (()+0xcb28e) [0x7f958210428e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7f958185dc8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2013-05-14 17:41:17.731554 7f9580bf6780 -1 *** Caught signal (Aborted) **
 in thread 7f9580bf6780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7f9582c1fff0]
 3: (gsignal()+0x35) [0x7f95818711b5]
 4: (abort()+0x180) [0x7f9581873fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f9582105dc5]
 6: (()+0xcb166) [0x7f9582104166]
 7: (()+0xcb193) [0x7f9582104193]
 8: (()+0xcb28e) [0x7f958210428e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7f958185dc8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   0/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:44:00.484562 7fbb1b89b780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 8366
2013-05-14 17:44:00.486662 7fbb1b89b780 10 needs_conversion
2013-05-14 17:44:00.843516 7fbb1b89b780 10 obtain_monmap
2013-05-14 17:44:00.843569 7fbb1b89b780 10 obtain_monmap read last committed monmap ver 4
2013-05-14 17:44:00.843727 7fbb1b89b780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:44:00.844305 7fbb1b89b780 10 mon.0@-1(probing) e4 check_fsid cluster_uuid contains '2ff06d61-7b8e-4b10-b39d-138cfcc1ee43'
2013-05-14 17:44:00.844322 7fbb1b89b780 10 mon.0@-1(probing) e4 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?)}
2013-05-14 17:44:00.844334 7fbb1b89b780 10 mon.0@-1(probing) e4 has_ever_joined = 1
2013-05-14 17:44:00.844359 7fbb1b89b780 10 mon.0@-1(probing) e4 init_paxos
2013-05-14 17:44:00.844402 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init last_pn: 516800 accepted_pn: 0 last_committed: 31732523 first_committed: 31732421
2013-05-14 17:44:00.844409 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init
2013-05-14 17:44:00.845089 7fbb1b89b780  7 mon.0@-1(probing).pg v0 update_from_paxos loading latest full pgmap v16013180
2013-05-14 17:44:00.847157 7fbb1b89b780  7 mon.0@-1(probing).pg v16013180 update_from_paxos  applying incremental 16013181
2013-05-14 17:44:00.847195 7fbb1b89b780 10 mon.0@-1(probing).pg v16013181 v16013181: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3771KB/s wr, 10op/s
2013-05-14 17:44:00.847773 7fbb1b89b780  7 mon.0@-1(probing).pg v16013181 update_from_paxos  applying incremental 16013182
2013-05-14 17:44:00.847933 7fbb1b89b780 10 mon.0@-1(probing).pg v16013182 v16013182: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4826KB/s wr, 39op/s
2013-05-14 17:44:00.847969 7fbb1b89b780  7 mon.0@-1(probing).pg v16013182 update_from_paxos  applying incremental 16013183
2013-05-14 17:44:00.848005 7fbb1b89b780 10 mon.0@-1(probing).pg v16013183 v16013183: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3502KB/s wr, 57op/s
2013-05-14 17:44:00.848085 7fbb1b89b780  7 mon.0@-1(probing).pg v16013183 update_from_paxos  applying incremental 16013184
2013-05-14 17:44:00.848135 7fbb1b89b780 10 mon.0@-1(probing).pg v16013184 v16013184: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 367KB/s wr, 19op/s
2013-05-14 17:44:00.848167 7fbb1b89b780  7 mon.0@-1(probing).pg v16013184 update_from_paxos  applying incremental 16013185
2013-05-14 17:44:00.848338 7fbb1b89b780 10 mon.0@-1(probing).pg v16013185 v16013185: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 7049KB/s wr, 86op/s
2013-05-14 17:44:00.848429 7fbb1b89b780  7 mon.0@-1(probing).pg v16013185 update_from_paxos  applying incremental 16013186
2013-05-14 17:44:00.848453 7fbb1b89b780 10 mon.0@-1(probing).pg v16013186 v16013186: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 6514KB/s wr, 60op/s
2013-05-14 17:44:00.848551 7fbb1b89b780  7 mon.0@-1(probing).pg v16013186 update_from_paxos  applying incremental 16013187
2013-05-14 17:44:00.848675 7fbb1b89b780 10 mon.0@-1(probing).pg v16013187 v16013187: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4045KB/s wr, 49op/s
2013-05-14 17:44:00.848769 7fbb1b89b780  7 mon.0@-1(probing).pg v16013187 update_from_paxos  applying incremental 16013188
2013-05-14 17:44:00.848792 7fbb1b89b780 10 mon.0@-1(probing).pg v16013188 v16013188: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 5804KB/s wr, 97op/s
2013-05-14 17:44:00.848810 7fbb1b89b780  7 mon.0@-1(probing).pg v16013188 update_from_paxos  applying incremental 16013189
2013-05-14 17:44:00.848822 7fbb1b89b780 10 mon.0@-1(probing).pg v16013189 v16013189: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 228KB/s wr, 5op/s
2013-05-14 17:44:00.848851 7fbb1b89b780  7 mon.0@-1(probing).pg v16013189 update_from_paxos  applying incremental 16013190
2013-05-14 17:44:00.849036 7fbb1b89b780 10 mon.0@-1(probing).pg v16013190 v16013190: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4611KB/s wr, 111op/s
2013-05-14 17:44:00.849060 7fbb1b89b780  7 mon.0@-1(probing).pg v16013190 update_from_paxos  applying incremental 16013191
2013-05-14 17:44:00.849093 7fbb1b89b780 10 mon.0@-1(probing).pg v16013191 v16013191: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4445KB/s wr, 113op/s
2013-05-14 17:44:00.849121 7fbb1b89b780  7 mon.0@-1(probing).pg v16013191 update_from_paxos  applying incremental 16013192
2013-05-14 17:44:00.849285 7fbb1b89b780 10 mon.0@-1(probing).pg v16013192 v16013192: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1523KB/s wr, 118op/s
2013-05-14 17:44:00.849367 7fbb1b89b780  7 mon.0@-1(probing).pg v16013192 update_from_paxos  applying incremental 16013193
2013-05-14 17:44:00.849394 7fbb1b89b780 10 mon.0@-1(probing).pg v16013193 v16013193: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1906KB/s wr, 210op/s
2013-05-14 17:44:00.849495 7fbb1b89b780  7 mon.0@-1(probing).pg v16013193 update_from_paxos  applying incremental 16013194
2013-05-14 17:44:00.849512 7fbb1b89b780 10 mon.0@-1(probing).pg v16013194 v16013194: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 202KB/s wr, 30op/s
2013-05-14 17:44:00.849542 7fbb1b89b780  7 mon.0@-1(probing).pg v16013194 update_from_paxos  applying incremental 16013195
2013-05-14 17:44:00.849946 7fbb1b89b780 10 mon.0@-1(probing).pg v16013195 v16013195: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 504KB/s wr, 70op/s
2013-05-14 17:44:00.850028 7fbb1b89b780  7 mon.0@-1(probing).pg v16013195 update_from_paxos  applying incremental 16013196
2013-05-14 17:44:00.850057 7fbb1b89b780 10 mon.0@-1(probing).pg v16013196 v16013196: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 925KB/s wr, 132op/s
2013-05-14 17:44:00.850135 7fbb1b89b780  7 mon.0@-1(probing).pg v16013196 update_from_paxos  applying incremental 16013197
2013-05-14 17:44:00.850155 7fbb1b89b780 10 mon.0@-1(probing).pg v16013197 v16013197: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 225KB/s wr, 36op/s
2013-05-14 17:44:00.850236 7fbb1b89b780  7 mon.0@-1(probing).pg v16013197 update_from_paxos  applying incremental 16013198
2013-05-14 17:44:00.850332 7fbb1b89b780 10 mon.0@-1(probing).pg v16013198 v16013198: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 436KB/s wr, 38op/s
2013-05-14 17:44:00.850410 7fbb1b89b780  7 mon.0@-1(probing).pg v16013198 update_from_paxos  applying incremental 16013199
2013-05-14 17:44:00.850436 7fbb1b89b780 10 mon.0@-1(probing).pg v16013199 v16013199: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 388KB/s wr, 32op/s
2013-05-14 17:44:00.850533 7fbb1b89b780  7 mon.0@-1(probing).pg v16013199 update_from_paxos  applying incremental 16013200
2013-05-14 17:44:00.850907 7fbb1b89b780 10 mon.0@-1(probing).pg v16013200 v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
2013-05-14 17:44:00.850932 7fbb1b89b780  7 mon.0@-1(probing).pg v16013200 update_from_paxos  applying incremental 16013201
2013-05-14 17:44:00.850955 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 v16013201: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 813KB/s wr, 73op/s
2013-05-14 17:44:00.850971 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 update_logger
2013-05-14 17:44:00.850988 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos version 4428, my e 0
2013-05-14 17:44:00.851006 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos got 4428
2013-05-14 17:44:00.851021 7fbb1b89b780  4 mon.0@-1(probing).mds e4428 new map
2013-05-14 17:44:00.851024 7fbb1b89b780  7 mon.0@-1(probing).mds e4428 print_map
epoch   4428
flags   0
created 2012-07-25 00:52:15.550913
modified        2013-05-14 14:16:54.190887
tableserver     0
root    0
session_timeout 60
session_autoclose       300
last_failure    4247
last_failure_osd_epoch  3816
compat  compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object}
max_mds 1
in      0
up      {0=46584}
failed
stopped
data_pools      0
metadata_pool   1
46584:  188.65.144.4:6800/3230 '0' mds.0.449 up:active seq 60926

2013-05-14 17:44:00.851046 7fbb1b89b780 10 mon.0@-1(probing).mds e4428 update_logger
2013-05-14 17:44:00.851226 7fbb1b89b780 15 mon.0@-1(probing).osd e0 update_from_paxos paxos e 3909, my e 0
2013-05-14 17:44:00.851408 7fbb1b89b780  7 mon.0@-1(probing).osd e0 update_from_paxos loading latest full map e3909
2013-05-14 17:44:00.851475 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 send_to_waiting 3909
2013-05-14 17:44:00.851482 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 share_map_with_random_osd no up osd on our session map
2013-05-14 17:44:00.851493 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 update_logger
2013-05-14 17:44:00.851500 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos
2013-05-14 17:44:00.851527 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos version 15702609 summary v 0
2013-05-14 17:44:00.851707 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos latest full 15702608
2013-05-14 17:44:00.851731 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos loading summary e15702608
2013-05-14 17:44:00.851867 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos loaded summary e15702608
2013-05-14 17:44:00.851891 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos applying incremental log 15702609 2013-05-14 14:34:11.768508 mon.1 188.65.144.5:6789/0 53 : [INF] pgmap v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
2013-05-14 17:44:00.852037 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 check_subs
2013-05-14 17:44:00.852062 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos
2013-05-14 17:44:00.852105 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos version 8372 keys ver 0 latest 8370
2013-05-14 17:44:00.852123 7fbb1b89b780  7 mon.0@-1(probing).auth v8372 update_from_paxos loading summary e 8370
2013-05-14 17:44:00.852242 7fbb1b89b780  7 mon.0@-1(probing).auth v8372 update_from_paxos latest length 1896
2013-05-14 17:44:00.852394 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos key server version 8370
2013-05-14 17:44:00.853232 7fbb1b89b780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7fbb1b89b780 time 2013-05-14 17:44:00.852652
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
  -100> 2013-05-14 17:44:00.483218 7fbb1b89b780  5 asok(0x1c4d000) register_command perfcounters_dump hook 0x1c41010
   -99> 2013-05-14 17:44:00.483232 7fbb1b89b780  5 asok(0x1c4d000) register_command 1 hook 0x1c41010
   -98> 2013-05-14 17:44:00.483237 7fbb1b89b780  5 asok(0x1c4d000) register_command perf dump hook 0x1c41010
   -97> 2013-05-14 17:44:00.483243 7fbb1b89b780  5 asok(0x1c4d000) register_command perfcounters_schema hook 0x1c41010
   -96> 2013-05-14 17:44:00.483246 7fbb1b89b780  5 asok(0x1c4d000) register_command 2 hook 0x1c41010
   -95> 2013-05-14 17:44:00.483248 7fbb1b89b780  5 asok(0x1c4d000) register_command perf schema hook 0x1c41010
   -94> 2013-05-14 17:44:00.483251 7fbb1b89b780  5 asok(0x1c4d000) register_command config show hook 0x1c41010
   -93> 2013-05-14 17:44:00.483254 7fbb1b89b780  5 asok(0x1c4d000) register_command config set hook 0x1c41010
   -92> 2013-05-14 17:44:00.483256 7fbb1b89b780  5 asok(0x1c4d000) register_command log flush hook 0x1c41010
   -91> 2013-05-14 17:44:00.483260 7fbb1b89b780  5 asok(0x1c4d000) register_command log dump hook 0x1c41010
   -90> 2013-05-14 17:44:00.483262 7fbb1b89b780  5 asok(0x1c4d000) register_command log reopen hook 0x1c41010
   -89> 2013-05-14 17:44:00.484562 7fbb1b89b780  0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 8366
   -88> 2013-05-14 17:44:00.484924 7fbb1b89b780  1 finished global_init_daemonize
   -87> 2013-05-14 17:44:00.486586 7fbb1b89b780  5 asok(0x1c4d000) init /var/run/ceph/ceph-mon.0.asok
   -86> 2013-05-14 17:44:00.486599 7fbb1b89b780  5 asok(0x1c4d000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
   -85> 2013-05-14 17:44:00.486629 7fbb1b89b780  5 asok(0x1c4d000) register_command 0 hook 0x1c400b8
   -84> 2013-05-14 17:44:00.486633 7fbb1b89b780  5 asok(0x1c4d000) register_command version hook 0x1c400b8
   -83> 2013-05-14 17:44:00.486636 7fbb1b89b780  5 asok(0x1c4d000) register_command git_version hook 0x1c400b8
   -82> 2013-05-14 17:44:00.486638 7fbb1b89b780  5 asok(0x1c4d000) register_command help hook 0x1c410b0
   -81> 2013-05-14 17:44:00.486662 7fbb1b89b780 10 needs_conversion
   -80> 2013-05-14 17:44:00.486668 7fbb1a16e700  5 asok(0x1c4d000) entry start
   -79> 2013-05-14 17:44:00.843516 7fbb1b89b780 10 obtain_monmap
   -78> 2013-05-14 17:44:00.843569 7fbb1b89b780 10 obtain_monmap read last committed monmap ver 4
   -77> 2013-05-14 17:44:00.843671 7fbb1b89b780  1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
   -76> 2013-05-14 17:44:00.843678 7fbb1b89b780  1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
   -75> 2013-05-14 17:44:00.843689 7fbb1b89b780  5 adding auth protocol: cephx
   -74> 2013-05-14 17:44:00.843691 7fbb1b89b780  5 adding auth protocol: cephx
   -73> 2013-05-14 17:44:00.843727 7fbb1b89b780  1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
   -72> 2013-05-14 17:44:00.844305 7fbb1b89b780 10 mon.0@-1(probing) e4 check_fsid cluster_uuid contains '2ff06d61-7b8e-4b10-b39d-138cfcc1ee43'
   -71> 2013-05-14 17:44:00.844322 7fbb1b89b780 10 mon.0@-1(probing) e4 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?)}
   -70> 2013-05-14 17:44:00.844334 7fbb1b89b780 10 mon.0@-1(probing) e4 has_ever_joined = 1
   -69> 2013-05-14 17:44:00.844359 7fbb1b89b780 10 mon.0@-1(probing) e4 init_paxos
   -68> 2013-05-14 17:44:00.844402 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init last_pn: 516800 accepted_pn: 0 last_committed: 31732523 first_committed: 31732421
   -67> 2013-05-14 17:44:00.844409 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init
   -66> 2013-05-14 17:44:00.845089 7fbb1b89b780  7 mon.0@-1(probing).pg v0 update_from_paxos loading latest full pgmap v16013180
   -65> 2013-05-14 17:44:00.847157 7fbb1b89b780  7 mon.0@-1(probing).pg v16013180 update_from_paxos  applying incremental 16013181
   -64> 2013-05-14 17:44:00.847195 7fbb1b89b780 10 mon.0@-1(probing).pg v16013181 v16013181: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3771KB/s wr, 10op/s
   -63> 2013-05-14 17:44:00.847773 7fbb1b89b780  7 mon.0@-1(probing).pg v16013181 update_from_paxos  applying incremental 16013182
   -62> 2013-05-14 17:44:00.847933 7fbb1b89b780 10 mon.0@-1(probing).pg v16013182 v16013182: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4826KB/s wr, 39op/s
   -61> 2013-05-14 17:44:00.847969 7fbb1b89b780  7 mon.0@-1(probing).pg v16013182 update_from_paxos  applying incremental 16013183
   -60> 2013-05-14 17:44:00.848005 7fbb1b89b780 10 mon.0@-1(probing).pg v16013183 v16013183: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3502KB/s wr, 57op/s
   -59> 2013-05-14 17:44:00.848085 7fbb1b89b780  7 mon.0@-1(probing).pg v16013183 update_from_paxos  applying incremental 16013184
   -58> 2013-05-14 17:44:00.848135 7fbb1b89b780 10 mon.0@-1(probing).pg v16013184 v16013184: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 367KB/s wr, 19op/s
   -57> 2013-05-14 17:44:00.848167 7fbb1b89b780  7 mon.0@-1(probing).pg v16013184 update_from_paxos  applying incremental 16013185
   -56> 2013-05-14 17:44:00.848338 7fbb1b89b780 10 mon.0@-1(probing).pg v16013185 v16013185: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 7049KB/s wr, 86op/s
   -55> 2013-05-14 17:44:00.848429 7fbb1b89b780  7 mon.0@-1(probing).pg v16013185 update_from_paxos  applying incremental 16013186
   -54> 2013-05-14 17:44:00.848453 7fbb1b89b780 10 mon.0@-1(probing).pg v16013186 v16013186: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 6514KB/s wr, 60op/s
   -53> 2013-05-14 17:44:00.848551 7fbb1b89b780  7 mon.0@-1(probing).pg v16013186 update_from_paxos  applying incremental 16013187
   -52> 2013-05-14 17:44:00.848675 7fbb1b89b780 10 mon.0@-1(probing).pg v16013187 v16013187: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4045KB/s wr, 49op/s
   -51> 2013-05-14 17:44:00.848769 7fbb1b89b780  7 mon.0@-1(probing).pg v16013187 update_from_paxos  applying incremental 16013188
   -50> 2013-05-14 17:44:00.848792 7fbb1b89b780 10 mon.0@-1(probing).pg v16013188 v16013188: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 5804KB/s wr, 97op/s
   -49> 2013-05-14 17:44:00.848810 7fbb1b89b780  7 mon.0@-1(probing).pg v16013188 update_from_paxos  applying incremental 16013189
   -48> 2013-05-14 17:44:00.848822 7fbb1b89b780 10 mon.0@-1(probing).pg v16013189 v16013189: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 228KB/s wr, 5op/s
   -47> 2013-05-14 17:44:00.848851 7fbb1b89b780  7 mon.0@-1(probing).pg v16013189 update_from_paxos  applying incremental 16013190
   -46> 2013-05-14 17:44:00.849036 7fbb1b89b780 10 mon.0@-1(probing).pg v16013190 v16013190: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4611KB/s wr, 111op/s
   -45> 2013-05-14 17:44:00.849060 7fbb1b89b780  7 mon.0@-1(probing).pg v16013190 update_from_paxos  applying incremental 16013191
   -44> 2013-05-14 17:44:00.849093 7fbb1b89b780 10 mon.0@-1(probing).pg v16013191 v16013191: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4445KB/s wr, 113op/s
   -43> 2013-05-14 17:44:00.849121 7fbb1b89b780  7 mon.0@-1(probing).pg v16013191 update_from_paxos  applying incremental 16013192
   -42> 2013-05-14 17:44:00.849285 7fbb1b89b780 10 mon.0@-1(probing).pg v16013192 v16013192: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1523KB/s wr, 118op/s
   -41> 2013-05-14 17:44:00.849367 7fbb1b89b780  7 mon.0@-1(probing).pg v16013192 update_from_paxos  applying incremental 16013193
   -40> 2013-05-14 17:44:00.849394 7fbb1b89b780 10 mon.0@-1(probing).pg v16013193 v16013193: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1906KB/s wr, 210op/s
   -39> 2013-05-14 17:44:00.849495 7fbb1b89b780  7 mon.0@-1(probing).pg v16013193 update_from_paxos  applying incremental 16013194
   -38> 2013-05-14 17:44:00.849512 7fbb1b89b780 10 mon.0@-1(probing).pg v16013194 v16013194: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 202KB/s wr, 30op/s
   -37> 2013-05-14 17:44:00.849542 7fbb1b89b780  7 mon.0@-1(probing).pg v16013194 update_from_paxos  applying incremental 16013195
   -36> 2013-05-14 17:44:00.849946 7fbb1b89b780 10 mon.0@-1(probing).pg v16013195 v16013195: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 504KB/s wr, 70op/s
   -35> 2013-05-14 17:44:00.850028 7fbb1b89b780  7 mon.0@-1(probing).pg v16013195 update_from_paxos  applying incremental 16013196
   -34> 2013-05-14 17:44:00.850057 7fbb1b89b780 10 mon.0@-1(probing).pg v16013196 v16013196: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 925KB/s wr, 132op/s
   -33> 2013-05-14 17:44:00.850135 7fbb1b89b780  7 mon.0@-1(probing).pg v16013196 update_from_paxos  applying incremental 16013197
   -32> 2013-05-14 17:44:00.850155 7fbb1b89b780 10 mon.0@-1(probing).pg v16013197 v16013197: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 225KB/s wr, 36op/s
   -31> 2013-05-14 17:44:00.850236 7fbb1b89b780  7 mon.0@-1(probing).pg v16013197 update_from_paxos  applying incremental 16013198
   -30> 2013-05-14 17:44:00.850332 7fbb1b89b780 10 mon.0@-1(probing).pg v16013198 v16013198: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 436KB/s wr, 38op/s
   -29> 2013-05-14 17:44:00.850410 7fbb1b89b780  7 mon.0@-1(probing).pg v16013198 update_from_paxos  applying incremental 16013199
   -28> 2013-05-14 17:44:00.850436 7fbb1b89b780 10 mon.0@-1(probing).pg v16013199 v16013199: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 388KB/s wr, 32op/s
   -27> 2013-05-14 17:44:00.850533 7fbb1b89b780  7 mon.0@-1(probing).pg v16013199 update_from_paxos  applying incremental 16013200
   -26> 2013-05-14 17:44:00.850907 7fbb1b89b780 10 mon.0@-1(probing).pg v16013200 v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
   -25> 2013-05-14 17:44:00.850932 7fbb1b89b780  7 mon.0@-1(probing).pg v16013200 update_from_paxos  applying incremental 16013201
   -24> 2013-05-14 17:44:00.850955 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 v16013201: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 813KB/s wr, 73op/s
   -23> 2013-05-14 17:44:00.850971 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 update_logger
   -22> 2013-05-14 17:44:00.850988 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos version 4428, my e 0
   -21> 2013-05-14 17:44:00.851006 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos got 4428
   -20> 2013-05-14 17:44:00.851021 7fbb1b89b780  4 mon.0@-1(probing).mds e4428 new map
   -19> 2013-05-14 17:44:00.851024 7fbb1b89b780  7 mon.0@-1(probing).mds e4428 print_map
epoch   4428
flags   0
created 2012-07-25 00:52:15.550913
modified        2013-05-14 14:16:54.190887
tableserver     0
root    0
session_timeout 60
session_autoclose       300
last_failure    4247
last_failure_osd_epoch  3816
compat  compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object}
max_mds 1
in      0
up      {0=46584}
failed
stopped
data_pools      0
metadata_pool   1
46584:  188.65.144.4:6800/3230 '0' mds.0.449 up:active seq 60926

   -18> 2013-05-14 17:44:00.851046 7fbb1b89b780 10 mon.0@-1(probing).mds e4428 update_logger
   -17> 2013-05-14 17:44:00.851226 7fbb1b89b780 15 mon.0@-1(probing).osd e0 update_from_paxos paxos e 3909, my e 0
   -16> 2013-05-14 17:44:00.851408 7fbb1b89b780  7 mon.0@-1(probing).osd e0 update_from_paxos loading latest full map e3909
   -15> 2013-05-14 17:44:00.851475 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 send_to_waiting 3909
   -14> 2013-05-14 17:44:00.851482 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 share_map_with_random_osd no up osd on our session map
   -13> 2013-05-14 17:44:00.851493 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 update_logger
   -12> 2013-05-14 17:44:00.851500 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos
   -11> 2013-05-14 17:44:00.851527 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos version 15702609 summary v 0
   -10> 2013-05-14 17:44:00.851707 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos latest full 15702608
    -9> 2013-05-14 17:44:00.851731 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos loading summary e15702608
    -8> 2013-05-14 17:44:00.851867 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos loaded summary e15702608
    -7> 2013-05-14 17:44:00.851891 7fbb1b89b780  7 mon.0@-1(probing).log v15702609 update_from_paxos applying incremental log 15702609 2013-05-14 14:34:11.768508 mon.1 188.65.144.5:6789/0 53 : [INF] pgmap v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
    -6> 2013-05-14 17:44:00.852037 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 check_subs
    -5> 2013-05-14 17:44:00.852062 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos
    -4> 2013-05-14 17:44:00.852105 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos version 8372 keys ver 0 latest 8370
    -3> 2013-05-14 17:44:00.852123 7fbb1b89b780  7 mon.0@-1(probing).auth v8372 update_from_paxos loading summary e 8370
    -2> 2013-05-14 17:44:00.852242 7fbb1b89b780  7 mon.0@-1(probing).auth v8372 update_from_paxos latest length 1896
    -1> 2013-05-14 17:44:00.852394 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos key server version 8370
     0> 2013-05-14 17:44:00.853232 7fbb1b89b780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7fbb1b89b780 time 2013-05-14 17:44:00.852652
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 2: (Monitor::init_paxos()+0xdf) [0x48292f]
 3: (Monitor::preinit()+0x679) [0x490a39]
 4: (main()+0x2b61) [0x478361]
 5: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
 6: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
  20/20 mon
   0/10 monc
  20/20 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:44:00.854313 7fbb1b89b780 -1 *** Caught signal (Aborted) **
 in thread 7fbb1b89b780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7fbb1d8c4ff0]
 3: (gsignal()+0x35) [0x7fbb1c5161b5]
 4: (abort()+0x180) [0x7fbb1c518fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7fbb1cdaadc5]
 6: (()+0xcb166) [0x7fbb1cda9166]
 7: (()+0xcb193) [0x7fbb1cda9193]
 8: (()+0xcb28e) [0x7fbb1cda928e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2013-05-14 17:44:00.854313 7fbb1b89b780 -1 *** Caught signal (Aborted) **
 in thread 7fbb1b89b780

 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
 1: /usr/bin/ceph-mon() [0x5957c9]
 2: (()+0xeff0) [0x7fbb1d8c4ff0]
 3: (gsignal()+0x35) [0x7fbb1c5161b5]
 4: (abort()+0x180) [0x7fbb1c518fc0]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7fbb1cdaadc5]
 6: (()+0xcb166) [0x7fbb1cda9166]
 7: (()+0xcb193) [0x7fbb1cda9193]
 8: (()+0xcb28e) [0x7fbb1cda928e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
 10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
 11: (Monitor::init_paxos()+0xdf) [0x48292f]
 12: (Monitor::preinit()+0x679) [0x490a39]
 13: (main()+0x2b61) [0x478361]
 14: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
 15: /usr/bin/ceph-mon() [0x474289]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
  20/20 mon
   0/10 monc
  20/20 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---