Project

General

Profile

Bug #5062 ยป XBqSDkrd.txt

Joao Eduardo Luis, 05/14/2013 09:01 AM

 
2013-05-14 17:40:59.458008 7f230f0bb780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 1471
2013-05-14 17:40:59.902657 7f230f0bb780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:40:59.910734 7f230f0bb780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f230f0bb780 time 2013-05-14 17:40:59.910363
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-26> 2013-05-14 17:40:59.456679 7f230f0bb780 5 asok(0x223f000) register_command perfcounters_dump hook 0x2232010
-25> 2013-05-14 17:40:59.456698 7f230f0bb780 5 asok(0x223f000) register_command 1 hook 0x2232010
-24> 2013-05-14 17:40:59.456700 7f230f0bb780 5 asok(0x223f000) register_command perf dump hook 0x2232010
-23> 2013-05-14 17:40:59.456705 7f230f0bb780 5 asok(0x223f000) register_command perfcounters_schema hook 0x2232010
-22> 2013-05-14 17:40:59.456707 7f230f0bb780 5 asok(0x223f000) register_command 2 hook 0x2232010
-21> 2013-05-14 17:40:59.456708 7f230f0bb780 5 asok(0x223f000) register_command perf schema hook 0x2232010
-20> 2013-05-14 17:40:59.456710 7f230f0bb780 5 asok(0x223f000) register_command config show hook 0x2232010
-19> 2013-05-14 17:40:59.456711 7f230f0bb780 5 asok(0x223f000) register_command config set hook 0x2232010
-18> 2013-05-14 17:40:59.456714 7f230f0bb780 5 asok(0x223f000) register_command log flush hook 0x2232010
-17> 2013-05-14 17:40:59.456716 7f230f0bb780 5 asok(0x223f000) register_command log dump hook 0x2232010
-16> 2013-05-14 17:40:59.456719 7f230f0bb780 5 asok(0x223f000) register_command log reopen hook 0x2232010
-15> 2013-05-14 17:40:59.458008 7f230f0bb780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 1471
-14> 2013-05-14 17:40:59.458377 7f230f0bb780 1 finished global_init_daemonize
-13> 2013-05-14 17:40:59.460046 7f230f0bb780 5 asok(0x223f000) init /var/run/ceph/ceph-mon.0.asok
-12> 2013-05-14 17:40:59.460059 7f230f0bb780 5 asok(0x223f000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
-11> 2013-05-14 17:40:59.460106 7f230f0bb780 5 asok(0x223f000) register_command 0 hook 0x22310b8
-10> 2013-05-14 17:40:59.460111 7f230f0bb780 5 asok(0x223f000) register_command version hook 0x22310b8
-9> 2013-05-14 17:40:59.460115 7f230f0bb780 5 asok(0x223f000) register_command git_version hook 0x22310b8
-8> 2013-05-14 17:40:59.460118 7f230f0bb780 5 asok(0x223f000) register_command help hook 0x22320b0
-7> 2013-05-14 17:40:59.460134 7f230d98e700 5 asok(0x223f000) entry start
-6> 2013-05-14 17:40:59.902603 7f230f0bb780 1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
-5> 2013-05-14 17:40:59.902611 7f230f0bb780 1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
-4> 2013-05-14 17:40:59.902622 7f230f0bb780 5 adding auth protocol: cephx
-3> 2013-05-14 17:40:59.902624 7f230f0bb780 5 adding auth protocol: cephx
-2> 2013-05-14 17:40:59.902657 7f230f0bb780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
-1> 2013-05-14 17:40:59.909012 7f230f0bb780 4 mon.0@-1(probing).mds e4428 new map
0> 2013-05-14 17:40:59.910734 7f230f0bb780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f230f0bb780 time 2013-05-14 17:40:59.910363
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:40:59.911539 7f230f0bb780 -1 *** Caught signal (Aborted) **
in thread 7f230f0bb780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7f23110e4ff0]
3: (gsignal()+0x35) [0x7f230fd361b5]
4: (abort()+0x180) [0x7f230fd38fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f23105cadc5]
6: (()+0xcb166) [0x7f23105c9166]
7: (()+0xcb193) [0x7f23105c9193]
8: (()+0xcb28e) [0x7f23105c928e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2013-05-14 17:40:59.911539 7f230f0bb780 -1 *** Caught signal (Aborted) **
in thread 7f230f0bb780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7f23110e4ff0]
3: (gsignal()+0x35) [0x7f230fd361b5]
4: (abort()+0x180) [0x7f230fd38fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f23105cadc5]
6: (()+0xcb166) [0x7f23105c9166]
7: (()+0xcb193) [0x7f23105c9193]
8: (()+0xcb28e) [0x7f23105c928e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7f230fd22c8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:41:17.113420 7f9580bf6780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 2340
2013-05-14 17:41:17.723588 7f9580bf6780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:41:17.730730 7f9580bf6780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f9580bf6780 time 2013-05-14 17:41:17.730342
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7f958185dc8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-26> 2013-05-14 17:41:17.111709 7f9580bf6780 5 asok(0x18c8000) register_command perfcounters_dump hook 0x18bc010
-25> 2013-05-14 17:41:17.111732 7f9580bf6780 5 asok(0x18c8000) register_command 1 hook 0x18bc010
-24> 2013-05-14 17:41:17.111743 7f9580bf6780 5 asok(0x18c8000) register_command perf dump hook 0x18bc010
-23> 2013-05-14 17:41:17.111747 7f9580bf6780 5 asok(0x18c8000) register_command perfcounters_schema hook 0x18bc010
-22> 2013-05-14 17:41:17.111750 7f9580bf6780 5 asok(0x18c8000) register_command 2 hook 0x18bc010
-21> 2013-05-14 17:41:17.111753 7f9580bf6780 5 asok(0x18c8000) register_command perf schema hook 0x18bc010
-20> 2013-05-14 17:41:17.111755 7f9580bf6780 5 asok(0x18c8000) register_command config show hook 0x18bc010
-19> 2013-05-14 17:41:17.111758 7f9580bf6780 5 asok(0x18c8000) register_command config set hook 0x18bc010
-18> 2013-05-14 17:41:17.111759 7f9580bf6780 5 asok(0x18c8000) register_command log flush hook 0x18bc010
-17> 2013-05-14 17:41:17.111764 7f9580bf6780 5 asok(0x18c8000) register_command log dump hook 0x18bc010
-16> 2013-05-14 17:41:17.111766 7f9580bf6780 5 asok(0x18c8000) register_command log reopen hook 0x18bc010
-15> 2013-05-14 17:41:17.113420 7f9580bf6780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 2340
-14> 2013-05-14 17:41:17.113749 7f9580bf6780 1 finished global_init_daemonize
-13> 2013-05-14 17:41:17.115358 7f9580bf6780 5 asok(0x18c8000) init /var/run/ceph/ceph-mon.0.asok
-12> 2013-05-14 17:41:17.115373 7f9580bf6780 5 asok(0x18c8000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
-11> 2013-05-14 17:41:17.115407 7f9580bf6780 5 asok(0x18c8000) register_command 0 hook 0x18bb0b8
-10> 2013-05-14 17:41:17.115413 7f9580bf6780 5 asok(0x18c8000) register_command version hook 0x18bb0b8
-9> 2013-05-14 17:41:17.115416 7f9580bf6780 5 asok(0x18c8000) register_command git_version hook 0x18bb0b8
-8> 2013-05-14 17:41:17.115420 7f9580bf6780 5 asok(0x18c8000) register_command help hook 0x18bc0b0
-7> 2013-05-14 17:41:17.115437 7f957f4c9700 5 asok(0x18c8000) entry start
-6> 2013-05-14 17:41:17.723528 7f9580bf6780 1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
-5> 2013-05-14 17:41:17.723536 7f9580bf6780 1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
-4> 2013-05-14 17:41:17.723549 7f9580bf6780 5 adding auth protocol: cephx
-3> 2013-05-14 17:41:17.723551 7f9580bf6780 5 adding auth protocol: cephx
-2> 2013-05-14 17:41:17.723588 7f9580bf6780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
-1> 2013-05-14 17:41:17.729667 7f9580bf6780 4 mon.0@-1(probing).mds e4428 new map
0> 2013-05-14 17:41:17.730730 7f9580bf6780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7f9580bf6780 time 2013-05-14 17:41:17.730342
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7f958185dc8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:41:17.731554 7f9580bf6780 -1 *** Caught signal (Aborted) **
in thread 7f9580bf6780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7f9582c1fff0]
3: (gsignal()+0x35) [0x7f95818711b5]
4: (abort()+0x180) [0x7f9581873fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f9582105dc5]
6: (()+0xcb166) [0x7f9582104166]
7: (()+0xcb193) [0x7f9582104193]
8: (()+0xcb28e) [0x7f958210428e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7f958185dc8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2013-05-14 17:41:17.731554 7f9580bf6780 -1 *** Caught signal (Aborted) **
in thread 7f9580bf6780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7f9582c1fff0]
3: (gsignal()+0x35) [0x7f95818711b5]
4: (abort()+0x180) [0x7f9581873fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7f9582105dc5]
6: (()+0xcb166) [0x7f9582104166]
7: (()+0xcb193) [0x7f9582104193]
8: (()+0xcb28e) [0x7f958210428e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7f958185dc8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:44:00.484562 7fbb1b89b780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 8366
2013-05-14 17:44:00.486662 7fbb1b89b780 10 needs_conversion
2013-05-14 17:44:00.843516 7fbb1b89b780 10 obtain_monmap
2013-05-14 17:44:00.843569 7fbb1b89b780 10 obtain_monmap read last committed monmap ver 4
2013-05-14 17:44:00.843727 7fbb1b89b780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
2013-05-14 17:44:00.844305 7fbb1b89b780 10 mon.0@-1(probing) e4 check_fsid cluster_uuid contains '2ff06d61-7b8e-4b10-b39d-138cfcc1ee43'
2013-05-14 17:44:00.844322 7fbb1b89b780 10 mon.0@-1(probing) e4 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?)}
2013-05-14 17:44:00.844334 7fbb1b89b780 10 mon.0@-1(probing) e4 has_ever_joined = 1
2013-05-14 17:44:00.844359 7fbb1b89b780 10 mon.0@-1(probing) e4 init_paxos
2013-05-14 17:44:00.844402 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init last_pn: 516800 accepted_pn: 0 last_committed: 31732523 first_committed: 31732421
2013-05-14 17:44:00.844409 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init
2013-05-14 17:44:00.845089 7fbb1b89b780 7 mon.0@-1(probing).pg v0 update_from_paxos loading latest full pgmap v16013180
2013-05-14 17:44:00.847157 7fbb1b89b780 7 mon.0@-1(probing).pg v16013180 update_from_paxos applying incremental 16013181
2013-05-14 17:44:00.847195 7fbb1b89b780 10 mon.0@-1(probing).pg v16013181 v16013181: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3771KB/s wr, 10op/s
2013-05-14 17:44:00.847773 7fbb1b89b780 7 mon.0@-1(probing).pg v16013181 update_from_paxos applying incremental 16013182
2013-05-14 17:44:00.847933 7fbb1b89b780 10 mon.0@-1(probing).pg v16013182 v16013182: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4826KB/s wr, 39op/s
2013-05-14 17:44:00.847969 7fbb1b89b780 7 mon.0@-1(probing).pg v16013182 update_from_paxos applying incremental 16013183
2013-05-14 17:44:00.848005 7fbb1b89b780 10 mon.0@-1(probing).pg v16013183 v16013183: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3502KB/s wr, 57op/s
2013-05-14 17:44:00.848085 7fbb1b89b780 7 mon.0@-1(probing).pg v16013183 update_from_paxos applying incremental 16013184
2013-05-14 17:44:00.848135 7fbb1b89b780 10 mon.0@-1(probing).pg v16013184 v16013184: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 367KB/s wr, 19op/s
2013-05-14 17:44:00.848167 7fbb1b89b780 7 mon.0@-1(probing).pg v16013184 update_from_paxos applying incremental 16013185
2013-05-14 17:44:00.848338 7fbb1b89b780 10 mon.0@-1(probing).pg v16013185 v16013185: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 7049KB/s wr, 86op/s
2013-05-14 17:44:00.848429 7fbb1b89b780 7 mon.0@-1(probing).pg v16013185 update_from_paxos applying incremental 16013186
2013-05-14 17:44:00.848453 7fbb1b89b780 10 mon.0@-1(probing).pg v16013186 v16013186: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 6514KB/s wr, 60op/s
2013-05-14 17:44:00.848551 7fbb1b89b780 7 mon.0@-1(probing).pg v16013186 update_from_paxos applying incremental 16013187
2013-05-14 17:44:00.848675 7fbb1b89b780 10 mon.0@-1(probing).pg v16013187 v16013187: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4045KB/s wr, 49op/s
2013-05-14 17:44:00.848769 7fbb1b89b780 7 mon.0@-1(probing).pg v16013187 update_from_paxos applying incremental 16013188
2013-05-14 17:44:00.848792 7fbb1b89b780 10 mon.0@-1(probing).pg v16013188 v16013188: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 5804KB/s wr, 97op/s
2013-05-14 17:44:00.848810 7fbb1b89b780 7 mon.0@-1(probing).pg v16013188 update_from_paxos applying incremental 16013189
2013-05-14 17:44:00.848822 7fbb1b89b780 10 mon.0@-1(probing).pg v16013189 v16013189: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 228KB/s wr, 5op/s
2013-05-14 17:44:00.848851 7fbb1b89b780 7 mon.0@-1(probing).pg v16013189 update_from_paxos applying incremental 16013190
2013-05-14 17:44:00.849036 7fbb1b89b780 10 mon.0@-1(probing).pg v16013190 v16013190: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4611KB/s wr, 111op/s
2013-05-14 17:44:00.849060 7fbb1b89b780 7 mon.0@-1(probing).pg v16013190 update_from_paxos applying incremental 16013191
2013-05-14 17:44:00.849093 7fbb1b89b780 10 mon.0@-1(probing).pg v16013191 v16013191: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4445KB/s wr, 113op/s
2013-05-14 17:44:00.849121 7fbb1b89b780 7 mon.0@-1(probing).pg v16013191 update_from_paxos applying incremental 16013192
2013-05-14 17:44:00.849285 7fbb1b89b780 10 mon.0@-1(probing).pg v16013192 v16013192: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1523KB/s wr, 118op/s
2013-05-14 17:44:00.849367 7fbb1b89b780 7 mon.0@-1(probing).pg v16013192 update_from_paxos applying incremental 16013193
2013-05-14 17:44:00.849394 7fbb1b89b780 10 mon.0@-1(probing).pg v16013193 v16013193: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1906KB/s wr, 210op/s
2013-05-14 17:44:00.849495 7fbb1b89b780 7 mon.0@-1(probing).pg v16013193 update_from_paxos applying incremental 16013194
2013-05-14 17:44:00.849512 7fbb1b89b780 10 mon.0@-1(probing).pg v16013194 v16013194: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 202KB/s wr, 30op/s
2013-05-14 17:44:00.849542 7fbb1b89b780 7 mon.0@-1(probing).pg v16013194 update_from_paxos applying incremental 16013195
2013-05-14 17:44:00.849946 7fbb1b89b780 10 mon.0@-1(probing).pg v16013195 v16013195: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 504KB/s wr, 70op/s
2013-05-14 17:44:00.850028 7fbb1b89b780 7 mon.0@-1(probing).pg v16013195 update_from_paxos applying incremental 16013196
2013-05-14 17:44:00.850057 7fbb1b89b780 10 mon.0@-1(probing).pg v16013196 v16013196: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 925KB/s wr, 132op/s
2013-05-14 17:44:00.850135 7fbb1b89b780 7 mon.0@-1(probing).pg v16013196 update_from_paxos applying incremental 16013197
2013-05-14 17:44:00.850155 7fbb1b89b780 10 mon.0@-1(probing).pg v16013197 v16013197: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 225KB/s wr, 36op/s
2013-05-14 17:44:00.850236 7fbb1b89b780 7 mon.0@-1(probing).pg v16013197 update_from_paxos applying incremental 16013198
2013-05-14 17:44:00.850332 7fbb1b89b780 10 mon.0@-1(probing).pg v16013198 v16013198: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 436KB/s wr, 38op/s
2013-05-14 17:44:00.850410 7fbb1b89b780 7 mon.0@-1(probing).pg v16013198 update_from_paxos applying incremental 16013199
2013-05-14 17:44:00.850436 7fbb1b89b780 10 mon.0@-1(probing).pg v16013199 v16013199: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 388KB/s wr, 32op/s
2013-05-14 17:44:00.850533 7fbb1b89b780 7 mon.0@-1(probing).pg v16013199 update_from_paxos applying incremental 16013200
2013-05-14 17:44:00.850907 7fbb1b89b780 10 mon.0@-1(probing).pg v16013200 v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
2013-05-14 17:44:00.850932 7fbb1b89b780 7 mon.0@-1(probing).pg v16013200 update_from_paxos applying incremental 16013201
2013-05-14 17:44:00.850955 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 v16013201: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 813KB/s wr, 73op/s
2013-05-14 17:44:00.850971 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 update_logger
2013-05-14 17:44:00.850988 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos version 4428, my e 0
2013-05-14 17:44:00.851006 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos got 4428
2013-05-14 17:44:00.851021 7fbb1b89b780 4 mon.0@-1(probing).mds e4428 new map
2013-05-14 17:44:00.851024 7fbb1b89b780 7 mon.0@-1(probing).mds e4428 print_map
epoch 4428
flags 0
created 2012-07-25 00:52:15.550913
modified 2013-05-14 14:16:54.190887
tableserver 0
root 0
session_timeout 60
session_autoclose 300
last_failure 4247
last_failure_osd_epoch 3816
compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object}
max_mds 1
in 0
up {0=46584}
failed
stopped
data_pools 0
metadata_pool 1
46584: 188.65.144.4:6800/3230 '0' mds.0.449 up:active seq 60926

2013-05-14 17:44:00.851046 7fbb1b89b780 10 mon.0@-1(probing).mds e4428 update_logger
2013-05-14 17:44:00.851226 7fbb1b89b780 15 mon.0@-1(probing).osd e0 update_from_paxos paxos e 3909, my e 0
2013-05-14 17:44:00.851408 7fbb1b89b780 7 mon.0@-1(probing).osd e0 update_from_paxos loading latest full map e3909
2013-05-14 17:44:00.851475 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 send_to_waiting 3909
2013-05-14 17:44:00.851482 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 share_map_with_random_osd no up osd on our session map
2013-05-14 17:44:00.851493 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 update_logger
2013-05-14 17:44:00.851500 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos
2013-05-14 17:44:00.851527 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos version 15702609 summary v 0
2013-05-14 17:44:00.851707 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos latest full 15702608
2013-05-14 17:44:00.851731 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos loading summary e15702608
2013-05-14 17:44:00.851867 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos loaded summary e15702608
2013-05-14 17:44:00.851891 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos applying incremental log 15702609 2013-05-14 14:34:11.768508 mon.1 188.65.144.5:6789/0 53 : [INF] pgmap v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
2013-05-14 17:44:00.852037 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 check_subs
2013-05-14 17:44:00.852062 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos
2013-05-14 17:44:00.852105 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos version 8372 keys ver 0 latest 8370
2013-05-14 17:44:00.852123 7fbb1b89b780 7 mon.0@-1(probing).auth v8372 update_from_paxos loading summary e 8370
2013-05-14 17:44:00.852242 7fbb1b89b780 7 mon.0@-1(probing).auth v8372 update_from_paxos latest length 1896
2013-05-14 17:44:00.852394 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos key server version 8370
2013-05-14 17:44:00.853232 7fbb1b89b780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7fbb1b89b780 time 2013-05-14 17:44:00.852652
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-100> 2013-05-14 17:44:00.483218 7fbb1b89b780 5 asok(0x1c4d000) register_command perfcounters_dump hook 0x1c41010
-99> 2013-05-14 17:44:00.483232 7fbb1b89b780 5 asok(0x1c4d000) register_command 1 hook 0x1c41010
-98> 2013-05-14 17:44:00.483237 7fbb1b89b780 5 asok(0x1c4d000) register_command perf dump hook 0x1c41010
-97> 2013-05-14 17:44:00.483243 7fbb1b89b780 5 asok(0x1c4d000) register_command perfcounters_schema hook 0x1c41010
-96> 2013-05-14 17:44:00.483246 7fbb1b89b780 5 asok(0x1c4d000) register_command 2 hook 0x1c41010
-95> 2013-05-14 17:44:00.483248 7fbb1b89b780 5 asok(0x1c4d000) register_command perf schema hook 0x1c41010
-94> 2013-05-14 17:44:00.483251 7fbb1b89b780 5 asok(0x1c4d000) register_command config show hook 0x1c41010
-93> 2013-05-14 17:44:00.483254 7fbb1b89b780 5 asok(0x1c4d000) register_command config set hook 0x1c41010
-92> 2013-05-14 17:44:00.483256 7fbb1b89b780 5 asok(0x1c4d000) register_command log flush hook 0x1c41010
-91> 2013-05-14 17:44:00.483260 7fbb1b89b780 5 asok(0x1c4d000) register_command log dump hook 0x1c41010
-90> 2013-05-14 17:44:00.483262 7fbb1b89b780 5 asok(0x1c4d000) register_command log reopen hook 0x1c41010
-89> 2013-05-14 17:44:00.484562 7fbb1b89b780 0 ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60), process ceph-mon, pid 8366
-88> 2013-05-14 17:44:00.484924 7fbb1b89b780 1 finished global_init_daemonize
-87> 2013-05-14 17:44:00.486586 7fbb1b89b780 5 asok(0x1c4d000) init /var/run/ceph/ceph-mon.0.asok
-86> 2013-05-14 17:44:00.486599 7fbb1b89b780 5 asok(0x1c4d000) bind_and_listen /var/run/ceph/ceph-mon.0.asok
-85> 2013-05-14 17:44:00.486629 7fbb1b89b780 5 asok(0x1c4d000) register_command 0 hook 0x1c400b8
-84> 2013-05-14 17:44:00.486633 7fbb1b89b780 5 asok(0x1c4d000) register_command version hook 0x1c400b8
-83> 2013-05-14 17:44:00.486636 7fbb1b89b780 5 asok(0x1c4d000) register_command git_version hook 0x1c400b8
-82> 2013-05-14 17:44:00.486638 7fbb1b89b780 5 asok(0x1c4d000) register_command help hook 0x1c410b0
-81> 2013-05-14 17:44:00.486662 7fbb1b89b780 10 needs_conversion
-80> 2013-05-14 17:44:00.486668 7fbb1a16e700 5 asok(0x1c4d000) entry start
-79> 2013-05-14 17:44:00.843516 7fbb1b89b780 10 obtain_monmap
-78> 2013-05-14 17:44:00.843569 7fbb1b89b780 10 obtain_monmap read last committed monmap ver 4
-77> 2013-05-14 17:44:00.843671 7fbb1b89b780 1 -- 188.65.144.4:6789/0 learned my addr 188.65.144.4:6789/0
-76> 2013-05-14 17:44:00.843678 7fbb1b89b780 1 accepter.accepter.bind my_inst.addr is 188.65.144.4:6789/0 need_addr=0
-75> 2013-05-14 17:44:00.843689 7fbb1b89b780 5 adding auth protocol: cephx
-74> 2013-05-14 17:44:00.843691 7fbb1b89b780 5 adding auth protocol: cephx
-73> 2013-05-14 17:44:00.843727 7fbb1b89b780 1 mon.0@-1(probing) e4 preinit fsid 2ff06d61-7b8e-4b10-b39d-138cfcc1ee43
-72> 2013-05-14 17:44:00.844305 7fbb1b89b780 10 mon.0@-1(probing) e4 check_fsid cluster_uuid contains '2ff06d61-7b8e-4b10-b39d-138cfcc1ee43'
-71> 2013-05-14 17:44:00.844322 7fbb1b89b780 10 mon.0@-1(probing) e4 features compat={},rocompat={},incompat={1=initial feature set (~v.18),3=single paxos with k/v store (v0.?)}
-70> 2013-05-14 17:44:00.844334 7fbb1b89b780 10 mon.0@-1(probing) e4 has_ever_joined = 1
-69> 2013-05-14 17:44:00.844359 7fbb1b89b780 10 mon.0@-1(probing) e4 init_paxos
-68> 2013-05-14 17:44:00.844402 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init last_pn: 516800 accepted_pn: 0 last_committed: 31732523 first_committed: 31732421
-67> 2013-05-14 17:44:00.844409 7fbb1b89b780 10 mon.0@-1(probing).paxos(paxos recovering c 31732421..31732523) init
-66> 2013-05-14 17:44:00.845089 7fbb1b89b780 7 mon.0@-1(probing).pg v0 update_from_paxos loading latest full pgmap v16013180
-65> 2013-05-14 17:44:00.847157 7fbb1b89b780 7 mon.0@-1(probing).pg v16013180 update_from_paxos applying incremental 16013181
-64> 2013-05-14 17:44:00.847195 7fbb1b89b780 10 mon.0@-1(probing).pg v16013181 v16013181: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3771KB/s wr, 10op/s
-63> 2013-05-14 17:44:00.847773 7fbb1b89b780 7 mon.0@-1(probing).pg v16013181 update_from_paxos applying incremental 16013182
-62> 2013-05-14 17:44:00.847933 7fbb1b89b780 10 mon.0@-1(probing).pg v16013182 v16013182: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4826KB/s wr, 39op/s
-61> 2013-05-14 17:44:00.847969 7fbb1b89b780 7 mon.0@-1(probing).pg v16013182 update_from_paxos applying incremental 16013183
-60> 2013-05-14 17:44:00.848005 7fbb1b89b780 10 mon.0@-1(probing).pg v16013183 v16013183: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 3502KB/s wr, 57op/s
-59> 2013-05-14 17:44:00.848085 7fbb1b89b780 7 mon.0@-1(probing).pg v16013183 update_from_paxos applying incremental 16013184
-58> 2013-05-14 17:44:00.848135 7fbb1b89b780 10 mon.0@-1(probing).pg v16013184 v16013184: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 367KB/s wr, 19op/s
-57> 2013-05-14 17:44:00.848167 7fbb1b89b780 7 mon.0@-1(probing).pg v16013184 update_from_paxos applying incremental 16013185
-56> 2013-05-14 17:44:00.848338 7fbb1b89b780 10 mon.0@-1(probing).pg v16013185 v16013185: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 7049KB/s wr, 86op/s
-55> 2013-05-14 17:44:00.848429 7fbb1b89b780 7 mon.0@-1(probing).pg v16013185 update_from_paxos applying incremental 16013186
-54> 2013-05-14 17:44:00.848453 7fbb1b89b780 10 mon.0@-1(probing).pg v16013186 v16013186: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 6514KB/s wr, 60op/s
-53> 2013-05-14 17:44:00.848551 7fbb1b89b780 7 mon.0@-1(probing).pg v16013186 update_from_paxos applying incremental 16013187
-52> 2013-05-14 17:44:00.848675 7fbb1b89b780 10 mon.0@-1(probing).pg v16013187 v16013187: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4045KB/s wr, 49op/s
-51> 2013-05-14 17:44:00.848769 7fbb1b89b780 7 mon.0@-1(probing).pg v16013187 update_from_paxos applying incremental 16013188
-50> 2013-05-14 17:44:00.848792 7fbb1b89b780 10 mon.0@-1(probing).pg v16013188 v16013188: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 5804KB/s wr, 97op/s
-49> 2013-05-14 17:44:00.848810 7fbb1b89b780 7 mon.0@-1(probing).pg v16013188 update_from_paxos applying incremental 16013189
-48> 2013-05-14 17:44:00.848822 7fbb1b89b780 10 mon.0@-1(probing).pg v16013189 v16013189: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 228KB/s wr, 5op/s
-47> 2013-05-14 17:44:00.848851 7fbb1b89b780 7 mon.0@-1(probing).pg v16013189 update_from_paxos applying incremental 16013190
-46> 2013-05-14 17:44:00.849036 7fbb1b89b780 10 mon.0@-1(probing).pg v16013190 v16013190: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4611KB/s wr, 111op/s
-45> 2013-05-14 17:44:00.849060 7fbb1b89b780 7 mon.0@-1(probing).pg v16013190 update_from_paxos applying incremental 16013191
-44> 2013-05-14 17:44:00.849093 7fbb1b89b780 10 mon.0@-1(probing).pg v16013191 v16013191: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 4445KB/s wr, 113op/s
-43> 2013-05-14 17:44:00.849121 7fbb1b89b780 7 mon.0@-1(probing).pg v16013191 update_from_paxos applying incremental 16013192
-42> 2013-05-14 17:44:00.849285 7fbb1b89b780 10 mon.0@-1(probing).pg v16013192 v16013192: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1523KB/s wr, 118op/s
-41> 2013-05-14 17:44:00.849367 7fbb1b89b780 7 mon.0@-1(probing).pg v16013192 update_from_paxos applying incremental 16013193
-40> 2013-05-14 17:44:00.849394 7fbb1b89b780 10 mon.0@-1(probing).pg v16013193 v16013193: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 1906KB/s wr, 210op/s
-39> 2013-05-14 17:44:00.849495 7fbb1b89b780 7 mon.0@-1(probing).pg v16013193 update_from_paxos applying incremental 16013194
-38> 2013-05-14 17:44:00.849512 7fbb1b89b780 10 mon.0@-1(probing).pg v16013194 v16013194: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 202KB/s wr, 30op/s
-37> 2013-05-14 17:44:00.849542 7fbb1b89b780 7 mon.0@-1(probing).pg v16013194 update_from_paxos applying incremental 16013195
-36> 2013-05-14 17:44:00.849946 7fbb1b89b780 10 mon.0@-1(probing).pg v16013195 v16013195: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 504KB/s wr, 70op/s
-35> 2013-05-14 17:44:00.850028 7fbb1b89b780 7 mon.0@-1(probing).pg v16013195 update_from_paxos applying incremental 16013196
-34> 2013-05-14 17:44:00.850057 7fbb1b89b780 10 mon.0@-1(probing).pg v16013196 v16013196: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 925KB/s wr, 132op/s
-33> 2013-05-14 17:44:00.850135 7fbb1b89b780 7 mon.0@-1(probing).pg v16013196 update_from_paxos applying incremental 16013197
-32> 2013-05-14 17:44:00.850155 7fbb1b89b780 10 mon.0@-1(probing).pg v16013197 v16013197: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 225KB/s wr, 36op/s
-31> 2013-05-14 17:44:00.850236 7fbb1b89b780 7 mon.0@-1(probing).pg v16013197 update_from_paxos applying incremental 16013198
-30> 2013-05-14 17:44:00.850332 7fbb1b89b780 10 mon.0@-1(probing).pg v16013198 v16013198: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 436KB/s wr, 38op/s
-29> 2013-05-14 17:44:00.850410 7fbb1b89b780 7 mon.0@-1(probing).pg v16013198 update_from_paxos applying incremental 16013199
-28> 2013-05-14 17:44:00.850436 7fbb1b89b780 10 mon.0@-1(probing).pg v16013199 v16013199: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 388KB/s wr, 32op/s
-27> 2013-05-14 17:44:00.850533 7fbb1b89b780 7 mon.0@-1(probing).pg v16013199 update_from_paxos applying incremental 16013200
-26> 2013-05-14 17:44:00.850907 7fbb1b89b780 10 mon.0@-1(probing).pg v16013200 v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
-25> 2013-05-14 17:44:00.850932 7fbb1b89b780 7 mon.0@-1(probing).pg v16013200 update_from_paxos applying incremental 16013201
-24> 2013-05-14 17:44:00.850955 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 v16013201: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 813KB/s wr, 73op/s
-23> 2013-05-14 17:44:00.850971 7fbb1b89b780 10 mon.0@-1(probing).pg v16013201 update_logger
-22> 2013-05-14 17:44:00.850988 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos version 4428, my e 0
-21> 2013-05-14 17:44:00.851006 7fbb1b89b780 10 mon.0@-1(probing).mds e0 update_from_paxos got 4428
-20> 2013-05-14 17:44:00.851021 7fbb1b89b780 4 mon.0@-1(probing).mds e4428 new map
-19> 2013-05-14 17:44:00.851024 7fbb1b89b780 7 mon.0@-1(probing).mds e4428 print_map
epoch 4428
flags 0
created 2012-07-25 00:52:15.550913
modified 2013-05-14 14:16:54.190887
tableserver 0
root 0
session_timeout 60
session_autoclose 300
last_failure 4247
last_failure_osd_epoch 3816
compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object}
max_mds 1
in 0
up {0=46584}
failed
stopped
data_pools 0
metadata_pool 1
46584: 188.65.144.4:6800/3230 '0' mds.0.449 up:active seq 60926

-18> 2013-05-14 17:44:00.851046 7fbb1b89b780 10 mon.0@-1(probing).mds e4428 update_logger
-17> 2013-05-14 17:44:00.851226 7fbb1b89b780 15 mon.0@-1(probing).osd e0 update_from_paxos paxos e 3909, my e 0
-16> 2013-05-14 17:44:00.851408 7fbb1b89b780 7 mon.0@-1(probing).osd e0 update_from_paxos loading latest full map e3909
-15> 2013-05-14 17:44:00.851475 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 send_to_waiting 3909
-14> 2013-05-14 17:44:00.851482 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 share_map_with_random_osd no up osd on our session map
-13> 2013-05-14 17:44:00.851493 7fbb1b89b780 10 mon.0@-1(probing).osd e3909 update_logger
-12> 2013-05-14 17:44:00.851500 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos
-11> 2013-05-14 17:44:00.851527 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos version 15702609 summary v 0
-10> 2013-05-14 17:44:00.851707 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 update_from_paxos latest full 15702608
-9> 2013-05-14 17:44:00.851731 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos loading summary e15702608
-8> 2013-05-14 17:44:00.851867 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos loaded summary e15702608
-7> 2013-05-14 17:44:00.851891 7fbb1b89b780 7 mon.0@-1(probing).log v15702609 update_from_paxos applying incremental log 15702609 2013-05-14 14:34:11.768508 mon.1 188.65.144.5:6789/0 53 : [INF] pgmap v16013200: 768 pgs: 768 active+clean; 1538 GB data, 3076 GB used, 3097 GB / 6359 GB avail; 827KB/s wr, 71op/s
-6> 2013-05-14 17:44:00.852037 7fbb1b89b780 10 mon.0@-1(probing).log v15702609 check_subs
-5> 2013-05-14 17:44:00.852062 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos
-4> 2013-05-14 17:44:00.852105 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos version 8372 keys ver 0 latest 8370
-3> 2013-05-14 17:44:00.852123 7fbb1b89b780 7 mon.0@-1(probing).auth v8372 update_from_paxos loading summary e 8370
-2> 2013-05-14 17:44:00.852242 7fbb1b89b780 7 mon.0@-1(probing).auth v8372 update_from_paxos latest length 1896
-1> 2013-05-14 17:44:00.852394 7fbb1b89b780 10 mon.0@-1(probing).auth v8372 update_from_paxos key server version 8370
0> 2013-05-14 17:44:00.853232 7fbb1b89b780 -1 mon/AuthMonitor.cc: In function 'virtual void AuthMonitor::update_from_paxos()' thread 7fbb1b89b780 time 2013-05-14 17:44:00.852652
mon/AuthMonitor.cc: 147: FAILED assert(ret == 0)

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
2: (Monitor::init_paxos()+0xdf) [0x48292f]
3: (Monitor::preinit()+0x679) [0x490a39]
4: (main()+0x2b61) [0x478361]
5: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
6: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
20/20 mon
0/10 monc
20/20 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
2013-05-14 17:44:00.854313 7fbb1b89b780 -1 *** Caught signal (Aborted) **
in thread 7fbb1b89b780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7fbb1d8c4ff0]
3: (gsignal()+0x35) [0x7fbb1c5161b5]
4: (abort()+0x180) [0x7fbb1c518fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7fbb1cdaadc5]
6: (()+0xcb166) [0x7fbb1cda9166]
7: (()+0xcb193) [0x7fbb1cda9193]
8: (()+0xcb28e) [0x7fbb1cda928e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2013-05-14 17:44:00.854313 7fbb1b89b780 -1 *** Caught signal (Aborted) **
in thread 7fbb1b89b780

ceph version 0.61.2 (fea782543a844bb277ae94d3391788b76c5bee60)
1: /usr/bin/ceph-mon() [0x5957c9]
2: (()+0xeff0) [0x7fbb1d8c4ff0]
3: (gsignal()+0x35) [0x7fbb1c5161b5]
4: (abort()+0x180) [0x7fbb1c518fc0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x115) [0x7fbb1cdaadc5]
6: (()+0xcb166) [0x7fbb1cda9166]
7: (()+0xcb193) [0x7fbb1cda9193]
8: (()+0xcb28e) [0x7fbb1cda928e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7c9) [0x5bb2c9]
10: (AuthMonitor::update_from_paxos()+0x25ad) [0x56f44d]
11: (Monitor::init_paxos()+0xdf) [0x48292f]
12: (Monitor::preinit()+0x679) [0x490a39]
13: (main()+0x2b61) [0x478361]
14: (__libc_start_main()+0xfd) [0x7fbb1c502c8d]
15: /usr/bin/ceph-mon() [0x474289]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
20/20 mon
0/10 monc
20/20 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-mon.0.log
--- end dump of recent events ---
    (1-1/1)