Project

General

Profile

Bug #665 ยป mds.0.log

mds log - longguang yue, 12/20/2010 09:20 PM

 
2010-12-21 03:23:05.658760 --- 13469 appending to log /var/log/ceph/mds.0.log ---
ceph version 0.22~rc ()
2010-12-21 03:23:45.667430 7f97db206710 monclient: hunting for new mon
2010-12-21 03:23:45.667531 7f97db206710 mds0.13 ms_handle_reset on 192.168.1.102:6789/0
2010-12-21 03:23:45.668791 7f97db206710 mds0.13 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 03:23:57.674350 7f97d9d03710 mds0.13 beacon_kill last_acked_stamp 2010-12-21 03:23:42.674160, setting laggy flag.
2010-12-21 03:23:58.405327 7f97db206710 mds0.13 clearing laggy flag
2010-12-21 04:29:32.781045 7f97db206710 monclient: hunting for new mon
2010-12-21 04:29:32.781126 7f97db206710 mds0.13 ms_handle_reset on 192.168.1.104:6789/0
2010-12-21 04:29:32.782246 7f97db206710 mds0.13 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:29:34.407447 7f97db206710 monclient: hunting for new mon
2010-12-21 04:29:34.407515 7f97db206710 mds0.13 ms_handle_reset on 192.168.1.103:6789/0
2010-12-21 04:29:34.408356 7f97d9e04710 -- 192.168.1.99:6800/13468 >> 192.168.1.102:6789/0 pipe(0x7f97d0007ca0 sd=-1 pgs=0 cs=0 l=0).fault first fault
2010-12-21 04:30:00.305626 --- 14800 appending to log /var/log/ceph/mds.0.log ---
ceph version 0.22~rc ()
2010-12-21 04:30:00.307723 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:03.307873 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:06.308364 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:09.309063 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:12.309145 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:15.309397 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:18.309762 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:21.310247 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:24.310357 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:27.310872 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:30.311017 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:33.311380 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:36.311770 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:39.312018 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:42.312464 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:45.312882 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:48.313157 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:30:51.313397 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:30:54.313617 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:30:57.313843 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:00.314342 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:03.314489 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:06.314951 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:09.315521 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:12.315722 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:15.316094 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:18.316557 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:21.316743 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:24.317150 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:27.317310 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:30.317770 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:33.318119 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:36.318355 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:39.318769 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:42.319027 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:45.319536 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:48.319959 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:31:51.320222 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:31:54.320446 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:31:57.320890 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:32:00.321151 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:32:03.321377 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:32:06.321746 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:32:09.321901 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:32:12.322233 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:32:15.322532 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:32:18.322937 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.103:6789/0
2010-12-21 04:32:21.323402 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:32:24.323654 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.102:6789/0
2010-12-21 04:32:27.324118 7f1253553710 mds-1.0 ms_handle_connect on 192.168.1.104:6789/0
2010-12-21 04:32:33.938354 7f1253553710 mds-1.0 handle_mds_map standby
2010-12-21 04:32:49.311437 7f1253553710 mds0.14 handle_mds_map i am now mds0.14
2010-12-21 04:32:49.311456 7f1253553710 mds0.14 handle_mds_map state change up:standby --> up:replay
2010-12-21 04:32:49.311468 7f1253553710 mds0.14 replay_start
2010-12-21 04:32:49.311477 7f1253553710 mds0.14 now replay. my recovery peers are
2010-12-21 04:32:49.311618 7f1253553710 mds0.cache handle_mds_failure mds0 : recovery peers are
2010-12-21 04:32:49.312325 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:49.312394 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=-1 pgs=0 cs=0 l=0).fault first fault
2010-12-21 04:32:49.312883 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:49.312915 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:49.312974 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=-1 pgs=0 cs=0 l=0).fault first fault
2010-12-21 04:32:49.313505 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:49.513919 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:49.514303 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:49.914951 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:49.915340 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:50.716005 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:50.716311 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:52.316974 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:52.317314 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:55.517965 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:55.518279 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:59.323778 7f125014c710 -- 192.168.1.99:6800/14800 >> 192.168.1.104:6800/24058 pipe(0x7f123c003d10 sd=11 pgs=0 cs=0 l=0).connect claims to be 192.168.1.104:6800/25855 not 192.168.1.104:6800/24058 - wrong node!
2010-12-21 04:32:59.323838 7f1256054710 -- 192.168.1.99:6800/14800 >> 192.168.1.103:6800/4028 pipe(0x7f123c003290 sd=10 pgs=0 cs=0 l=0).connect claims to be 192.168.1.103:6800/5799 not 192.168.1.103:6800/4028 - wrong node!
2010-12-21 04:32:59.324694 7f1253553710 mds0.14 ms_handle_connect on 192.168.1.104:6800/25855
2010-12-21 04:32:59.325173 7f1253553710 mds0.14 ms_handle_connect on 192.168.1.103:6800/5799
    (1-1/1)