Project

General

Profile

Bug #1047 » mds.log

Fyodor Ustinov, 05/06/2011 04:42 AM

 
2011-05-03 15:21:48.473024 7fa0b2e2a700 mds-1.0 ms_handle_connect on 77.120.112.193:6789/0
2011-05-03 15:21:50.262309 7fa0b0d25700 -- 77.120.112.193:6800/16763 >> 77.120.104.104:0/2755595419 pipe(0x1a77a50 sd=6 pgs=0 cs=0 l=0).accept peer addr is really 77.120.104.104:0/2755595419 (socket is 77.120.104.104:36278/0)
2011-05-03 15:21:50.262370 7fa0b0d25700 -- 77.120.112.193:6800/16763 >> 77.120.104.104:0/2755595419 pipe(0x1a77a50 sd=6 pgs=0 cs=0 l=0).accept we reset (peer sent cseq 1), sending RESETSESSION
2011-05-03 15:21:52.478861 7fa0b2e2a700 mds-1.0 handle_mds_map standby
2011-05-03 15:22:04.076065 7fa0b2e2a700 mds0.7 handle_mds_map i am now mds0.7
2011-05-03 15:22:04.076083 7fa0b2e2a700 mds0.7 handle_mds_map state change up:standby --> up:replay
2011-05-03 15:22:04.076091 7fa0b2e2a700 mds0.7 replay_start
2011-05-03 15:22:04.076101 7fa0b2e2a700 mds0.7 recovery set is
2011-05-03 15:22:04.076109 7fa0b2e2a700 mds0.7 need osdmap epoch 54, have 53
2011-05-03 15:22:04.076140 7fa0b2e2a700 mds0.cache handle_mds_failure mds0 : recovery peers are
2011-05-03 15:22:04.077382 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.207:6800/12057
2011-05-03 15:22:04.078141 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.77:6800/9822
2011-05-03 15:22:04.078183 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.72:6800/9957
2011-05-03 15:22:04.078241 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.78:6800/9696
2011-05-03 15:22:04.086732 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.208:6800/12004
2011-05-03 15:22:04.086757 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.200:6800/13221
2011-05-03 15:22:04.104590 7fa0b2e2a700 mds0.cache creating system inode with ino:100
2011-05-03 15:22:04.104804 7fa0b2e2a700 mds0.cache creating system inode with ino:1
2011-05-03 15:22:04.105723 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.209:6800/12232
2011-05-03 15:22:04.105745 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.204:6800/12065
2011-05-03 15:22:04.105825 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.205:6800/12219
2011-05-03 15:22:04.105848 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.202:6800/13279
2011-05-03 15:22:04.113888 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.201:6800/14073
2011-05-03 15:22:04.114785 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.75:6800/9766
2011-05-03 15:22:04.114807 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.73:6800/9942
2011-05-03 15:22:04.581386 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.79:6800/9890
2011-05-03 15:22:04.691403 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.70:6800/11390
2011-05-03 15:22:04.691428 7fa0b2e2a700 mds0.7 ms_handle_connect on 77.120.112.206:6800/12188
2011-05-03 15:22:04.854370 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.74:6800/9624
2011-05-03 15:22:04.904559 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.71:6800/10429
2011-05-03 15:22:05.172045 7fa0ab8f8700 mds0.7 replay_done in=1 failed=0
2011-05-03 15:22:05.174740 7fa0b2e2a700 mds0.7 handle_mds_map i am now mds0.7
2011-05-03 15:22:05.174753 7fa0b2e2a700 mds0.7 handle_mds_map state change up:replay --> up:reconnect
2011-05-03 15:22:05.174760 7fa0b2e2a700 mds0.7 reconnect_start
2011-05-03 15:22:05.174766 7fa0b2e2a700 mds0.7 reopen_log
2011-05-03 15:22:05.174780 7fa0b2e2a700 mds0.server reconnect_clients -- 1 sessions
2011-05-03 15:22:05.176338 7fa0aaaf1700 -- 77.120.112.193:6800/16763 >> 77.120.104.104:0/2755595419 pipe(0x1ce1af0 sd=24 pgs=0 cs=0 l=0).accept peer addr is really 77.120.104.104:0/2755595419 (socket is 77.120.104.104:36279/0)
2011-05-03 15:22:05.176959 7fa0b2e2a700 log [DBG] : reconnect by client4209 77.120.104.104:0/2755595419 after 0.002125
2011-05-03 15:22:05.177164 7fa0b2e2a700 mds0.server missing 100000082b6 #1/dcvolia/backup03/cpool/d (mine), will load later
2011-05-03 15:22:05.177190 7fa0b2e2a700 mds0.server missing 100000082c1 #1/dcvolia/backup03/cpool/a (mine), will load later
2011-05-03 15:22:05.177217 7fa0b2e2a700 mds0.server missing 100000082e7 #1/dcvolia/backup03/cpool/9/a (mine), will load later
2011-05-03 15:22:05.177229 7fa0b2e2a700 mds0.server missing 100000082f5 #1/dcvolia/backup03/cpool/b (mine), will load later
2011-05-03 15:22:05.177239 7fa0b2e2a700 mds0.server missing 10000008310 #1/dcvolia/backup03/cpool/c (mine), will load later
2011-05-03 15:22:05.177249 7fa0b2e2a700 mds0.server missing 10000008332 #1/dcvolia/backup03/cpool/e (mine), will load later
2011-05-03 15:22:05.177260 7fa0b2e2a700 mds0.server missing 100000083d3 #1/dcvolia/backup03/cpool/9/f (mine), will load later
2011-05-03 15:22:05.177271 7fa0b2e2a700 mds0.server missing 100000083e6 #1/dcvolia/backup03/cpool/9/d (mine), will load later
2011-05-03 15:22:05.177281 7fa0b2e2a700 mds0.server missing 10000008420 #1/dcvolia/backup03/cpool/9/9 (mine), will load later
2011-05-03 15:22:05.177292 7fa0b2e2a700 mds0.server missing 10000008498 #1/dcvolia/backup03/cpool/9/e (mine), will load later
2011-05-03 15:22:05.177306 7fa0b2e2a700 mds0.server missing 100000084b3 #1/dcvolia/backup03/cpool/9/8/f (mine), will load later
2011-05-03 15:22:05.177317 7fa0b2e2a700 mds0.server missing 10000008584 #1/dcvolia/backup03/cpool/9/c (mine), will load later
2011-05-03 15:22:05.177328 7fa0b2e2a700 mds0.server missing 100000085a1 #1/dcvolia/backup03/cpool/9/b (mine), will load later
2011-05-03 15:22:05.177339 7fa0b2e2a700 mds0.server missing 100000085ce #1/dcvolia/backup03/cpool/9/8/c (mine), will load later
2011-05-03 15:22:05.177350 7fa0b2e2a700 mds0.server missing 100000089af #1/dcvolia/backup03/cpool/9/8/b (mine), will load later
2011-05-03 15:22:05.177361 7fa0b2e2a700 mds0.server missing 10000008c0d #1/dcvolia/backup03/cpool/9/8/e (mine), will load later
2011-05-03 15:22:05.177372 7fa0b2e2a700 mds0.server missing 10000008dbe #1/dcvolia/backup03/cpool/9/8/a (mine), will load later
2011-05-03 15:22:05.177383 7fa0b2e2a700 mds0.server missing 10000008ddd #1/dcvolia/backup03/cpool/9/8/9 (mine), will load later
2011-05-03 15:22:05.177394 7fa0b2e2a700 mds0.server missing 1000000932c #1/dcvolia/backup03/cpool/9/8/d (mine), will load later
2011-05-03 15:22:05.177402 7fa0b2e2a700 mds0.7 reconnect_done
2011-05-03 15:22:05.179664 7fa0b2e2a700 mds0.7 handle_mds_map i am now mds0.7
2011-05-03 15:22:05.179681 7fa0b2e2a700 mds0.7 handle_mds_map state change up:reconnect --> up:rejoin
2011-05-03 15:22:05.179693 7fa0b2e2a700 mds0.7 rejoin_joint_start
2011-05-03 15:22:05.182930 7fa0b2e2a700 mds0.7 ms_handle_connect on 82.144.220.76:6800/9772
2011-05-03 15:22:05.192186 7fa0b2e2a700 mds0.7 rejoin_done
2011-05-03 15:22:05.196966 7fa0b2e2a700 mds0.7 handle_mds_map i am now mds0.7
2011-05-03 15:22:05.196980 7fa0b2e2a700 mds0.7 handle_mds_map state change up:rejoin --> up:active
2011-05-03 15:22:05.196987 7fa0b2e2a700 mds0.7 recovery_done -- successful recovery!
2011-05-03 15:22:05.197367 7fa0b2e2a700 mds0.7 active_start
2011-05-03 15:22:05.210712 7fa0b2e2a700 mds0.7 cluster recovered.
mds/AnchorServer.cc: In function 'virtual void AnchorServer::handle_query(MMDSTableRequest*)', in thread '0x7fa0b2e2a700'
mds/AnchorServer.cc: 242: FAILED assert(anchor_map.count(curino) == 1)
ceph version 0.27 (commit:793034c62c8e9ffab4af675ca97135fd1b193c9c)
1: (AnchorServer::handle_query(MMDSTableRequest*)+0x115) [0x664c65]
2: (MDS::handle_deferrable_message(Message*)+0xae4) [0x4a0564]
3: (MDS::_dispatch(Message*)+0x146e) [0x4b132e]
4: (MDS::ms_dispatch(Message*)+0x5b) [0x4b1aeb]
5: (SimpleMessenger::dispatch_entry()+0x7ea) [0x483eba]
6: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x47a95c]
7: (()+0x6d8c) [0x7fa0b5060d8c]
8: (clone()+0x6d) [0x7fa0b3f1304d]
ceph version 0.27 (commit:793034c62c8e9ffab4af675ca97135fd1b193c9c)
1: (AnchorServer::handle_query(MMDSTableRequest*)+0x115) [0x664c65]
2: (MDS::handle_deferrable_message(Message*)+0xae4) [0x4a0564]
3: (MDS::_dispatch(Message*)+0x146e) [0x4b132e]
4: (MDS::ms_dispatch(Message*)+0x5b) [0x4b1aeb]
5: (SimpleMessenger::dispatch_entry()+0x7ea) [0x483eba]
6: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x47a95c]
7: (()+0x6d8c) [0x7fa0b5060d8c]
8: (clone()+0x6d) [0x7fa0b3f1304d]
*** Caught signal (Aborted) **
in thread 0x7fa0b2e2a700
ceph version 0.27 (commit:793034c62c8e9ffab4af675ca97135fd1b193c9c)
1: /usr/bin/cmds() [0x70fc5e]
2: (()+0xfc60) [0x7fa0b5069c60]
3: (gsignal()+0x35) [0x7fa0b3e60d05]
4: (abort()+0x186) [0x7fa0b3e64ab6]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fa0b47176dd]
6: (()+0xb9926) [0x7fa0b4715926]
7: (()+0xb9953) [0x7fa0b4715953]
8: (()+0xb9a5e) [0x7fa0b4715a5e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x36a) [0x6f3b4a]
10: (AnchorServer::handle_query(MMDSTableRequest*)+0x115) [0x664c65]
11: (MDS::handle_deferrable_message(Message*)+0xae4) [0x4a0564]
12: (MDS::_dispatch(Message*)+0x146e) [0x4b132e]
13: (MDS::ms_dispatch(Message*)+0x5b) [0x4b1aeb]
14: (SimpleMessenger::dispatch_entry()+0x7ea) [0x483eba]
15: (SimpleMessenger::DispatchThread::entry()+0x1c) [0x47a95c]
16: (()+0x6d8c) [0x7fa0b5060d8c]
17: (clone()+0x6d) [0x7fa0b3f1304d]

(1-1/3)