Project

General

Profile

ceph-mds.v6.us-imm-cephmon1.log

Aurimas Lapiene, 11/25/2016 02:28 PM

Download (9.35 KB)

 
1
2016-11-25 12:13:46.458000 7f0f7b30a700  1 mds.0.141894 handle_mds_map i am now mds.0.141894
2
2016-11-25 12:13:46.458005 7f0f7b30a700  1 mds.0.141894 handle_mds_map state change up:boot --> up:replay
3
2016-11-25 12:13:46.458013 7f0f7b30a700  1 mds.0.141894 replay_start
4
2016-11-25 12:13:46.458018 7f0f7b30a700  1 mds.0.141894  recovery set is
5
2016-11-25 12:13:46.458022 7f0f7b30a700  1 mds.0.141894  waiting for osdmap 3941 (which blacklists prior instance)
6
2016-11-25 12:13:47.497733 7f0f76a00700  0 mds.0.cache creating system inode with ino:100
7
2016-11-25 12:13:47.497888 7f0f76a00700  0 mds.0.cache creating system inode with ino:1
8
2016-11-25 12:13:49.816424 7f0f74bf7700  1 mds.0.141894 replay_done
9
2016-11-25 12:13:49.816439 7f0f74bf7700  1 mds.0.141894 making mds journal writeable
10
2016-11-25 12:13:50.494105 7f0f7b30a700  1 mds.0.141894 handle_mds_map i am now mds.0.141894
11
2016-11-25 12:13:50.494108 7f0f7b30a700  1 mds.0.141894 handle_mds_map state change up:replay --> up:reconnect
12
2016-11-25 12:13:50.494118 7f0f7b30a700  1 mds.0.141894 reconnect_start
13
2016-11-25 12:13:50.494119 7f0f7b30a700  1 mds.0.141894 reopen_log
14
2016-11-25 12:13:50.494124 7f0f7b30a700  1 mds.0.server reconnect_clients -- 6 sessions
15
2016-11-25 12:13:50.871887 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.3935504 [2a02:4780:bad:f00d::4]:0/319192201 after 0.377741
16
2016-11-25 12:13:50.901690 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.13091763 [2a02:4780:bad:c0de::2]:0/3523720559 after 0.407552
17
2016-11-25 12:13:50.939035 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.9022264 [2a02:4780:bad:c0de::3]:0/4069917085 after 0.444895
18
2016-11-25 12:13:51.117611 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.3857990 [2a02:4780:bad:f00d::3]:0/2232148986 after 0.623470
19
2016-11-25 12:13:51.549062 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.9435910 [2a02:4780:bad:c0de::10]:0/1406418688 after 1.054921
20
2016-11-25 12:13:51.564713 7f0f7b30a700  0 log_channel(cluster) log [DBG] : reconnect by client.3939857 [2a02:4780:bad:f00d::2]:0/3199769431 after 1.070575
21
2016-11-25 12:13:51.565617 7f0f7b30a700  1 mds.0.141894 reconnect_done
22
2016-11-25 12:13:52.515982 7f0f7b30a700  1 mds.0.141894 handle_mds_map i am now mds.0.141894
23
2016-11-25 12:13:52.515985 7f0f7b30a700  1 mds.0.141894 handle_mds_map state change up:reconnect --> up:rejoin
24
2016-11-25 12:13:52.515994 7f0f7b30a700  1 mds.0.141894 rejoin_start
25
2016-11-25 12:13:52.518110 7f0f7b30a700  1 mds.0.141894 rejoin_joint_start
26
2016-11-25 12:13:52.919604 7f0f77201700  1 mds.0.141894 rejoin_done
27
2016-11-25 12:13:53.525810 7f0f7b30a700  1 mds.0.141894 handle_mds_map i am now mds.0.141894
28
2016-11-25 12:13:53.525814 7f0f7b30a700  1 mds.0.141894 handle_mds_map state change up:rejoin --> up:clientreplay
29
2016-11-25 12:13:53.525820 7f0f7b30a700  1 mds.0.141894 recovery_done -- successful recovery!
30
2016-11-25 12:13:53.526117 7f0f7b30a700  1 mds.0.141894 clientreplay_start
31
2016-11-25 12:13:53.571677 7f0f7b30a700  1 mds.0.141894 clientreplay_done
32
2016-11-25 12:13:54.532179 7f0f7b30a700  1 mds.0.141894 handle_mds_map i am now mds.0.141894
33
2016-11-25 12:13:54.532181 7f0f7b30a700  1 mds.0.141894 handle_mds_map state change up:clientreplay --> up:active
34
2016-11-25 12:13:54.532187 7f0f7b30a700  1 mds.0.141894 active_start
35
2016-11-25 12:13:55.262692 7f0f7b30a700  1 mds.0.141894 cluster recovered.
36
2016-11-25 12:14:23.503176 7f0f78a04700  0 log_channel(cluster) log [WRN] : 3 slow requests, 3 included below; oldest blocked for > 32.428118 secs
37
2016-11-25 12:14:23.503182 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.428118 seconds old, received at 2016-11-25 12:13:51.075008: client_request(client.3857990:15975668 setfilelockrule 2, type 2, owner 17585101599794939330, pid 34454, start 0, length 0, wait 1 #1000b58d15c 2016-11-24 08:02:39.174177 RETRY=3) currently acquired locks
38
2016-11-25 12:14:23.503186 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.428093 seconds old, received at 2016-11-25 12:13:51.075033: client_request(client.3857990:15985352 setfilelockrule 2, type 2, owner 17585101464735930818, pid 29283, start 0, length 0, wait 1 #1000b58d15c 2016-11-24 08:03:40.908105 RETRY=3) currently acquired locks
39
2016-11-25 12:14:23.503188 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.428085 seconds old, received at 2016-11-25 12:13:51.075041: client_request(client.3857990:15986859 setfilelockrule 2, type 2, owner 17585101588803107010, pid 37017, start 0, length 0, wait 1 #1000b58d15c 2016-11-24 08:04:11.015582 RETRY=3) currently acquired locks
40
2016-11-25 12:14:33.503363 7f0f78a04700  0 log_channel(cluster) log [WRN] : 87 slow requests, 5 included below; oldest blocked for > 42.428302 secs
41
2016-11-25 12:14:33.503369 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.490452 seconds old, received at 2016-11-25 12:14:01.012858: client_request(client.3935504:3845971 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.014120) currently failed to rdlock, waiting
42
2016-11-25 12:14:33.503372 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.426702 seconds old, received at 2016-11-25 12:14:01.076609: client_request(client.3935504:3846248 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.078119) currently failed to rdlock, waiting
43
2016-11-25 12:14:33.503374 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.421360 seconds old, received at 2016-11-25 12:14:01.081951: client_request(client.3935504:3846268 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.083119) currently failed to rdlock, waiting
44
2016-11-25 12:14:33.503376 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 31.729498 seconds old, received at 2016-11-25 12:14:01.773813: client_request(client.3935504:3847032 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.775109) currently failed to rdlock, waiting
45
2016-11-25 12:14:33.503378 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 32.452550 seconds old, received at 2016-11-25 12:14:01.050760: client_request(client.3857990:28058347 getattr pAsLsXsFs #1000b231346 2016-11-25 12:14:01.052928) currently failed to rdlock, waiting
46
2016-11-25 12:14:38.503466 7f0f78a04700  0 log_channel(cluster) log [WRN] : 101 slow requests, 5 included below; oldest blocked for > 47.428407 secs
47
2016-11-25 12:14:38.503472 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 37.437234 seconds old, received at 2016-11-25 12:14:01.066181: client_request(client.3857990:28058406 create #1000bafbd6c/sess_pusr5vjnf6l5hd8815t725fhe2 2016-11-25 12:14:01.067928) currently failed to wrlock, waiting
48
2016-11-25 12:14:38.503475 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 37.421429 seconds old, received at 2016-11-25 12:14:01.081986: client_request(client.3935504:3846269 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.083119) currently failed to rdlock, waiting
49
2016-11-25 12:14:38.503478 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 37.418315 seconds old, received at 2016-11-25 12:14:01.085100: client_request(client.3935504:3846289 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.086119) currently failed to rdlock, waiting
50
2016-11-25 12:14:38.503480 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 37.229396 seconds old, received at 2016-11-25 12:14:01.274019: client_request(client.3935504:3846755 getattr pAsLsXsFs #1000265e9ee 2016-11-25 12:14:01.275116) currently failed to rdlock, waiting
51
2016-11-25 12:14:38.503483 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 37.421412 seconds old, received at 2016-11-25 12:14:01.082003: client_request(client.3935504:3846270 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.083119) currently failed to rdlock, waiting
52
2016-11-25 12:14:43.503527 7f0f78a04700  0 log_channel(cluster) log [WRN] : 122 slow requests, 5 included below; oldest blocked for > 52.428464 secs
53
2016-11-25 12:14:43.503533 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 42.418326 seconds old, received at 2016-11-25 12:14:01.085146: client_request(client.3935504:3846290 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.086119) currently failed to rdlock, waiting
54
2016-11-25 12:14:43.503546 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 31.967243 seconds old, received at 2016-11-25 12:14:11.536229: client_request(client.3935504:3858159 create #1000bb76685/temp-write-test-1480076051 2016-11-25 12:14:11.536972) currently failed to wrlock, waiting
55
2016-11-25 12:14:43.503549 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 42.490285 seconds old, received at 2016-11-25 12:14:01.013187: client_request(client.3935504:3845996 getattr pAsLsXsFs #1000a874078 2016-11-25 12:14:01.014120) currently failed to rdlock, waiting
56
2016-11-25 12:14:43.503553 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 42.452557 seconds old, received at 2016-11-25 12:14:01.050915: client_request(client.3857990:28058348 getattr pAsLsXsFs #1000b231346 2016-11-25 12:14:01.052928) currently failed to rdlock, waiting
57
2016-11-25 12:14:43.503556 7f0f78a04700  0 log_channel(cluster) log [WRN] : slow request 42.418393 seconds old, received at 2016-11-25 12:14:01.085079: client_request(client.3857990:28058472 unlink #1000968991e/cookie1.txt 2016-11-25 12:14:01.086928) currently failed to wrlock, waiting
58
2016-11-25 12:14:48.503592 7f0f78a04700  0 log_channel(cluster) log [WRN] : 136 slow requests, 5 included below; oldest blocked for > 57.428524 secs