Project

General

Profile

STALE_on_remount.mds.log

Brian Chrisman, 06/20/2011 03:08 PM

Download (51.1 KB)

 
1
2011-06-20 21:50:43.993622 7f1dd1de2720 ceph version 0.24.2-2395-g8385500.commit: 83855002553932cb25da648d7728e5cf3cee364a. process: cmds. pid: 28486
2
2011-06-20 21:50:43.996412 7f1dcefc3710 mds-1.0 ms_handle_connect on 192.168.98.110:6789/0
3
2011-06-20 21:51:02.139721 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.111:6810/29037
4
2011-06-20 21:51:02.147065 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.109:6810/27180
5
2011-06-20 21:51:02.170840 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.110:6801/2696
6
2011-06-20 21:51:02.257169 7f1dcefc3710 mds0.cache creating system inode with ino:100
7
2011-06-20 21:51:02.257484 7f1dcefc3710 mds0.cache creating system inode with ino:1
8
2011-06-20 21:51:02.258499 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.111:6804/28629
9
2011-06-20 21:51:02.258562 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.110:6804/2829
10
2011-06-20 21:51:02.273536 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.111:6807/28881
11
2011-06-20 21:51:02.274391 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.109:6807/27083
12
2011-06-20 21:51:02.274496 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.109:6804/26947
13
2011-06-20 21:51:02.914406 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.109:6801/26856
14
2011-06-20 21:51:03.181419 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.110:6807/2968
15
2011-06-20 21:51:03.319023 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.111:6801/28549
16
2011-06-20 21:51:03.610523 7f1dcefc3710 mds0.6 ms_handle_connect on 192.168.98.110:6810/3127
17
2011-06-20 21:51:04.637629 7f1d8f5fc710 -- 192.168.98.111:6800/28486 >> 192.168.98.111:0/2049295103 pipe(0x34b70c0 sd=18 pgs=0 cs=0 l=0).accept peer addr is really 192.168.98.111:0/2049295103 (socket is 192.168.98.111:57675/0)
18
2011-06-20 21:51:04.638311 7f1dcefc3710 log [DBG] : reconnect by client4100 192.168.98.111:0/2049295103 after 0.006388
19
2011-06-20 21:51:04.638436 7f1dcefc3710 mds0.server missing 1000000000e #1/fsscale0/lib/state.db (mine), will load later
20
2011-06-20 21:51:04.638468 7f1dcefc3710 mds0.server missing 10000000045 #1/fsscale0/lock (mine), will load later
21
2011-06-20 21:51:04.638487 7f1dcefc3710 mds0.server missing 10000000085 #1/fsscale0/lib/scconfigd/repos/storage.db (mine), will load later
22
2011-06-20 21:51:04.638501 7f1dcefc3710 mds0.server missing 10000000128 #1/fsscale0/lock/state_db (mine), will load later
23
2011-06-20 21:51:04.638515 7f1dcefc3710 mds0.server missing 10000001096 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old (mine), will load later
24
2011-06-20 21:51:04.638529 7f1dcefc3710 mds0.server missing 100000010da #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster (mine), will load later
25
2011-06-20 21:51:04.638551 7f1dcefc3710 mds0.server missing 100000010e7 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/cluster (mine), will load later
26
2011-06-20 21:51:04.638584 7f1dcefc3710 mds0.server missing 10000001101 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/cluster/opt/scale (mine), will load later
27
2011-06-20 21:51:04.638612 7f1dcefc3710 mds0.server missing 10000001128 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/cluster/etc (mine), will load later
28
2011-06-20 21:51:04.638634 7f1dcefc3710 mds0.server missing 10000001144 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage (mine), will load later
29
2011-06-20 21:51:04.638655 7f1dcefc3710 mds0.server missing 10000001151 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/opt (mine), will load later
30
2011-06-20 21:51:04.638681 7f1dcefc3710 mds0.server missing 1000000116b #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/opt/scale/etc (mine), will load later
31
2011-06-20 21:51:04.638722 7f1dcefc3710 mds0.server missing 100000011b7 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/fs0/iscsi (mine), will load later
32
2011-06-20 21:51:04.638760 7f1dcefc3710 mds0.server missing 100000011cc #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/etc (mine), will load later
33
2011-06-20 21:51:04.638784 7f1dcefc3710 mds0.server missing 100000011d9 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/etc/cron.d (mine), will load later
34
2011-06-20 21:51:04.639045 7f1dcefc3710 mds0.server missing 10000001219 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/cluster/storage/etc/ctdb (mine), will load later
35
2011-06-20 21:51:04.639070 7f1dcefc3710 mds0.server missing 10000001236 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/db (mine), will load later
36
2011-06-20 21:51:04.639125 7f1dcefc3710 mds0.server missing 100000012a3 #1/fsscale0/lib/scconfigd/repos/wc.1308596713.old/192.168.98.111/storage (mine), will load later
37
2011-06-20 21:51:04.639160 7f1dcefc3710 mds0.server missing 10000001305 #1/fsscale0/lib/scconfigd/repos/wc (mine), will load later
38
2011-06-20 21:51:04.639175 7f1dcefc3710 mds0.server missing 10000001f54 #1/fsscale0/lib/scconfigd/repos/cluster.db (mine), will load later
39
2011-06-20 21:51:04.639524 7f1d8f4fb710 -- 192.168.98.111:6800/28486 >> 192.168.98.109:0/3628510399 pipe(0x34af4b0 sd=19 pgs=0 cs=0 l=0).accept peer addr is really 192.168.98.109:0/3628510399 (socket is 192.168.98.109:52980/0)
40
2011-06-20 21:51:04.639924 7f1dcefc3710 log [DBG] : reconnect by client4107 192.168.98.109:0/3628510399 after 0.008086
41
2011-06-20 21:51:04.639973 7f1dcefc3710 mds0.server missing 10000000007 #1/fsscale0/lib/cluster_stor.db (mine), will load later
42
2011-06-20 21:51:04.640005 7f1dcefc3710 mds0.server missing 1000000002b #1/fsscale0/lib/scconfigd/svn/config/db/fs-type (mine), will load later
43
2011-06-20 21:51:04.640044 7f1dcefc3710 mds0.server missing 10000000030 #1/fsscale0/lib/scconfigd/svn/config/db/transactions (mine), will load later
44
2011-06-20 21:51:04.640063 7f1dcefc3710 mds0.server missing 10000000033 #1/fsscale0/lib/scconfigd/svn/config/db/write-lock (mine), will load later
45
2011-06-20 21:51:04.640077 7f1dcefc3710 mds0.server missing 10000000034 #1/fsscale0/lib/scconfigd/svn/config/db/uuid (mine), will load later
46
2011-06-20 21:51:04.640094 7f1dcefc3710 mds0.server missing 10000000035 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/0 (mine), will load later
47
2011-06-20 21:51:04.640109 7f1dcefc3710 mds0.server missing 10000000037 #1/fsscale0/lib/scconfigd/svn/config/db/fsfs.conf (mine), will load later
48
2011-06-20 21:51:04.640123 7f1dcefc3710 mds0.server missing 10000000038 #1/fsscale0/lib/scconfigd/svn/config/db/rep-cache.db (mine), will load later
49
2011-06-20 21:51:04.640137 7f1dcefc3710 mds0.server missing 1000000003c #1/fsscale0/lib/scconfigd/svn/config/db/min-unpacked-rev (mine), will load later
50
2011-06-20 21:51:04.640152 7f1dcefc3710 mds0.server missing 1000000003e #1/fsscale0/lib/scconfigd/svn/config/db/txn-current-lock (mine), will load later
51
2011-06-20 21:51:04.640179 7f1dcefc3710 mds0.server missing 1000000003f #1/fsscale0/lib/scconfigd/svn/config/db/format (mine), will load later
52
2011-06-20 21:51:04.640192 7f1dcefc3710 mds0.server missing 10000000045 #1/fsscale0/lock (mine), will load later
53
2011-06-20 21:51:04.640209 7f1dcefc3710 mds0.server missing 10000000085 #1/fsscale0/lib/scconfigd/repos/storage.db (mine), will load later
54
2011-06-20 21:51:04.640222 7f1dcefc3710 mds0.server missing 1000000012a #1/fsscale0/lib/scconfigd/repos/wc.1308596282.old (mine), will load later
55
2011-06-20 21:51:04.640236 7f1dcefc3710 mds0.server missing 1000000013b #1/fsscale0/lib/scconfigd/repos/wc.1308596282.old/cluster (mine), will load later
56
2011-06-20 21:51:04.640290 7f1dcefc3710 mds0.server missing 10000000147 #1/fsscale0/lib/scconfigd/repos/wc.1308596282.old/db (mine), will load later
57
2011-06-20 21:51:04.640304 7f1dcefc3710 mds0.server missing 1000000014e #1/fsscale0/lib/scconfigd/repos/wc.1308596282.old/cluster/storage (mine), will load later
58
2011-06-20 21:51:04.640322 7f1dcefc3710 mds0.server missing 1000000016a #1/fsscale0/lib/scconfigd/repos/wc.1308596282.old/dump (mine), will load later
59
2011-06-20 21:51:04.640361 7f1dcefc3710 mds0.server missing 1000000035b #1/fsscale0/lib/scconfigd/svn/config/db/revprops/0/1 (mine), will load later
60
2011-06-20 21:51:04.640376 7f1dcefc3710 mds0.server missing 100000003c6 #1/fsscale0/lock/ads_join (mine), will load later
61
2011-06-20 21:51:04.640392 7f1dcefc3710 mds0.server missing 10000000623 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/4 (mine), will load later
62
2011-06-20 21:51:04.640431 7f1dcefc3710 mds0.server missing 10000000642 #1/fsscale0/lib/scconfigd/svn/config/db/revprops/0/4 (mine), will load later
63
2011-06-20 21:51:04.640451 7f1dcefc3710 mds0.server missing 10000000a01 #1/fsscale0/lib/scconfigd/svn/config/db/revprops/0/2 (mine), will load later
64
2011-06-20 21:51:04.640467 7f1dcefc3710 mds0.server missing 10000000c71 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/6 (mine), will load later
65
2011-06-20 21:51:04.640483 7f1dcefc3710 mds0.server missing 10000000c7e #1/fsscale0/lib/scconfigd/svn/config/db/revprops/0/6 (mine), will load later
66
2011-06-20 21:51:04.640510 7f1dcefc3710 mds0.server missing 10000001085 #1/fsscale0/lib/scconfigd/svn/config/db/revprops/0/3 (mine), will load later
67
2011-06-20 21:51:04.640527 7f1dcefc3710 mds0.server missing 100000012f5 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/8 (mine), will load later
68
2011-06-20 21:51:04.640545 7f1dcefc3710 mds0.server missing 10000001305 #1/fsscale0/lib/scconfigd/repos/wc (mine), will load later
69
2011-06-20 21:51:04.640559 7f1dcefc3710 mds0.server missing 10000001313 #1/fsscale0/lib/scconfigd/repos/wc/192.168.98.109 (mine), will load later
70
2011-06-20 21:51:04.640573 7f1dcefc3710 mds0.server missing 10000001574 #1/fsscale0/lib/scconfigd/svn/config/db/txn-current (mine), will load later
71
2011-06-20 21:51:04.640589 7f1dcefc3710 mds0.server missing 10000001577 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/5 (mine), will load later
72
2011-06-20 21:51:04.640613 7f1dcefc3710 mds0.server missing 10000001f45 #1/fsscale0/lib/scconfigd/svn/config/db/revs/0/9 (mine), will load later
73
2011-06-20 21:51:04.640631 7f1dcefc3710 mds0.server missing 10000001f53 #1/fsscale0/lib/scconfigd/svn/config/db/current (mine), will load later
74
2011-06-20 21:51:04.640652 7f1dcefc3710 mds0.server missing 10000001f54 #1/fsscale0/lib/scconfigd/repos/cluster.db (mine), will load later
75
2011-06-20 21:52:21.566500 7f1d8f1f8710 -- 192.168.98.111:6800/28486 >> 192.168.98.110:0/1893654265 pipe(0x34bb7d0 sd=20 pgs=0 cs=0 l=0).accept peer addr is really 192.168.98.110:0/1893654265 (socket is 192.168.98.110:45937/0)
76
2011-06-20 21:55:19.010924 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
77
2011-06-20 21:55:19.010985 7f1dcdec0710 mds0.cache trim_client_leases
78
2011-06-20 21:55:19.012375 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53296, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
79
2011-06-20 21:55:19.012410 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
80
2011-06-20 21:55:19.012492 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
81
2011-06-20 21:55:19.012551 7f1dcdec0710 mds0.locker scatter_tick
82
2011-06-20 21:55:19.012590 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
83
2011-06-20 21:55:19.012629 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399
84
2011-06-20 21:55:19.012662 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399 and sufficiently new (2011-06-20 21:55:01.729701)
85
2011-06-20 21:55:19.012692 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:19.012687
86
2011-06-20 21:55:19.012776 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
87
2011-06-20 21:55:19.012849 7f1dcdec0710 mds0.bal mds0 epoch 25 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
88
2011-06-20 21:55:19.012869 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
89
2011-06-20 21:55:20.763554 7f1dcdec0710 mds0.6 beacon_send up:active seq 73 (currently up:active)
90
2011-06-20 21:55:20.764383 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 73 rtt 0.000761
91
2011-06-20 21:55:21.730152 7f1dcefc3710 mds0.server get_session have 0x7f1dc40078e0 client4107 192.168.98.109:0/3628510399 state open
92
2011-06-20 21:55:21.730196 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 540) from client4107
93
2011-06-20 21:55:22.856459 7f1dcefc3710 mds0.server get_session have 0x7f1dc4007b10 client4100 192.168.98.111:0/2049295103 state open
94
2011-06-20 21:55:22.856500 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 518) from client4100
95
2011-06-20 21:55:24.010987 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
96
2011-06-20 21:55:24.011044 7f1dcdec0710 mds0.cache trim_client_leases
97
2011-06-20 21:55:24.052467 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53300, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
98
2011-06-20 21:55:24.052500 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
99
2011-06-20 21:55:24.052571 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
100
2011-06-20 21:55:24.052598 7f1dcdec0710 mds0.locker scatter_tick
101
2011-06-20 21:55:24.052612 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
102
2011-06-20 21:55:24.052628 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
103
2011-06-20 21:55:24.052641 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:16.620004)
104
2011-06-20 21:55:24.052650 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:24.052647
105
2011-06-20 21:55:24.052660 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
106
2011-06-20 21:55:24.763761 7f1dcdec0710 mds0.6 beacon_send up:active seq 74 (currently up:active)
107
2011-06-20 21:55:24.764524 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 74 rtt 0.000707
108
2011-06-20 21:55:28.763908 7f1dcdec0710 mds0.6 beacon_send up:active seq 75 (currently up:active)
109
2011-06-20 21:55:28.764681 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 75 rtt 0.000713
110
2011-06-20 21:55:29.011053 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
111
2011-06-20 21:55:29.011179 7f1dcdec0710 mds0.cache trim_client_leases
112
2011-06-20 21:55:29.074722 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
113
2011-06-20 21:55:29.074758 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
114
2011-06-20 21:55:29.074836 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
115
2011-06-20 21:55:29.074864 7f1dcdec0710 mds0.locker scatter_tick
116
2011-06-20 21:55:29.074878 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
117
2011-06-20 21:55:29.074893 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
118
2011-06-20 21:55:29.074905 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:16.620004)
119
2011-06-20 21:55:29.074914 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:29.074911
120
2011-06-20 21:55:29.074976 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
121
2011-06-20 21:55:29.075019 7f1dcdec0710 mds0.bal mds0 epoch 26 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.03>
122
2011-06-20 21:55:29.075036 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
123
2011-06-20 21:55:32.764058 7f1dcdec0710 mds0.6 beacon_send up:active seq 76 (currently up:active)
124
2011-06-20 21:55:32.764885 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 76 rtt 0.000769
125
2011-06-20 21:55:34.011125 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
126
2011-06-20 21:55:34.011262 7f1dcdec0710 mds0.cache trim_client_leases
127
2011-06-20 21:55:34.072409 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
128
2011-06-20 21:55:34.072440 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
129
2011-06-20 21:55:34.072551 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
130
2011-06-20 21:55:34.072582 7f1dcdec0710 mds0.locker scatter_tick
131
2011-06-20 21:55:34.072595 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
132
2011-06-20 21:55:34.072610 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
133
2011-06-20 21:55:34.072622 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:16.620004)
134
2011-06-20 21:55:34.072633 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:34.072629
135
2011-06-20 21:55:34.072641 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
136
2011-06-20 21:55:36.631027 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
137
2011-06-20 21:55:36.631072 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 10) from client5215
138
2011-06-20 21:55:36.764202 7f1dcdec0710 mds0.6 beacon_send up:active seq 77 (currently up:active)
139
2011-06-20 21:55:36.764987 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 77 rtt 0.000731
140
2011-06-20 21:55:39.011195 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
141
2011-06-20 21:55:39.082872 7f1dcdec0710 mds0.cache trim_client_leases
142
2011-06-20 21:55:39.084313 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
143
2011-06-20 21:55:39.084349 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
144
2011-06-20 21:55:39.084429 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
145
2011-06-20 21:55:39.084458 7f1dcdec0710 mds0.locker scatter_tick
146
2011-06-20 21:55:39.084471 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
147
2011-06-20 21:55:39.084488 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399
148
2011-06-20 21:55:39.084501 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399 and sufficiently new (2011-06-20 21:55:21.730257)
149
2011-06-20 21:55:39.084511 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:39.084508
150
2011-06-20 21:55:39.084577 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
151
2011-06-20 21:55:39.084620 7f1dcdec0710 mds0.bal mds0 epoch 27 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
152
2011-06-20 21:55:39.084637 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
153
2011-06-20 21:55:40.764349 7f1dcdec0710 mds0.6 beacon_send up:active seq 78 (currently up:active)
154
2011-06-20 21:55:40.765104 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 78 rtt 0.000697
155
2011-06-20 21:55:41.555052 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:2 lookuphash  #1/830207683)
156
2011-06-20 21:55:41.555110 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
157
2011-06-20 21:55:41.555119 7f1dcefc3710 mds0.server  oldest_client_tid=2
158
2011-06-20 21:55:41.555143 7f1dcefc3710 mds0.cache request_start request(client5215:2 cr=0x7f1d9c001220)
159
2011-06-20 21:55:41.555153 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:2 lookuphash  #1/830207683)
160
2011-06-20 21:55:41.555239 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
161
2011-06-20 21:55:41.555266 7f1dcefc3710 mds0.server  fg is *
162
2011-06-20 21:55:41.555291 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
163
2011-06-20 21:55:41.555370 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:2 lookuphash  #1/830207683)
164
2011-06-20 21:55:41.555387 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
165
2011-06-20 21:55:41.555410 7f1dcefc3710 mds0.server lat 0.000383
166
2011-06-20 21:55:41.555425 7f1dcefc3710 mds0.cache request_finish request(client5215:2 cr=0x7f1d9c001220)
167
2011-06-20 21:55:41.555433 7f1dcefc3710 mds0.cache request_cleanup request(client5215:2 cr=0x7f1d9c001220)
168
2011-06-20 21:55:41.665710 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:3 lookuphash  #1/830207683)
169
2011-06-20 21:55:41.665753 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
170
2011-06-20 21:55:41.665762 7f1dcefc3710 mds0.server  oldest_client_tid=3
171
2011-06-20 21:55:41.665774 7f1dcefc3710 mds0.cache request_start request(client5215:3 cr=0x7f1d9c001220)
172
2011-06-20 21:55:41.665782 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:3 lookuphash  #1/830207683)
173
2011-06-20 21:55:41.665848 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
174
2011-06-20 21:55:41.665861 7f1dcefc3710 mds0.server  fg is *
175
2011-06-20 21:55:41.665881 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
176
2011-06-20 21:55:41.665898 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:3 lookuphash  #1/830207683)
177
2011-06-20 21:55:41.665907 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
178
2011-06-20 21:55:41.665924 7f1dcefc3710 mds0.server lat 0.000238
179
2011-06-20 21:55:41.665941 7f1dcefc3710 mds0.cache request_finish request(client5215:3 cr=0x7f1d9c001220)
180
2011-06-20 21:55:41.665952 7f1dcefc3710 mds0.cache request_cleanup request(client5215:3 cr=0x7f1d9c001220)
181
2011-06-20 21:55:41.730503 7f1dcefc3710 mds0.server get_session have 0x7f1dc40078e0 client4107 192.168.98.109:0/3628510399 state open
182
2011-06-20 21:55:41.730537 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 541) from client4107
183
2011-06-20 21:55:41.769348 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:4 lookuphash  #1/830207683)
184
2011-06-20 21:55:41.769378 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
185
2011-06-20 21:55:41.769386 7f1dcefc3710 mds0.server  oldest_client_tid=4
186
2011-06-20 21:55:41.769398 7f1dcefc3710 mds0.cache request_start request(client5215:4 cr=0x7f1d9c001220)
187
2011-06-20 21:55:41.769409 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:4 lookuphash  #1/830207683)
188
2011-06-20 21:55:41.769462 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
189
2011-06-20 21:55:41.769475 7f1dcefc3710 mds0.server  fg is *
190
2011-06-20 21:55:41.769496 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
191
2011-06-20 21:55:41.769512 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:4 lookuphash  #1/830207683)
192
2011-06-20 21:55:41.769552 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
193
2011-06-20 21:55:41.769574 7f1dcefc3710 mds0.server lat 0.000241
194
2011-06-20 21:55:41.769594 7f1dcefc3710 mds0.cache request_finish request(client5215:4 cr=0x7f1d9c001220)
195
2011-06-20 21:55:41.769603 7f1dcefc3710 mds0.cache request_cleanup request(client5215:4 cr=0x7f1d9c001220)
196
2011-06-20 21:55:41.870092 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:5 lookuphash  #1/830207683)
197
2011-06-20 21:55:41.870134 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
198
2011-06-20 21:55:41.870142 7f1dcefc3710 mds0.server  oldest_client_tid=5
199
2011-06-20 21:55:41.870155 7f1dcefc3710 mds0.cache request_start request(client5215:5 cr=0x7f1d9c001220)
200
2011-06-20 21:55:41.870164 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:5 lookuphash  #1/830207683)
201
2011-06-20 21:55:41.870219 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
202
2011-06-20 21:55:41.870234 7f1dcefc3710 mds0.server  fg is *
203
2011-06-20 21:55:41.870264 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
204
2011-06-20 21:55:41.870283 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:5 lookuphash  #1/830207683)
205
2011-06-20 21:55:41.870292 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
206
2011-06-20 21:55:41.870310 7f1dcefc3710 mds0.server lat 0.000242
207
2011-06-20 21:55:41.870324 7f1dcefc3710 mds0.cache request_finish request(client5215:5 cr=0x7f1d9c001220)
208
2011-06-20 21:55:41.870335 7f1dcefc3710 mds0.cache request_cleanup request(client5215:5 cr=0x7f1d9c001220)
209
2011-06-20 21:55:41.987231 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:6 lookuphash  #1/830207683)
210
2011-06-20 21:55:41.987271 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
211
2011-06-20 21:55:41.987280 7f1dcefc3710 mds0.server  oldest_client_tid=6
212
2011-06-20 21:55:41.987291 7f1dcefc3710 mds0.cache request_start request(client5215:6 cr=0x7f1d9c001220)
213
2011-06-20 21:55:41.987300 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:6 lookuphash  #1/830207683)
214
2011-06-20 21:55:41.987350 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
215
2011-06-20 21:55:41.987366 7f1dcefc3710 mds0.server  fg is *
216
2011-06-20 21:55:41.987385 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
217
2011-06-20 21:55:41.987400 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:6 lookuphash  #1/830207683)
218
2011-06-20 21:55:41.987409 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
219
2011-06-20 21:55:41.987428 7f1dcefc3710 mds0.server lat 0.000214
220
2011-06-20 21:55:41.987447 7f1dcefc3710 mds0.cache request_finish request(client5215:6 cr=0x7f1d9c001220)
221
2011-06-20 21:55:41.987456 7f1dcefc3710 mds0.cache request_cleanup request(client5215:6 cr=0x7f1d9c001220)
222
2011-06-20 21:55:42.088044 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:7 lookuphash  #1/830207683)
223
2011-06-20 21:55:42.088097 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
224
2011-06-20 21:55:42.088106 7f1dcefc3710 mds0.server  oldest_client_tid=7
225
2011-06-20 21:55:42.088120 7f1dcefc3710 mds0.cache request_start request(client5215:7 cr=0x7f1d9c001220)
226
2011-06-20 21:55:42.088130 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:7 lookuphash  #1/830207683)
227
2011-06-20 21:55:42.088190 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
228
2011-06-20 21:55:42.088207 7f1dcefc3710 mds0.server  fg is *
229
2011-06-20 21:55:42.088227 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
230
2011-06-20 21:55:42.088259 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:7 lookuphash  #1/830207683)
231
2011-06-20 21:55:42.088279 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
232
2011-06-20 21:55:42.088300 7f1dcefc3710 mds0.server lat 0.000278
233
2011-06-20 21:55:42.088316 7f1dcefc3710 mds0.cache request_finish request(client5215:7 cr=0x7f1d9c001220)
234
2011-06-20 21:55:42.088327 7f1dcefc3710 mds0.cache request_cleanup request(client5215:7 cr=0x7f1d9c001220)
235
2011-06-20 21:55:42.158675 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:8 lookuphash  #1/830207683)
236
2011-06-20 21:55:42.158715 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
237
2011-06-20 21:55:42.158723 7f1dcefc3710 mds0.server  oldest_client_tid=8
238
2011-06-20 21:55:42.158735 7f1dcefc3710 mds0.cache request_start request(client5215:8 cr=0x7f1d9c001220)
239
2011-06-20 21:55:42.158744 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:8 lookuphash  #1/830207683)
240
2011-06-20 21:55:42.158800 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
241
2011-06-20 21:55:42.158815 7f1dcefc3710 mds0.server  fg is *
242
2011-06-20 21:55:42.158835 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
243
2011-06-20 21:55:42.158853 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:8 lookuphash  #1/830207683)
244
2011-06-20 21:55:42.158863 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
245
2011-06-20 21:55:42.158882 7f1dcefc3710 mds0.server lat 0.00023
246
2011-06-20 21:55:42.158941 7f1dcefc3710 mds0.cache request_finish request(client5215:8 cr=0x7f1d9c001220)
247
2011-06-20 21:55:42.158958 7f1dcefc3710 mds0.cache request_cleanup request(client5215:8 cr=0x7f1d9c001220)
248
2011-06-20 21:55:42.160108 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:9 lookuphash  #1/830207683)
249
2011-06-20 21:55:42.160133 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
250
2011-06-20 21:55:42.160140 7f1dcefc3710 mds0.server  oldest_client_tid=9
251
2011-06-20 21:55:42.160155 7f1dcefc3710 mds0.cache request_start request(client5215:9 cr=0x7f1d9c001220)
252
2011-06-20 21:55:42.160197 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:9 lookuphash  #1/830207683)
253
2011-06-20 21:55:42.160256 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
254
2011-06-20 21:55:42.160273 7f1dcefc3710 mds0.server  fg is *
255
2011-06-20 21:55:42.160293 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
256
2011-06-20 21:55:42.160308 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:9 lookuphash  #1/830207683)
257
2011-06-20 21:55:42.160319 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
258
2011-06-20 21:55:42.160335 7f1dcefc3710 mds0.server lat 0.000249
259
2011-06-20 21:55:42.160390 7f1dcefc3710 mds0.cache request_finish request(client5215:9 cr=0x7f1d9c001220)
260
2011-06-20 21:55:42.160406 7f1dcefc3710 mds0.cache request_cleanup request(client5215:9 cr=0x7f1d9c001220)
261
2011-06-20 21:55:42.161056 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:10 lookuphash  #1/830207683)
262
2011-06-20 21:55:42.161080 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
263
2011-06-20 21:55:42.161088 7f1dcefc3710 mds0.server  oldest_client_tid=10
264
2011-06-20 21:55:42.161098 7f1dcefc3710 mds0.cache request_start request(client5215:10 cr=0x7f1d9c001220)
265
2011-06-20 21:55:42.161107 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:10 lookuphash  #1/830207683)
266
2011-06-20 21:55:42.161150 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
267
2011-06-20 21:55:42.161166 7f1dcefc3710 mds0.server  fg is *
268
2011-06-20 21:55:42.161185 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
269
2011-06-20 21:55:42.161202 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:10 lookuphash  #1/830207683)
270
2011-06-20 21:55:42.161213 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
271
2011-06-20 21:55:42.161228 7f1dcefc3710 mds0.server lat 0.00019
272
2011-06-20 21:55:42.161300 7f1dcefc3710 mds0.cache request_finish request(client5215:10 cr=0x7f1d9c001220)
273
2011-06-20 21:55:42.161317 7f1dcefc3710 mds0.cache request_cleanup request(client5215:10 cr=0x7f1d9c001220)
274
2011-06-20 21:55:42.162167 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:11 lookuphash  #1/830207683)
275
2011-06-20 21:55:42.162190 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
276
2011-06-20 21:55:42.162198 7f1dcefc3710 mds0.server  oldest_client_tid=11
277
2011-06-20 21:55:42.162207 7f1dcefc3710 mds0.cache request_start request(client5215:11 cr=0x7f1d9c001220)
278
2011-06-20 21:55:42.162219 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:11 lookuphash  #1/830207683)
279
2011-06-20 21:55:42.162271 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
280
2011-06-20 21:55:42.162307 7f1dcefc3710 mds0.server  fg is *
281
2011-06-20 21:55:42.162326 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
282
2011-06-20 21:55:42.162341 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:11 lookuphash  #1/830207683)
283
2011-06-20 21:55:42.162349 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
284
2011-06-20 21:55:42.162364 7f1dcefc3710 mds0.server lat 0.000218
285
2011-06-20 21:55:42.162383 7f1dcefc3710 mds0.cache request_finish request(client5215:11 cr=0x7f1d9c001220)
286
2011-06-20 21:55:42.162391 7f1dcefc3710 mds0.cache request_cleanup request(client5215:11 cr=0x7f1d9c001220)
287
2011-06-20 21:55:42.163257 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:12 lookuphash  #1/830207683)
288
2011-06-20 21:55:42.163287 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
289
2011-06-20 21:55:42.163295 7f1dcefc3710 mds0.server  oldest_client_tid=12
290
2011-06-20 21:55:42.163306 7f1dcefc3710 mds0.cache request_start request(client5215:12 cr=0x7f1d9c001220)
291
2011-06-20 21:55:42.163315 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:12 lookuphash  #1/830207683)
292
2011-06-20 21:55:42.163361 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
293
2011-06-20 21:55:42.163376 7f1dcefc3710 mds0.server  fg is *
294
2011-06-20 21:55:42.163395 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
295
2011-06-20 21:55:42.163412 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:12 lookuphash  #1/830207683)
296
2011-06-20 21:55:42.163422 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
297
2011-06-20 21:55:42.163436 7f1dcefc3710 mds0.server lat 0.000205
298
2011-06-20 21:55:42.163488 7f1dcefc3710 mds0.cache request_finish request(client5215:12 cr=0x7f1d9c001220)
299
2011-06-20 21:55:42.163505 7f1dcefc3710 mds0.cache request_cleanup request(client5215:12 cr=0x7f1d9c001220)
300
2011-06-20 21:55:42.164192 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:13 lookuphash  #1/830207683)
301
2011-06-20 21:55:42.164214 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
302
2011-06-20 21:55:42.164222 7f1dcefc3710 mds0.server  oldest_client_tid=13
303
2011-06-20 21:55:42.164232 7f1dcefc3710 mds0.cache request_start request(client5215:13 cr=0x7f1d9c001220)
304
2011-06-20 21:55:42.164251 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:13 lookuphash  #1/830207683)
305
2011-06-20 21:55:42.164297 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
306
2011-06-20 21:55:42.164309 7f1dcefc3710 mds0.server  fg is *
307
2011-06-20 21:55:42.164327 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
308
2011-06-20 21:55:42.164367 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:13 lookuphash  #1/830207683)
309
2011-06-20 21:55:42.164376 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
310
2011-06-20 21:55:42.164391 7f1dcefc3710 mds0.server lat 0.000212
311
2011-06-20 21:55:42.164409 7f1dcefc3710 mds0.cache request_finish request(client5215:13 cr=0x7f1d9c001220)
312
2011-06-20 21:55:42.164418 7f1dcefc3710 mds0.cache request_cleanup request(client5215:13 cr=0x7f1d9c001220)
313
2011-06-20 21:55:42.165145 7f1dcefc3710 mds0.server handle_client_request client_request(client5215:14 lookuphash  #1/830207683)
314
2011-06-20 21:55:42.165172 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
315
2011-06-20 21:55:42.165179 7f1dcefc3710 mds0.server  oldest_client_tid=14
316
2011-06-20 21:55:42.165189 7f1dcefc3710 mds0.cache request_start request(client5215:14 cr=0x7f1d9c001220)
317
2011-06-20 21:55:42.165201 7f1dcefc3710 mds0.server dispatch_client_request client_request(client5215:14 lookuphash  #1/830207683)
318
2011-06-20 21:55:42.165254 7f1dcefc3710 mds0.server  have diri [inode 1 [...2,head] / auth v1519 snaprealm=0x7f1dc40054f0 f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148)/n(v0 1=0+1) (inest lock) (iversion lock) caps={4100=pAsLsXsFs/p@3,4107=pAsLsXsFs/p@3,5215=pAsLsXsFs/-@1} | pastsnapparent dirfrag caps dirty 0x7f1dc4008800]
319
2011-06-20 21:55:42.165270 7f1dcefc3710 mds0.server  fg is *
320
2011-06-20 21:55:42.165289 7f1dcefc3710 mds0.server  have dir [dir 1 / [2,head] auth v=4482 cv=4400/4400 dir_auth=0 state=1610612738|complete f(v4 m2011-06-20 20:13:50.314420 4=0+4) n(v428 rc2011-06-20 21:15:11.104916 b400279383 9414=8266+1148) hs=4+0,ss=0+0 dirty=1 | child subtree dirty 0x2690190]
321
2011-06-20 21:55:42.165305 7f1dcefc3710 mds0.server reply_request -116 (Stale NFS file handle) client_request(client5215:14 lookuphash  #1/830207683)
322
2011-06-20 21:55:42.165315 7f1dcefc3710 mds0.server apply_allocated_inos 0 / [] / 0
323
2011-06-20 21:55:42.165331 7f1dcefc3710 mds0.server lat 0.000203
324
2011-06-20 21:55:42.165384 7f1dcefc3710 mds0.cache request_finish request(client5215:14 cr=0x7f1d9c001220)
325
2011-06-20 21:55:42.165401 7f1dcefc3710 mds0.cache request_cleanup request(client5215:14 cr=0x7f1d9c001220)
326
2011-06-20 21:55:42.856374 7f1dcefc3710 mds0.server get_session have 0x7f1dc4007b10 client4100 192.168.98.111:0/2049295103 state open
327
2011-06-20 21:55:42.856416 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 519) from client4100
328
2011-06-20 21:55:44.011269 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
329
2011-06-20 21:55:44.069500 7f1dcdec0710 mds0.cache trim_client_leases
330
2011-06-20 21:55:44.070960 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
331
2011-06-20 21:55:44.070993 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
332
2011-06-20 21:55:44.071066 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
333
2011-06-20 21:55:44.071095 7f1dcdec0710 mds0.locker scatter_tick
334
2011-06-20 21:55:44.071109 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
335
2011-06-20 21:55:44.071124 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
336
2011-06-20 21:55:44.071139 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:36.631082)
337
2011-06-20 21:55:44.071152 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:44.071148
338
2011-06-20 21:55:44.071160 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
339
2011-06-20 21:55:44.764497 7f1dcdec0710 mds0.6 beacon_send up:active seq 79 (currently up:active)
340
2011-06-20 21:55:44.765271 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 79 rtt 0.000718
341
2011-06-20 21:55:48.764647 7f1dcdec0710 mds0.6 beacon_send up:active seq 80 (currently up:active)
342
2011-06-20 21:55:48.765460 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 80 rtt 0.000715
343
2011-06-20 21:55:49.011338 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
344
2011-06-20 21:55:49.064069 7f1dcdec0710 mds0.cache trim_client_leases
345
2011-06-20 21:55:49.065494 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
346
2011-06-20 21:55:49.065526 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
347
2011-06-20 21:55:49.065598 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
348
2011-06-20 21:55:49.065624 7f1dcdec0710 mds0.locker scatter_tick
349
2011-06-20 21:55:49.065638 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
350
2011-06-20 21:55:49.065653 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
351
2011-06-20 21:55:49.065666 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:36.631082)
352
2011-06-20 21:55:49.065679 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:49.065675
353
2011-06-20 21:55:49.065724 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
354
2011-06-20 21:55:49.065766 7f1dcdec0710 mds0.bal mds0 epoch 28 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
355
2011-06-20 21:55:49.065797 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
356
2011-06-20 21:55:52.764837 7f1dcdec0710 mds0.6 beacon_send up:active seq 81 (currently up:active)
357
2011-06-20 21:55:52.765615 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 81 rtt 0.000720
358
2011-06-20 21:55:54.011410 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
359
2011-06-20 21:55:54.011543 7f1dcdec0710 mds0.cache trim_client_leases
360
2011-06-20 21:55:54.071414 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
361
2011-06-20 21:55:54.071446 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
362
2011-06-20 21:55:54.071521 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
363
2011-06-20 21:55:54.071549 7f1dcdec0710 mds0.locker scatter_tick
364
2011-06-20 21:55:54.071563 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
365
2011-06-20 21:55:54.071578 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
366
2011-06-20 21:55:54.071592 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:36.631082)
367
2011-06-20 21:55:54.071604 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:54.071601
368
2011-06-20 21:55:54.071613 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
369
2011-06-20 21:55:56.646883 7f1dcefc3710 mds0.server get_session have 0x7f1d9c000920 client5215 192.168.98.110:0/1893654265 state open
370
2011-06-20 21:55:56.646929 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 11) from client5215
371
2011-06-20 21:55:56.765043 7f1dcdec0710 mds0.6 beacon_send up:active seq 82 (currently up:active)
372
2011-06-20 21:55:56.765815 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 82 rtt 0.000730
373
2011-06-20 21:55:59.011477 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
374
2011-06-20 21:55:59.011610 7f1dcdec0710 mds0.cache trim_client_leases
375
2011-06-20 21:55:59.077492 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
376
2011-06-20 21:55:59.077524 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
377
2011-06-20 21:55:59.077594 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
378
2011-06-20 21:55:59.085136 7f1dcdec0710 mds0.locker scatter_tick
379
2011-06-20 21:55:59.085159 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
380
2011-06-20 21:55:59.085175 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399
381
2011-06-20 21:55:59.085191 7f1dcdec0710 mds0.server laggiest active session is client4107 192.168.98.109:0/3628510399 and sufficiently new (2011-06-20 21:55:41.730545)
382
2011-06-20 21:55:59.085225 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:55:59.085219
383
2011-06-20 21:55:59.085342 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
384
2011-06-20 21:55:59.085395 7f1dcdec0710 mds0.bal mds0 epoch 29 load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
385
2011-06-20 21:55:59.085412 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
386
2011-06-20 21:56:00.765224 7f1dcdec0710 mds0.6 beacon_send up:active seq 83 (currently up:active)
387
2011-06-20 21:56:00.766033 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 83 rtt 0.000744
388
2011-06-20 21:56:01.730958 7f1dcefc3710 mds0.server get_session have 0x7f1dc40078e0 client4107 192.168.98.109:0/3628510399 state open
389
2011-06-20 21:56:01.731002 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 542) from client4107
390
2011-06-20 21:56:02.856455 7f1dcefc3710 mds0.server get_session have 0x7f1dc4007b10 client4100 192.168.98.111:0/2049295103 state open
391
2011-06-20 21:56:02.856498 7f1dcefc3710 mds0.server handle_client_session client_session(request_renewcaps seq 520) from client4100
392
2011-06-20 21:56:04.011547 7f1dcdec0710 mds0.cache trim max=100000  cur=9303
393
2011-06-20 21:56:04.011680 7f1dcdec0710 mds0.cache trim_client_leases
394
2011-06-20 21:56:04.075113 7f1dcdec0710 mds0.cache check_memory_usage total 1154648, rss 53340, heap 144540, malloc 14170 mmap 0, baseline 135880, buffers 0, max 1048576, 108 / 9173 inodes have caps, 118 caps, 0.0128638 caps per inode
395
2011-06-20 21:56:04.075148 7f1dcdec0710 mds0.log trim 30 / 30 segments, 31543 / -1 events, 0 (0) expiring, 0 (0) expired
396
2011-06-20 21:56:04.075222 7f1dcdec0710 mds0.bal get_load mdsload<[0,0 0]/[0,0 0], req 0, hr 0, qlen 0, cpu 0.02>
397
2011-06-20 21:56:04.075265 7f1dcdec0710 mds0.locker scatter_tick
398
2011-06-20 21:56:04.075280 7f1dcdec0710 mds0.server find_idle_sessions.  laggy until 0.000000
399
2011-06-20 21:56:04.075295 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265
400
2011-06-20 21:56:04.075309 7f1dcdec0710 mds0.server laggiest active session is client5215 192.168.98.110:0/1893654265 and sufficiently new (2011-06-20 21:55:56.646938)
401
2011-06-20 21:56:04.075322 7f1dcdec0710 mds0.bal tick last_sample now 2011-06-20 21:56:04.075319
402
2011-06-20 21:56:04.075331 7f1dcdec0710 mds0.snap check_osd_map - version unchanged
403
2011-06-20 21:56:04.765380 7f1dcdec0710 mds0.6 beacon_send up:active seq 84 (currently up:active)
404
2011-06-20 21:56:04.766227 7f1dcefc3710 mds0.6 handle_mds_beacon up:active seq 84 rtt 0.000792
405
2011-06-20 21:56:04.846443 7f1dcefc3710 mds0.6 handle_command args: [injectargs,--debug_mds 0]