Actions
Bug #4999
closedmonitor sync failure
% Done:
0%
Source:
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
I've been testing v0.61 and v0.61.1 with 262,144 PGs in
a single pool, 576 OSDs, 3 mon, 3 MDS (1 active + 2 standby).
After starting a new filesystem and writing data to it
for a while from various numbers of CephFS clients, one
of my monitors dies like this:
-19> 2013-05-09 16:02:28.707657 7ffff2a8c700 5 mon.cs30@2(peon).paxos(paxos recovering c 4713..4749) handle_collect leader's lowest version is too high for our last committed (theirs: 4775; ours: 4749) -- bootstrap!
then
0> 2013-05-09 16:03:29.389081 7ffff328d700 -1 mon/Monitor.cc: In function 'void Monitor::sync_start_reply_timeout()' thread 7ffff328d700 time 2013-05-09 16:03:29.384508
If it matters, here's the mon leveldb config I'm using to make 256K PGs work:
[mon]
mon leveldb block size = 524288
mon leveldb write buffer size = 4194304
mon leveldb cache size = 134217728
mon leveldb compression = false
mon compact on trim = true
mon sync max payload size = 4194304
This issue is reproducible for me, so let me know what I can
do to help sort it out.
Restarting the monitor after it fails like this never works;
reinitializing it and then restarting it sometimes works,
sometimes runs into a sync issue again.
Here's what log I have leading up to the assert:
-758> 2013-05-09 16:00:20.218489 7ffff2a8c700 7 mon.cs30@2(peon).log v1412 update_from_paxos applying incremental log 1412 2013-05-09 16:00:15.025852 mon.0 172.17.131.28:6789/0 3354 : [INF] pgmap v1744: 262144 pgs: 262144 active+clean; 133 GB data, 277 GB used, 502 TB / 503 TB avail; 118MB/s rd, 101KB/s wr, 7579op/s -757> 2013-05-09 16:00:20.288434 7ffff2a8c700 1 mon.cs30@2(peon).log v1412 check_sub sending message to client.? 172.17.131.31:0/19500 with 1 entries (version 1412) -756> 2013-05-09 16:00:21.554919 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4739) is_readable now=2013-05-09 16:00:21.554928 lease_expire=2013-05-09 16:00:25.508722 has v0 lc 4739 -755> 2013-05-09 16:00:21.555737 7ffff2a8c700 7 mon.cs30@2(peon).pg v1745 update_from_paxos applying incremental 1746 -754> 2013-05-09 16:00:21.926047 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4740) is_readable now=2013-05-09 16:00:21.926052 lease_expire=2013-05-09 16:00:26.913232 has v0 lc 4740 -753> 2013-05-09 16:00:21.926319 7ffff2a8c700 7 mon.cs30@2(peon).log v1413 update_from_paxos applying incremental log 1413 2013-05-09 16:00:18.306005 mon.0 172.17.131.28:6789/0 3355 : [INF] pgmap v1745: 262144 pgs: 262144 active+clean; 136 GB data, 279 GB used, 502 TB / 503 TB avail; 299KB/s rd, 405MB/s wr, 521op/s -752> 2013-05-09 16:00:21.926401 7ffff2a8c700 7 mon.cs30@2(peon).log v1413 update_from_paxos applying incremental log 1413 2013-05-09 16:00:20.544090 mon.0 172.17.131.28:6789/0 3356 : [INF] pgmap v1746: 262144 pgs: 262144 active+clean; 136 GB data, 283 GB used, 502 TB / 503 TB avail; 114MB/s rd, 360MB/s wr, 7807op/s -751> 2013-05-09 16:00:21.992427 7ffff2a8c700 1 mon.cs30@2(peon).log v1413 check_sub sending message to client.? 172.17.131.31:0/19500 with 2 entries (version 1413) -750> 2013-05-09 16:00:22.409225 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4741) is_readable now=2013-05-09 16:00:22.409232 lease_expire=2013-05-09 16:00:27.410054 has v0 lc 4741 -749> 2013-05-09 16:00:22.409627 7ffff2a8c700 7 mon.cs30@2(peon).pg v1746 update_from_paxos applying incremental 1747 -748> 2013-05-09 16:00:23.889938 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4742) is_readable now=2013-05-09 16:00:23.889946 lease_expire=2013-05-09 16:00:28.890657 has v0 lc 4742 -747> 2013-05-09 16:00:23.890471 7ffff2a8c700 7 mon.cs30@2(peon).pg v1747 update_from_paxos applying incremental 1748 -746> 2013-05-09 16:00:23.978122 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4743) is_readable now=2013-05-09 16:00:23.978130 lease_expire=0.000000 has v0 lc 4743 -745> 2013-05-09 16:00:24.010636 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4743) is_readable now=2013-05-09 16:00:24.010639 lease_expire=2013-05-09 16:00:29.011452 has v0 lc 4743 -744> 2013-05-09 16:00:24.011108 7ffff2a8c700 7 mon.cs30@2(peon).log v1414 update_from_paxos applying incremental log 1414 2013-05-09 16:00:22.452001 mon.0 172.17.131.28:6789/0 3357 : [INF] pgmap v1747: 262144 pgs: 262144 active+clean; 136 GB data, 283 GB used, 502 TB / 503 TB avail; 347MB/s rd, 49210KB/s wr, 22438op/s -743> 2013-05-09 16:00:24.082920 7ffff2a8c700 1 mon.cs30@2(peon).log v1414 check_sub sending message to client.? 172.17.131.31:0/19500 with 1 entries (version 1414) -742> 2013-05-09 16:00:25.163082 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4744) is_readable now=2013-05-09 16:00:25.163086 lease_expire=2013-05-09 16:00:30.141842 has v0 lc 4744 -741> 2013-05-09 16:00:25.163219 7ffff2a8c700 7 mon.cs30@2(peon).pg v1748 update_from_paxos applying incremental 1749 -740> 2013-05-09 16:00:26.767771 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4745) is_readable now=2013-05-09 16:00:26.767776 lease_expire=2013-05-09 16:00:31.733366 has v0 lc 4745 -739> 2013-05-09 16:00:26.768050 7ffff2a8c700 7 mon.cs30@2(peon).pg v1749 update_from_paxos applying incremental 1750 -738> 2013-05-09 16:00:26.797412 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4746) is_readable now=2013-05-09 16:00:26.797415 lease_expire=2013-05-09 16:00:31.793233 has v0 lc 4746 -737> 2013-05-09 16:00:26.797646 7ffff2a8c700 7 mon.cs30@2(peon).log v1415 update_from_paxos applying incremental log 1415 2013-05-09 16:00:23.908847 mon.0 172.17.131.28:6789/0 3358 : [INF] pgmap v1748: 262144 pgs: 262144 active+clean; 137 GB data, 283 GB used, 502 TB / 503 TB avail; 463MB/s rd, 231MB/s wr, 30026op/s -736> 2013-05-09 16:00:26.797723 7ffff2a8c700 7 mon.cs30@2(peon).log v1415 update_from_paxos applying incremental log 1415 2013-05-09 16:00:25.144683 mon.0 172.17.131.28:6789/0 3359 : [INF] pgmap v1749: 262144 pgs: 262144 active+clean; 137 GB data, 283 GB used, 502 TB / 503 TB avail; 213MB/s rd, 364MB/s wr, 14033op/s -735> 2013-05-09 16:00:26.873853 7ffff2a8c700 1 mon.cs30@2(peon).log v1415 check_sub sending message to client.? 172.17.131.31:0/19500 with 2 entries (version 1415) -734> 2013-05-09 16:00:28.373677 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4747) is_readable now=2013-05-09 16:00:28.373683 lease_expire=2013-05-09 16:00:33.324894 has v0 lc 4747 -733> 2013-05-09 16:00:28.374061 7ffff2a8c700 7 mon.cs30@2(peon).pg v1750 update_from_paxos applying incremental 1751 -732> 2013-05-09 16:00:29.832912 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos active c 4713..4748) is_readable now=2013-05-09 16:00:29.832915 lease_expire=2013-05-09 16:00:34.826224 has v0 lc 4748 -731> 2013-05-09 16:00:29.833081 7ffff2a8c700 7 mon.cs30@2(peon).log v1416 update_from_paxos applying incremental log 1416 2013-05-09 16:00:26.748039 mon.0 172.17.131.28:6789/0 3360 : [INF] pgmap v1750: 262144 pgs: 262144 active+clean; 138 GB data, 285 GB used, 502 TB / 503 TB avail; 134MB/s rd, 411MB/s wr, 9137op/s -730> 2013-05-09 16:00:29.833135 7ffff2a8c700 7 mon.cs30@2(peon).log v1416 update_from_paxos applying incremental log 1416 2013-05-09 16:00:28.350306 mon.0 172.17.131.28:6789/0 3361 : [INF] pgmap v1751: 262144 pgs: 262144 active+clean; 139 GB data, 288 GB used, 502 TB / 503 TB avail; 589MB/s rd, 606MB/s wr, 38694op/s -729> 2013-05-09 16:00:30.018122 7ffff2a8c700 1 mon.cs30@2(peon).log v1416 check_sub sending message to client.? 172.17.131.31:0/19500 with 2 entries (version 1416) -728> 2013-05-09 16:00:30.650742 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:30.650748 lease_expire=0.000000 has v0 lc 4748 -727> 2013-05-09 16:00:30.881629 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:30.881634 lease_expire=0.000000 has v0 lc 4748 -726> 2013-05-09 16:00:31.245967 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:31.245973 lease_expire=0.000000 has v0 lc 4748 -725> 2013-05-09 16:00:31.821365 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:31.821369 lease_expire=0.000000 has v0 lc 4748 -724> 2013-05-09 16:00:31.874846 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:31.874849 lease_expire=0.000000 has v0 lc 4748 -723> 2013-05-09 16:00:31.946519 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:31.946522 lease_expire=0.000000 has v0 lc 4748 -722> 2013-05-09 16:00:31.996611 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:31.996615 lease_expire=0.000000 has v0 lc 4748 -721> 2013-05-09 16:00:33.980641 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:33.980647 lease_expire=0.000000 has v0 lc 4748 -720> 2013-05-09 16:00:35.652313 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:35.652318 lease_expire=0.000000 has v0 lc 4748 -719> 2013-05-09 16:00:35.882745 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:35.882750 lease_expire=0.000000 has v0 lc 4748 -718> 2013-05-09 16:00:36.247011 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:36.247015 lease_expire=0.000000 has v0 lc 4748 -717> 2013-05-09 16:00:36.822630 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:36.822636 lease_expire=0.000000 has v0 lc 4748 -716> 2013-05-09 16:00:36.875907 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:36.875911 lease_expire=0.000000 has v0 lc 4748 -715> 2013-05-09 16:00:36.947733 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:36.947736 lease_expire=0.000000 has v0 lc 4748 -714> 2013-05-09 16:00:36.997834 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:36.997837 lease_expire=0.000000 has v0 lc 4748 -713> 2013-05-09 16:00:38.981939 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:38.981945 lease_expire=0.000000 has v0 lc 4748 -712> 2013-05-09 16:00:39.833039 7ffff328d700 5 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) lease_timeout -- calling new election -711> 2013-05-09 16:00:39.834053 7ffff2a8c700 0 log [INF] : mon.cs30 calling new monitor election -710> 2013-05-09 16:00:39.834065 7ffff2a8c700 10 send_log to self -709> 2013-05-09 16:00:39.834067 7ffff2a8c700 10 log_queue is 2 last_log 16 sent 14 num 2 unsent 2 sending 2 -708> 2013-05-09 16:00:39.834071 7ffff2a8c700 10 will send 2013-05-09 15:43:50.856021 mon.2 172.17.131.30:6789/0 15 : [INF] mon.cs30 calling new monitor election -707> 2013-05-09 16:00:39.834088 7ffff2a8c700 10 will send 2013-05-09 16:00:39.834064 mon.2 172.17.131.30:6789/0 16 : [INF] mon.cs30 calling new monitor election -706> 2013-05-09 16:00:39.834101 7ffff2a8c700 5 mon.cs30@2(electing).elector(74) start -- can i be leader? -705> 2013-05-09 16:00:39.834242 7ffff2a8c700 1 mon.cs30@2(electing).elector(74) init, last seen epoch 74 -704> 2013-05-09 16:00:39.842804 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:39.842807 lease_expire=0.000000 has v0 lc 4748 -703> 2013-05-09 16:00:39.850432 7ffff2a8c700 5 mon.cs30@2(electing).elector(75) handle_propose from mon.1 -702> 2013-05-09 16:00:39.850449 7ffff2a8c700 5 mon.cs30@2(electing).elector(75) defer to 1 -701> 2013-05-09 16:00:40.653354 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:40.653360 lease_expire=0.000000 has v0 lc 4748 -700> 2013-05-09 16:00:40.884331 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:40.884336 lease_expire=0.000000 has v0 lc 4748 -699> 2013-05-09 16:00:41.248140 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:41.248143 lease_expire=0.000000 has v0 lc 4748 -698> 2013-05-09 16:00:41.824077 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:41.824083 lease_expire=0.000000 has v0 lc 4748 -697> 2013-05-09 16:00:41.877141 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:41.877144 lease_expire=0.000000 has v0 lc 4748 -696> 2013-05-09 16:00:41.949003 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:41.949007 lease_expire=0.000000 has v0 lc 4748 -695> 2013-05-09 16:00:41.999048 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:41.999051 lease_expire=0.000000 has v0 lc 4748 -694> 2013-05-09 16:00:43.983226 7ffff2a8c700 1 mon.cs30@2(electing).paxos(paxos recovering c 4713..4748) is_readable now=2013-05-09 16:00:43.983233 lease_expire=0.000000 has v0 lc 4748 -693> 2013-05-09 16:00:44.861844 7ffff2a8c700 5 mon.cs30@2(electing).elector(75) handle_victory from mon.1 quorum_features 4294967295 -692> 2013-05-09 16:00:45.654641 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:45.654647 lease_expire=0.000000 has v0 lc 4748 -691> 2013-05-09 16:00:45.885517 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:45.885521 lease_expire=0.000000 has v0 lc 4748 -690> 2013-05-09 16:00:46.249584 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:46.249588 lease_expire=0.000000 has v0 lc 4748 -689> 2013-05-09 16:00:46.825535 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:46.825539 lease_expire=0.000000 has v0 lc 4748 -688> 2013-05-09 16:00:46.878323 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:46.878327 lease_expire=0.000000 has v0 lc 4748 -687> 2013-05-09 16:00:46.950454 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:46.950459 lease_expire=0.000000 has v0 lc 4748 -686> 2013-05-09 16:00:47.000176 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:47.000182 lease_expire=0.000000 has v0 lc 4748 -685> 2013-05-09 16:00:48.984636 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:48.984644 lease_expire=0.000000 has v0 lc 4748 -684> 2013-05-09 16:00:50.656177 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:50.656182 lease_expire=0.000000 has v0 lc 4748 -683> 2013-05-09 16:00:50.886886 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:50.886890 lease_expire=0.000000 has v0 lc 4748 -682> 2013-05-09 16:00:50.898396 7ffff328d700 0 mon.cs30@2(peon).data_health(76) update_stats avail 90% total 134219776 used 6056888 avail 121856968 -681> 2013-05-09 16:00:51.250843 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:51.250846 lease_expire=0.000000 has v0 lc 4748 -680> 2013-05-09 16:00:51.826790 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:51.826793 lease_expire=0.000000 has v0 lc 4748 -679> 2013-05-09 16:00:51.879377 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:51.879381 lease_expire=0.000000 has v0 lc 4748 -678> 2013-05-09 16:00:51.951876 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:51.951880 lease_expire=0.000000 has v0 lc 4748 -677> 2013-05-09 16:00:52.001374 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:52.001378 lease_expire=0.000000 has v0 lc 4748 -676> 2013-05-09 16:00:53.986059 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:53.986066 lease_expire=0.000000 has v0 lc 4748 -675> 2013-05-09 16:00:55.657449 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:55.657455 lease_expire=0.000000 has v0 lc 4748 -674> 2013-05-09 16:00:55.664852 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:55.664855 lease_expire=0.000000 has v0 lc 4748 -673> 2013-05-09 16:00:55.828756 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:55.828761 lease_expire=0.000000 has v0 lc 4748 -672> 2013-05-09 16:00:55.887962 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:55.887964 lease_expire=0.000000 has v0 lc 4748 -671> 2013-05-09 16:00:56.138393 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.138397 lease_expire=0.000000 has v0 lc 4748 -670> 2013-05-09 16:00:56.252211 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.252214 lease_expire=0.000000 has v0 lc 4748 -669> 2013-05-09 16:00:56.336613 7fffe599e700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2c3dc380 sd=35 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -668> 2013-05-09 16:00:56.343350 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.343353 lease_expire=0.000000 has v0 lc 4748 -667> 2013-05-09 16:00:56.363084 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.363087 lease_expire=0.000000 has v0 lc 4748 -666> 2013-05-09 16:00:56.389686 7fffe599e700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1dc7b200 sd=35 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -665> 2013-05-09 16:00:56.397941 7fffe599e700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e1d0000 sd=35 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -664> 2013-05-09 16:00:56.407029 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.407032 lease_expire=0.000000 has v0 lc 4748 -663> 2013-05-09 16:00:56.749560 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.749564 lease_expire=0.000000 has v0 lc 4748 -662> 2013-05-09 16:00:56.769930 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.769934 lease_expire=0.000000 has v0 lc 4748 -661> 2013-05-09 16:00:56.789890 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.789893 lease_expire=0.000000 has v0 lc 4748 -660> 2013-05-09 16:00:56.811713 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.811716 lease_expire=0.000000 has v0 lc 4748 -659> 2013-05-09 16:00:56.828068 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.828072 lease_expire=0.000000 has v0 lc 4748 -658> 2013-05-09 16:00:56.854565 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.854568 lease_expire=0.000000 has v0 lc 4748 -657> 2013-05-09 16:00:56.862594 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.862597 lease_expire=0.000000 has v0 lc 4748 -656> 2013-05-09 16:00:56.880736 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.880740 lease_expire=0.000000 has v0 lc 4748 -655> 2013-05-09 16:00:56.881426 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.881429 lease_expire=0.000000 has v0 lc 4748 -654> 2013-05-09 16:00:56.913497 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.913500 lease_expire=0.000000 has v0 lc 4748 -653> 2013-05-09 16:00:56.953228 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.953231 lease_expire=0.000000 has v0 lc 4748 -652> 2013-05-09 16:00:56.953969 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.953972 lease_expire=0.000000 has v0 lc 4748 -651> 2013-05-09 16:00:56.972440 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.972443 lease_expire=0.000000 has v0 lc 4748 -650> 2013-05-09 16:00:56.977989 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.977993 lease_expire=0.000000 has v0 lc 4748 -649> 2013-05-09 16:00:56.978232 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.978235 lease_expire=0.000000 has v0 lc 4748 -648> 2013-05-09 16:00:56.982305 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:56.982309 lease_expire=0.000000 has v0 lc 4748 -647> 2013-05-09 16:00:57.002581 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.002585 lease_expire=0.000000 has v0 lc 4748 -646> 2013-05-09 16:00:57.021532 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.021535 lease_expire=0.000000 has v0 lc 4748 -645> 2013-05-09 16:00:57.048447 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.048450 lease_expire=0.000000 has v0 lc 4748 -644> 2013-05-09 16:00:57.081478 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.081481 lease_expire=0.000000 has v0 lc 4748 -643> 2013-05-09 16:00:57.085974 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.085977 lease_expire=0.000000 has v0 lc 4748 -642> 2013-05-09 16:00:57.328897 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.328900 lease_expire=0.000000 has v0 lc 4748 -641> 2013-05-09 16:00:57.329399 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.329402 lease_expire=0.000000 has v0 lc 4748 -640> 2013-05-09 16:00:57.457705 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.457708 lease_expire=0.000000 has v0 lc 4748 -639> 2013-05-09 16:00:57.495871 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.495875 lease_expire=0.000000 has v0 lc 4748 -638> 2013-05-09 16:00:57.526175 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.526179 lease_expire=0.000000 has v0 lc 4748 -637> 2013-05-09 16:00:57.543841 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.543845 lease_expire=0.000000 has v0 lc 4748 -636> 2013-05-09 16:00:57.573740 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.573743 lease_expire=0.000000 has v0 lc 4748 -635> 2013-05-09 16:00:57.578190 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.578193 lease_expire=0.000000 has v0 lc 4748 -634> 2013-05-09 16:00:57.590892 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.590896 lease_expire=0.000000 has v0 lc 4748 -633> 2013-05-09 16:00:57.626610 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.626614 lease_expire=0.000000 has v0 lc 4748 -632> 2013-05-09 16:00:57.630873 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.630875 lease_expire=0.000000 has v0 lc 4748 -631> 2013-05-09 16:00:57.663602 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.663606 lease_expire=0.000000 has v0 lc 4748 -630> 2013-05-09 16:00:57.722794 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.722798 lease_expire=0.000000 has v0 lc 4748 -629> 2013-05-09 16:00:57.735836 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.735840 lease_expire=0.000000 has v0 lc 4748 -628> 2013-05-09 16:00:57.751602 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.751606 lease_expire=0.000000 has v0 lc 4748 -627> 2013-05-09 16:00:57.765883 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:57.765886 lease_expire=0.000000 has v0 lc 4748 -626> 2013-05-09 16:00:57.775332 7fffd854f700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x17caf80 sd=65 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -625> 2013-05-09 16:00:57.887592 7fffd854f700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55f080 sd=65 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -624> 2013-05-09 16:00:58.365623 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.365626 lease_expire=0.000000 has v0 lc 4748 -623> 2013-05-09 16:00:58.385940 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.385943 lease_expire=0.000000 has v0 lc 4748 -622> 2013-05-09 16:00:58.696600 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.696603 lease_expire=0.000000 has v0 lc 4748 -621> 2013-05-09 16:00:58.716013 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.716017 lease_expire=0.000000 has v0 lc 4748 -620> 2013-05-09 16:00:58.719212 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.719215 lease_expire=0.000000 has v0 lc 4748 -619> 2013-05-09 16:00:58.735507 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.735511 lease_expire=0.000000 has v0 lc 4748 -618> 2013-05-09 16:00:58.760920 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.760923 lease_expire=0.000000 has v0 lc 4748 -617> 2013-05-09 16:00:58.801826 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.801829 lease_expire=0.000000 has v0 lc 4748 -616> 2013-05-09 16:00:58.810915 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.810919 lease_expire=0.000000 has v0 lc 4748 -615> 2013-05-09 16:00:58.814235 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.814239 lease_expire=0.000000 has v0 lc 4748 -614> 2013-05-09 16:00:58.827913 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.827916 lease_expire=0.000000 has v0 lc 4748 -613> 2013-05-09 16:00:58.830793 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.830797 lease_expire=0.000000 has v0 lc 4748 -612> 2013-05-09 16:00:58.839604 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.839607 lease_expire=0.000000 has v0 lc 4748 -611> 2013-05-09 16:00:58.841718 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.841722 lease_expire=0.000000 has v0 lc 4748 -610> 2013-05-09 16:00:58.844592 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.844595 lease_expire=0.000000 has v0 lc 4748 -609> 2013-05-09 16:00:58.887915 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.887918 lease_expire=0.000000 has v0 lc 4748 -608> 2013-05-09 16:00:58.894158 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.894162 lease_expire=0.000000 has v0 lc 4748 -607> 2013-05-09 16:00:58.901221 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.901225 lease_expire=0.000000 has v0 lc 4748 -606> 2013-05-09 16:00:58.904751 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.904754 lease_expire=0.000000 has v0 lc 4748 -605> 2013-05-09 16:00:58.908906 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.908909 lease_expire=0.000000 has v0 lc 4748 -604> 2013-05-09 16:00:58.971693 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.971696 lease_expire=0.000000 has v0 lc 4748 -603> 2013-05-09 16:00:58.976650 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.976653 lease_expire=0.000000 has v0 lc 4748 -602> 2013-05-09 16:00:58.980244 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.980320 lease_expire=0.000000 has v0 lc 4748 -601> 2013-05-09 16:00:58.983739 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:58.983742 lease_expire=0.000000 has v0 lc 4748 -600> 2013-05-09 16:00:59.007008 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.007011 lease_expire=0.000000 has v0 lc 4748 -599> 2013-05-09 16:00:59.014895 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.014898 lease_expire=0.000000 has v0 lc 4748 -598> 2013-05-09 16:00:59.018003 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.018007 lease_expire=0.000000 has v0 lc 4748 -597> 2013-05-09 16:00:59.032983 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.032986 lease_expire=0.000000 has v0 lc 4748 -596> 2013-05-09 16:00:59.033569 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.033572 lease_expire=0.000000 has v0 lc 4748 -595> 2013-05-09 16:00:59.037753 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.037756 lease_expire=0.000000 has v0 lc 4748 -594> 2013-05-09 16:00:59.041010 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.041013 lease_expire=0.000000 has v0 lc 4748 -593> 2013-05-09 16:00:59.044868 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.044872 lease_expire=0.000000 has v0 lc 4748 -592> 2013-05-09 16:00:59.057053 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.057056 lease_expire=0.000000 has v0 lc 4748 -591> 2013-05-09 16:00:59.060844 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.060847 lease_expire=0.000000 has v0 lc 4748 -590> 2013-05-09 16:00:59.068189 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.068192 lease_expire=0.000000 has v0 lc 4748 -589> 2013-05-09 16:00:59.092781 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.092784 lease_expire=0.000000 has v0 lc 4748 -588> 2013-05-09 16:00:59.108753 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.108756 lease_expire=0.000000 has v0 lc 4748 -587> 2013-05-09 16:00:59.120583 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.120587 lease_expire=0.000000 has v0 lc 4748 -586> 2013-05-09 16:00:59.120744 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.120746 lease_expire=0.000000 has v0 lc 4748 -585> 2013-05-09 16:00:59.153375 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.153379 lease_expire=0.000000 has v0 lc 4748 -584> 2013-05-09 16:00:59.170391 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.170395 lease_expire=0.000000 has v0 lc 4748 -583> 2013-05-09 16:00:59.203869 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.203872 lease_expire=0.000000 has v0 lc 4748 -582> 2013-05-09 16:00:59.204469 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.204472 lease_expire=0.000000 has v0 lc 4748 -581> 2013-05-09 16:00:59.222996 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.222998 lease_expire=0.000000 has v0 lc 4748 -580> 2013-05-09 16:00:59.262496 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:00:59.262500 lease_expire=0.000000 has v0 lc 4748 -579> 2013-05-09 16:01:00.304472 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:00.304476 lease_expire=0.000000 has v0 lc 4748 -578> 2013-05-09 16:01:00.505113 7fffce2fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x19e2a80 sd=106 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -577> 2013-05-09 16:01:00.551049 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:00.551052 lease_expire=0.000000 has v0 lc 4748 -576> 2013-05-09 16:01:00.574986 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:00.574989 lease_expire=0.000000 has v0 lc 4748 -575> 2013-05-09 16:01:00.840825 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:00.840829 lease_expire=0.000000 has v0 lc 4748 -574> 2013-05-09 16:01:00.982624 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:00.982630 lease_expire=0.000000 has v0 lc 4748 -573> 2013-05-09 16:01:01.246841 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.246846 lease_expire=0.000000 has v0 lc 4748 -572> 2013-05-09 16:01:01.265371 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.265375 lease_expire=0.000000 has v0 lc 4748 -571> 2013-05-09 16:01:01.376445 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.376448 lease_expire=0.000000 has v0 lc 4748 -570> 2013-05-09 16:01:01.485106 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.485110 lease_expire=0.000000 has v0 lc 4748 -569> 2013-05-09 16:01:01.900726 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.900730 lease_expire=0.000000 has v0 lc 4748 -568> 2013-05-09 16:01:01.923581 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.923584 lease_expire=0.000000 has v0 lc 4748 -567> 2013-05-09 16:01:01.957579 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.957583 lease_expire=0.000000 has v0 lc 4748 -566> 2013-05-09 16:01:01.968737 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:01.968741 lease_expire=0.000000 has v0 lc 4748 -565> 2013-05-09 16:01:02.036873 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.036877 lease_expire=0.000000 has v0 lc 4748 -564> 2013-05-09 16:01:02.191458 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.191461 lease_expire=0.000000 has v0 lc 4748 -563> 2013-05-09 16:01:02.203621 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.203624 lease_expire=0.000000 has v0 lc 4748 -562> 2013-05-09 16:01:02.217761 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.217765 lease_expire=0.000000 has v0 lc 4748 -561> 2013-05-09 16:01:02.253631 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.253634 lease_expire=0.000000 has v0 lc 4748 -560> 2013-05-09 16:01:02.278668 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.278671 lease_expire=0.000000 has v0 lc 4748 -559> 2013-05-09 16:01:02.330197 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.330200 lease_expire=0.000000 has v0 lc 4748 -558> 2013-05-09 16:01:02.367334 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.367337 lease_expire=0.000000 has v0 lc 4748 -557> 2013-05-09 16:01:02.392663 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.392667 lease_expire=0.000000 has v0 lc 4748 -556> 2013-05-09 16:01:02.403220 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.403223 lease_expire=0.000000 has v0 lc 4748 -555> 2013-05-09 16:01:02.414029 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.414032 lease_expire=0.000000 has v0 lc 4748 -554> 2013-05-09 16:01:02.419120 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.419123 lease_expire=0.000000 has v0 lc 4748 -553> 2013-05-09 16:01:02.516863 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.516867 lease_expire=0.000000 has v0 lc 4748 -552> 2013-05-09 16:01:02.529030 7fffc2a78700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x19e5f00 sd=128 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -551> 2013-05-09 16:01:02.550505 7fffc2a78700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x19e0c80 sd=128 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -550> 2013-05-09 16:01:02.605428 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.605431 lease_expire=0.000000 has v0 lc 4748 -549> 2013-05-09 16:01:02.611718 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.611722 lease_expire=0.000000 has v0 lc 4748 -548> 2013-05-09 16:01:02.621636 7fffc2674700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x377eee00 sd=130 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -547> 2013-05-09 16:01:02.721658 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.721663 lease_expire=0.000000 has v0 lc 4748 -546> 2013-05-09 16:01:02.763380 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.763384 lease_expire=0.000000 has v0 lc 4748 -545> 2013-05-09 16:01:02.770441 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.770444 lease_expire=0.000000 has v0 lc 4748 -544> 2013-05-09 16:01:02.783948 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.783951 lease_expire=0.000000 has v0 lc 4748 -543> 2013-05-09 16:01:02.801166 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.801169 lease_expire=0.000000 has v0 lc 4748 -542> 2013-05-09 16:01:02.831132 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.831135 lease_expire=0.000000 has v0 lc 4748 -541> 2013-05-09 16:01:02.848451 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.848455 lease_expire=0.000000 has v0 lc 4748 -540> 2013-05-09 16:01:02.878101 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.878105 lease_expire=0.000000 has v0 lc 4748 -539> 2013-05-09 16:01:02.898922 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.898925 lease_expire=0.000000 has v0 lc 4748 -538> 2013-05-09 16:01:02.921561 7fffb8279700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x19e6900 sd=136 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -537> 2013-05-09 16:01:02.952908 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.952911 lease_expire=0.000000 has v0 lc 4748 -536> 2013-05-09 16:01:02.955987 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.955991 lease_expire=0.000000 has v0 lc 4748 -535> 2013-05-09 16:01:02.956387 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:02.956390 lease_expire=0.000000 has v0 lc 4748 -534> 2013-05-09 16:01:03.066893 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:03.066896 lease_expire=0.000000 has v0 lc 4748 -533> 2013-05-09 16:01:03.123566 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:03.123570 lease_expire=0.000000 has v0 lc 4748 -532> 2013-05-09 16:01:03.154610 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:03.154613 lease_expire=0.000000 has v0 lc 4748 -531> 2013-05-09 16:01:03.215423 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:03.215427 lease_expire=0.000000 has v0 lc 4748 -530> 2013-05-09 16:01:03.221199 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1cbddf00 sd=141 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -529> 2013-05-09 16:01:04.080156 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:04.080159 lease_expire=0.000000 has v0 lc 4748 -528> 2013-05-09 16:01:05.370957 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:05.370964 lease_expire=0.000000 has v0 lc 4748 -527> 2013-05-09 16:01:05.484004 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x377ec600 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -526> 2013-05-09 16:01:05.704446 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:05.704450 lease_expire=0.000000 has v0 lc 4748 -525> 2013-05-09 16:01:05.816471 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:05.816474 lease_expire=0.000000 has v0 lc 4748 -524> 2013-05-09 16:01:05.856470 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:05.856473 lease_expire=0.000000 has v0 lc 4748 -523> 2013-05-09 16:01:05.894483 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab08f00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -522> 2013-05-09 16:01:05.897439 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c231180 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -521> 2013-05-09 16:01:05.917116 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:05.917119 lease_expire=0.000000 has v0 lc 4748 -520> 2013-05-09 16:01:05.974063 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55e400 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -519> 2013-05-09 16:01:06.003271 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e05f00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -518> 2013-05-09 16:01:06.005386 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x377e8780 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -517> 2013-05-09 16:01:06.018264 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55fa80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -516> 2013-05-09 16:01:06.022523 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55dc80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -515> 2013-05-09 16:01:06.029093 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.029096 lease_expire=0.000000 has v0 lc 4748 -514> 2013-05-09 16:01:06.050732 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c235500 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -513> 2013-05-09 16:01:06.081430 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e1d2800 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -512> 2013-05-09 16:01:06.088402 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.088406 lease_expire=0.000000 has v0 lc 4748 -511> 2013-05-09 16:01:06.107584 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.107588 lease_expire=0.000000 has v0 lc 4748 -510> 2013-05-09 16:01:06.115206 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.115209 lease_expire=0.000000 has v0 lc 4748 -509> 2013-05-09 16:01:06.199428 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c232d00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -508> 2013-05-09 16:01:06.207277 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.207281 lease_expire=0.000000 has v0 lc 4748 -507> 2013-05-09 16:01:06.208318 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.208321 lease_expire=0.000000 has v0 lc 4748 -506> 2013-05-09 16:01:06.263484 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.263488 lease_expire=0.000000 has v0 lc 4748 -505> 2013-05-09 16:01:06.290038 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.290042 lease_expire=0.000000 has v0 lc 4748 -504> 2013-05-09 16:01:06.294526 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.294530 lease_expire=0.000000 has v0 lc 4748 -503> 2013-05-09 16:01:06.302407 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.302412 lease_expire=0.000000 has v0 lc 4748 -502> 2013-05-09 16:01:06.343344 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.343348 lease_expire=0.000000 has v0 lc 4748 -501> 2013-05-09 16:01:06.370106 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.370110 lease_expire=0.000000 has v0 lc 4748 -500> 2013-05-09 16:01:06.390469 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ade3200 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -499> 2013-05-09 16:01:06.398858 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ade6400 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -498> 2013-05-09 16:01:06.402293 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x7e70000 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -497> 2013-05-09 16:01:06.406004 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1dc7f800 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -496> 2013-05-09 16:01:06.425831 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e1d3200 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -495> 2013-05-09 16:01:06.436929 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.436932 lease_expire=0.000000 has v0 lc 4748 -494> 2013-05-09 16:01:06.448475 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.448478 lease_expire=0.000000 has v0 lc 4748 -493> 2013-05-09 16:01:06.570189 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:06.570193 lease_expire=0.000000 has v0 lc 4748 -492> 2013-05-09 16:01:06.721951 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e02580 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -491> 2013-05-09 16:01:07.102733 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.102737 lease_expire=0.000000 has v0 lc 4748 -490> 2013-05-09 16:01:07.175365 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.175368 lease_expire=0.000000 has v0 lc 4748 -489> 2013-05-09 16:01:07.246116 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e7a3700 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -488> 2013-05-09 16:01:07.252240 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e06180 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -487> 2013-05-09 16:01:07.255736 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e02800 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -486> 2013-05-09 16:01:07.259957 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab0b700 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -485> 2013-05-09 16:01:07.266757 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.266761 lease_expire=0.000000 has v0 lc 4748 -484> 2013-05-09 16:01:07.275409 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.275413 lease_expire=0.000000 has v0 lc 4748 -483> 2013-05-09 16:01:07.304209 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c237d00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -482> 2013-05-09 16:01:07.312541 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.312544 lease_expire=0.000000 has v0 lc 4748 -481> 2013-05-09 16:01:07.337785 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.337789 lease_expire=0.000000 has v0 lc 4748 -480> 2013-05-09 16:01:07.340662 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.340665 lease_expire=0.000000 has v0 lc 4748 -479> 2013-05-09 16:01:07.370190 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.370193 lease_expire=0.000000 has v0 lc 4748 -478> 2013-05-09 16:01:07.385369 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e559900 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -477> 2013-05-09 16:01:07.386620 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.386623 lease_expire=0.000000 has v0 lc 4748 -476> 2013-05-09 16:01:07.452446 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c233480 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -475> 2013-05-09 16:01:07.455007 7fffb13fb700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55f580 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -474> 2013-05-09 16:01:07.464515 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.464519 lease_expire=0.000000 has v0 lc 4748 -473> 2013-05-09 16:01:07.489085 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.489089 lease_expire=0.000000 has v0 lc 4748 -472> 2013-05-09 16:01:07.776155 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.776158 lease_expire=0.000000 has v0 lc 4748 -471> 2013-05-09 16:01:07.778675 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:07.778679 lease_expire=0.000000 has v0 lc 4748 -470> 2013-05-09 16:01:10.402797 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:10.402801 lease_expire=0.000000 has v0 lc 4748 -469> 2013-05-09 16:01:10.405768 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:10.405772 lease_expire=0.000000 has v0 lc 4748 -468> 2013-05-09 16:01:10.488517 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c231400 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -467> 2013-05-09 16:01:10.490495 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:10.490499 lease_expire=0.000000 has v0 lc 4748 -466> 2013-05-09 16:01:10.586468 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c236180 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -465> 2013-05-09 16:01:10.593408 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c230c80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -464> 2013-05-09 16:01:10.665761 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e05000 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -463> 2013-05-09 16:01:10.712149 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e55bc00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -462> 2013-05-09 16:01:10.785341 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c237a80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -461> 2013-05-09 16:01:10.863938 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1dc79180 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -460> 2013-05-09 16:01:10.903660 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:10.903664 lease_expire=0.000000 has v0 lc 4748 -459> 2013-05-09 16:01:10.944374 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:10.944377 lease_expire=0.000000 has v0 lc 4748 -458> 2013-05-09 16:01:10.975160 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e03700 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -457> 2013-05-09 16:01:10.982010 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e03480 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -456> 2013-05-09 16:01:10.985538 7fffb7c73700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e03c00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -455> 2013-05-09 16:01:10.985781 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c232300 sd=139 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -454> 2013-05-09 16:01:11.022195 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.022198 lease_expire=0.000000 has v0 lc 4748 -453> 2013-05-09 16:01:11.037022 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.037025 lease_expire=0.000000 has v0 lc 4748 -452> 2013-05-09 16:01:11.046771 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.046775 lease_expire=0.000000 has v0 lc 4748 -451> 2013-05-09 16:01:11.058649 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c235000 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -450> 2013-05-09 16:01:11.060227 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.060231 lease_expire=0.000000 has v0 lc 4748 -449> 2013-05-09 16:01:11.091236 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab0f800 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -448> 2013-05-09 16:01:11.095497 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1dc78780 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -447> 2013-05-09 16:01:11.095976 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab0c600 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -446> 2013-05-09 16:01:11.110770 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab0ee00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -445> 2013-05-09 16:01:11.111235 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e02d00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -444> 2013-05-09 16:01:11.115334 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.115337 lease_expire=0.000000 has v0 lc 4748 -443> 2013-05-09 16:01:11.118311 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.118315 lease_expire=0.000000 has v0 lc 4748 -442> 2013-05-09 16:01:11.123070 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.123074 lease_expire=0.000000 has v0 lc 4748 -441> 2013-05-09 16:01:11.129133 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.129137 lease_expire=0.000000 has v0 lc 4748 -440> 2013-05-09 16:01:11.144328 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c231e00 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -439> 2013-05-09 16:01:11.145707 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.145711 lease_expire=0.000000 has v0 lc 4748 -438> 2013-05-09 16:01:11.147493 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c235780 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -437> 2013-05-09 16:01:11.152330 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1ab0d780 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -436> 2013-05-09 16:01:11.185101 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2e559680 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -435> 2013-05-09 16:01:11.207800 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e04100 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -434> 2013-05-09 16:01:11.209224 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.209227 lease_expire=0.000000 has v0 lc 4748 -433> 2013-05-09 16:01:11.217965 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c233200 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -432> 2013-05-09 16:01:11.221132 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e7a6900 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -431> 2013-05-09 16:01:11.266301 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x2c3d9b80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -430> 2013-05-09 16:01:11.270371 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1dc7b480 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -429> 2013-05-09 16:01:11.301240 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1e1d0500 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -428> 2013-05-09 16:01:11.302880 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.302884 lease_expire=0.000000 has v0 lc 4748 -427> 2013-05-09 16:01:11.305194 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1b1baf80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -426> 2013-05-09 16:01:11.306489 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.306492 lease_expire=0.000000 has v0 lc 4748 -425> 2013-05-09 16:01:11.318907 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.318911 lease_expire=0.000000 has v0 lc 4748 -424> 2013-05-09 16:01:11.329907 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c232800 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -423> 2013-05-09 16:01:11.330368 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x1c237300 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -422> 2013-05-09 16:01:11.350082 7fffb7b72700 0 -- 172.17.131.30:6789/0 >> :/0 pipe(0x27e07a80 sd=105 :6789 s=0 pgs=0 cs=0 l=0).accept failed to getpeername 107 Transport endpoint is not connected -421> 2013-05-09 16:01:11.352177 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.352181 lease_expire=0.000000 has v0 lc 4748 -420> 2013-05-09 16:01:11.355009 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.355012 lease_expire=0.000000 has v0 lc 4748 -419> 2013-05-09 16:01:11.356144 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.356148 lease_expire=0.000000 has v0 lc 4748 -418> 2013-05-09 16:01:11.363298 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.363301 lease_expire=0.000000 has v0 lc 4748 -417> 2013-05-09 16:01:11.365622 7ffff2a8c700 1 mon.cs30@2(peon).paxos(paxos updating c 4713..4748) is_readable now=2013-05-09 16:01:11.365626 lease_expire=0.000000 has v0 l
Files
Actions