Project

General

Profile

Bug #3287 » mon.alpha.log

Mike Lowe, 10/11/2012 10:50 AM

 
2012-10-11 13:43:39.054808 7f16282a6780 0 ceph version 0.52 (commit:e48859474c4944d4ff201ddc9f5fd400e8898173), process ceph-mon, pid 31507
2012-10-11 13:43:39.054892 7f16282a6780 1 store(/data/mon.alpha) mount
2012-10-11 13:43:39.060384 7f16282a6780 1 mon.alpha@-1(probing) e0 init fsid b97addff-ffbd-4548-a694-506885186f16
2012-10-11 13:43:39.061415 7f16282a6780 0 mon.alpha@-1(probing) e0 my rank is now 0 (was -1)
2012-10-11 13:43:39.061441 7f16282a6780 1 mon.alpha@0(probing) e0 win_standalone_election
2012-10-11 13:43:39.125315 7f16282a6780 0 log [INF] : mon.alpha@0 won leader election with quorum 0
2012-10-11 13:43:39.260378 7f16282a6780 0 log [INF] : pgmap v1: 0 pgs: ; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:39.379529 7f16282a6780 0 log [INF] : mdsmap e1: 0/0/1 up
2012-10-11 13:43:39.498308 7f16282a6780 1 mon.alpha@0(leader).osd e1 e1: 0 osds: 0 up, 0 in
2012-10-11 13:43:39.669967 7f16282a6780 0 log [INF] : pgmap v2: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:39.680016 7f16282a6780 0 log [INF] : osdmap e1: 0 osds: 0 up, 0 in
2012-10-11 13:43:39.947110 7f16282a6780 1 mon.alpha@0(probing) e1 win_standalone_election
2012-10-11 13:43:39.972841 7f16282a6780 0 log [INF] : mon.alpha@0 won leader election with quorum 0
2012-10-11 13:43:39.972902 7f16282a6780 0 log [INF] : pgmap v2: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:39.972931 7f16282a6780 0 log [INF] : mdsmap e1: 0/0/1 up
2012-10-11 13:43:39.972958 7f16282a6780 0 log [INF] : osdmap e1: 0 osds: 0 up, 0 in
2012-10-11 13:43:40.006780 7f16282a6780 0 log [INF] : monmap e1: 1 mons at {alpha=149.165.228.10:6789/0}
2012-10-11 13:43:40.142348 7f16282a6780 0 log [INF] : monmap e1: 1 mons at {alpha=149.165.228.10:6789/0}
2012-10-11 13:43:40.731408 7f1622564700 1 mon.alpha@0(leader).mds e2 adding standby 149.165.228.10:6800/32146 as mds.0
2012-10-11 13:43:40.824610 7f1622564700 0 log [INF] : mdsmap e3: 1/1/1 up {0=alpha=up:creating}
2012-10-11 13:43:40.824654 7f1622564700 0 log [INF] : mdsmap e3: 1/1/1 up {0=alpha=up:creating}
2012-10-11 13:43:40.824667 7f1622564700 0 log [INF] : mds.? 149.165.228.10:6800/32146 up:boot
2012-10-11 13:43:42.195677 7f1622564700 1 mon.alpha@0(leader).osd e2 e2: 1 osds: 1 up, 1 in
2012-10-11 13:43:42.342732 7f1622564700 0 log [INF] : pgmap v3: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:42.353066 7f1622564700 0 log [INF] : osdmap e2: 1 osds: 1 up, 1 in
2012-10-11 13:43:42.353091 7f1622564700 0 log [INF] : osd.0 149.165.228.10:6801/338 boot
2012-10-11 13:43:43.234947 7f1621d63700 1 mon.alpha@0(leader).osd e3 e3: 1 osds: 1 up, 1 in
2012-10-11 13:43:43.369013 7f1621d63700 0 log [INF] : pgmap v4: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:43.376872 7f1621d63700 0 log [INF] : osdmap e3: 1 osds: 1 up, 1 in
2012-10-11 13:43:44.268380 7f1621d63700 1 mon.alpha@0(leader).osd e4 e4: 2 osds: 2 up, 2 in
2012-10-11 13:43:44.416405 7f1621d63700 0 log [INF] : pgmap v5: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:44.418457 7f1621d63700 0 log [INF] : osdmap e4: 2 osds: 2 up, 2 in
2012-10-11 13:43:44.418478 7f1621d63700 0 log [INF] : osd.1 149.165.228.10:6804/1098 boot
2012-10-11 13:43:45.308554 7f1621d63700 1 mon.alpha@0(leader).osd e5 e5: 3 osds: 3 up, 3 in
2012-10-11 13:43:45.454643 7f1621d63700 0 log [INF] : pgmap v6: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:45.456116 7f1621d63700 0 log [INF] : osdmap e5: 3 osds: 3 up, 3 in
2012-10-11 13:43:45.456131 7f1621d63700 0 log [INF] : osd.2 149.165.228.10:6807/1657 boot
2012-10-11 13:43:46.330300 7f1621d63700 1 mon.alpha@0(leader).osd e6 e6: 3 osds: 3 up, 3 in
2012-10-11 13:43:46.475810 7f1621d63700 0 log [INF] : pgmap v7: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:46.477496 7f1621d63700 0 log [INF] : osdmap e6: 3 osds: 3 up, 3 in
2012-10-11 13:43:47.369006 7f1621d63700 1 mon.alpha@0(leader).osd e7 e7: 3 osds: 3 up, 3 in
2012-10-11 13:43:47.506766 7f1621d63700 0 log [INF] : pgmap v8: 768 pgs: 768 creating; 0 bytes data, 0 KB used, 0 KB / 0 KB avail
2012-10-11 13:43:47.508312 7f1621d63700 0 log [INF] : osdmap e7: 3 osds: 3 up, 3 in
2012-10-11 13:43:48.702060 7f1621d63700 1 mon.alpha@0(leader).osd e8 e8: 3 osds: 3 up, 3 in
2012-10-11 13:43:48.844869 7f1621d63700 0 log [INF] : pgmap v9: 768 pgs: 441 creating, 42 active, 285 peering; 1202 bytes data, 1028 MB used, 2733 GB / 2734 GB avail
2012-10-11 13:43:48.845957 7f1621d63700 0 log [INF] : osdmap e8: 3 osds: 3 up, 3 in
2012-10-11 13:43:49.882398 7f1621d63700 0 log [INF] : pgmap v10: 768 pgs: 201 creating, 276 active, 288 peering, 3 active+recovering; 2210 bytes data, 2043 MB used, 5466 GB / 5468 GB avail; 4/24 degraded (16.667%)
2012-10-11 13:43:50.947237 7f1621d63700 0 log [INF] : pgmap v11: 768 pgs: 11 creating, 323 active, 127 active+clean, 304 peering, 3 active+recovering; 2126 bytes data, 3058 MB used, 8199 GB / 8202 GB avail; 4/22 degraded (18.182%)
2012-10-11 13:43:55.096086 7f1621d63700 0 log [INF] : pgmap v12: 768 pgs: 136 active, 325 active+clean, 301 peering, 6 active+recovering; 3066 bytes data, 3060 MB used, 8199 GB / 8202 GB avail; 11/30 degraded (36.667%)
2012-10-11 13:43:56.133017 7f1621d63700 0 log [INF] : pgmap v13: 768 pgs: 188 active, 338 active+clean, 232 peering, 10 active+recovering; 7854 bytes data, 3064 MB used, 8199 GB / 8202 GB avail; 18/40 degraded (45.000%)
2012-10-11 13:44:14.523821 7f1622564700 1 mon.alpha@0(leader).osd e8 prepare_failure osd.0 149.165.228.10:6801/338 from osd.1 149.165.228.10:6804/1098 is reporting failure:1
2012-10-11 13:44:14.523838 7f1622564700 0 log [INF] : osd.0 149.165.228.10:6801/338 failed (by osd.1 149.165.228.10:6804/1098)
2012-10-11 13:44:15.425977 7f1622564700 1 mon.alpha@0(leader).osd e8 prepare_failure osd.0 149.165.228.10:6801/338 from osd.2 149.165.228.10:6807/1657 is reporting failure:1
2012-10-11 13:44:15.425994 7f1622564700 0 log [INF] : osd.0 149.165.228.10:6801/338 failed (by osd.2 149.165.228.10:6807/1657)
2012-10-11 13:44:19.524573 7f1622564700 1 mon.alpha@0(leader).osd e8 prepare_failure osd.0 149.165.228.10:6801/338 from osd.1 149.165.228.10:6804/1098 is reporting failure:1
2012-10-11 13:44:19.524590 7f1622564700 0 log [INF] : osd.0 149.165.228.10:6801/338 failed (by osd.1 149.165.228.10:6804/1098)
2012-10-11 13:44:19.524608 7f1622564700 1 mon.alpha@0(leader).osd e8 have enough reports/reporters to mark osd.0 as down
2012-10-11 13:44:19.717283 7f1621d63700 1 mon.alpha@0(leader).osd e9 e9: 3 osds: 2 up, 3 in
2012-10-11 13:44:19.878512 7f1621d63700 0 log [INF] : pgmap v14: 768 pgs: 146 active, 338 active+clean, 42 stale+active, 1 peering, 231 stale+peering, 10 active+recovering; 7854 bytes data, 3064 MB used, 8199 GB / 8202 GB avail; 18/40 degraded (45.000%)
2012-10-11 13:44:19.878699 7f1621d63700 0 log [INF] : osdmap e9: 3 osds: 2 up, 3 in
2012-10-11 13:44:20.756504 7f1621d63700 1 mon.alpha@0(leader).osd e10 e10: 3 osds: 2 up, 3 in
2012-10-11 13:44:20.909573 7f1621d63700 0 log [INF] : pgmap v15: 768 pgs: 146 active, 338 active+clean, 42 stale+active, 1 peering, 231 stale+peering, 10 active+recovering; 7854 bytes data, 3064 MB used, 8199 GB / 8202 GB avail; 18/40 degraded (45.000%)
2012-10-11 13:44:20.909812 7f1621d63700 0 log [INF] : osdmap e10: 3 osds: 2 up, 3 in
2012-10-11 13:44:25.636753 7f1621d63700 0 log [INF] : pgmap v16: 768 pgs: 47 active, 338 active+clean, 42 stale+active, 188 down+peering, 146 stale+peering, 7 active+recovering; 7854 bytes data, 3064 MB used, 8199 GB / 8202 GB avail; 14/40 degraded (35.000%); 1/20 unfound (5.000%)
2012-10-11 13:44:26.690576 7f1621d63700 0 log [INF] : pgmap v17: 768 pgs: 270 active+clean, 294 down+peering, 145 active+degraded, 56 stale+peering, 3 active+recovering; 7854 bytes data, 3068 MB used, 8199 GB / 8202 GB avail; 9/40 degraded (22.500%); 3/20 unfound (15.000%)
2012-10-11 13:44:31.882912 7f1622564700 0 mon.alpha@0(leader) e1 handle_command mon_command(status v 0) v1
2012-10-11 13:46:25.616888 7f1621d63700 0 log [INF] : pgmap v18: 768 pgs: 270 active+clean, 294 down+peering, 145 active+degraded, 56 stale+peering, 3 active+recovering; 7854 bytes data, 3068 MB used, 8199 GB / 8202 GB avail; 9/40 degraded (22.500%); 3/20 unfound (15.000%)
2012-10-11 13:46:26.670374 7f1621d63700 0 log [INF] : pgmap v19: 768 pgs: 270 active+clean, 294 down+peering, 145 active+degraded, 56 stale+peering, 3 active+recovering; 7854 bytes data, 3068 MB used, 8199 GB / 8202 GB avail; 9/40 degraded (22.500%); 3/20 unfound (15.000%)
(4-4/5)