|
2018-07-25 11:16:51.075810 mon.mimir mon.2 [fda8:941:2491:1699:60fa:e622:8345:2162]:6789/0 120835 : cluster [INF] mon.mimir calling monitor election
|
|
2018-07-25 11:16:51.096540 mon.fenrir mon.1 [fda8:941:2491:1699:b45:a2e6:1383:2b98]:6789/0 5 : cluster [INF] mon.fenrir calling monitor election
|
|
2018-07-25 11:16:56.149598 mon.fenrir mon.1 [fda8:941:2491:1699:b45:a2e6:1383:2b98]:6789/0 6 : cluster [INF] mon.fenrir is new leader, mons fenrir,mimir in quorum (ranks 1,2)
|
|
2018-07-25 11:16:56.175763 mon.fenrir mon.1 [fda8:941:2491:1699:b45:a2e6:1383:2b98]:6789/0 11 : cluster [WRN] Health check failed: 1/3 mons down, quorum fenrir,mimir (MON_DOWN)
|
|
2018-07-25 11:16:56.185470 mon.fenrir mon.1 [fda8:941:2491:1699:b45:a2e6:1383:2b98]:6789/0 12 : cluster [WRN] MDS health message (mds.0): Client mimir failing to respond to cache pressure
|
|
2018-07-25 11:16:56.207259 mon.fenrir mon.1 [fda8:941:2491:1699:b45:a2e6:1383:2b98]:6789/0 13 : cluster [WRN] overall HEALTH_WARN 1 clients failing to respond to cache pressure; 1/3 mons down, quorum fenrir,mimir
|
|
2018-07-25 11:16:58.370472 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19003 : cluster [INF] mon.hoenir calling monitor election
|
|
2018-07-25 11:16:58.432356 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19004 : cluster [INF] mon.hoenir is new leader, mons hoenir,fenrir,mimir in quorum (ranks 0,1,2)
|
|
2018-07-25 11:16:58.532884 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19009 : cluster [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum fenrir,mimir)
|
|
2018-07-25 11:16:58.639711 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19010 : cluster [WRN] overall HEALTH_WARN 1 clients failing to respond to cache pressure
|
|
2018-07-25 11:18:02.010555 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19028 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY)
|
|
2018-07-25 11:18:05.150792 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19030 : cluster [WRN] Health check failed: 1673/12160226 objects misplaced (0.014%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:07.156182 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19031 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 4 pgs peering)
|
|
2018-07-25 11:18:11.232557 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19032 : cluster [WRN] Health check update: 71484/12160284 objects misplaced (0.588%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:17.179004 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19033 : cluster [WRN] Health check update: 71335/12160320 objects misplaced (0.587%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:23.187805 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19034 : cluster [WRN] Health check update: 71208/12160360 objects misplaced (0.586%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:28.363306 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19037 : cluster [WRN] Health check update: 71088/12160394 objects misplaced (0.585%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:33.365755 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19038 : cluster [WRN] Health check update: 70964/12160434 objects misplaced (0.584%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:38.366553 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19039 : cluster [WRN] Health check update: 70933/12160450 objects misplaced (0.583%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:43.367585 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19040 : cluster [WRN] Health check update: 70748/12160494 objects misplaced (0.582%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:48.368880 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19041 : cluster [WRN] Health check update: 70595/12160554 objects misplaced (0.581%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:53.369904 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19044 : cluster [WRN] Health check update: 70464/12160592 objects misplaced (0.579%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:18:58.371075 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19045 : cluster [WRN] Health check update: 70332/12160616 objects misplaced (0.578%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:03.889088 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19046 : cluster [WRN] Health check update: 70207/12160642 objects misplaced (0.577%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:11.333058 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19047 : cluster [WRN] Health check update: 69994/12160710 objects misplaced (0.576%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:17.378865 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19050 : cluster [WRN] Health check update: 69843/12160766 objects misplaced (0.574%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:23.309746 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19051 : cluster [WRN] Health check update: 69722/12160794 objects misplaced (0.573%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:31.753229 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19052 : cluster [WRN] Health check update: 69502/12160862 objects misplaced (0.572%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:37.332275 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19053 : cluster [WRN] Health check update: 69350/12160902 objects misplaced (0.570%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:43.419402 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19056 : cluster [WRN] Health check update: 69255/12160930 objects misplaced (0.569%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:48.434747 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19057 : cluster [WRN] Health check update: 69100/12160984 objects misplaced (0.568%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:19:55.061323 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19058 : cluster [WRN] Health check update: 68977/12161020 objects misplaced (0.567%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:01.402503 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19059 : cluster [WRN] Health check update: 68763/12161090 objects misplaced (0.565%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:07.846723 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19060 : cluster [WRN] Health check update: 68626/12161132 objects misplaced (0.564%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:14.043534 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19063 : cluster [WRN] Health check update: 68543/12161162 objects misplaced (0.564%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:19.447280 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19064 : cluster [WRN] Health check update: 68515/12161178 objects misplaced (0.563%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:24.694984 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19065 : cluster [WRN] Health check update: 68310/12161234 objects misplaced (0.562%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:29.754774 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19066 : cluster [WRN] Health check update: 68197/12161261 objects misplaced (0.561%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:34.800401 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19067 : cluster [WRN] Health check update: 68081/12161291 objects misplaced (0.560%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:39.903271 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19068 : cluster [WRN] Health check update: 67965/12161331 objects misplaced (0.559%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:44.983384 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19069 : cluster [WRN] Health check update: 67841/12161365 objects misplaced (0.558%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:20:52.748693 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19070 : cluster [WRN] Health check update: 67660/12161405 objects misplaced (0.556%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:21:00.146390 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19071 : cluster [WRN] Health check update: 67497/12161455 objects misplaced (0.555%) (OBJECT_MISPLACED)
|
|
2018-07-25 11:21:05.219464 mon.hoenir mon.0 [fda8:941:2491:1699:b3b:5c9b:ccb6:3024]:6789/0 19072 : cluster [WRN] Health check update: 67372/12161495 objects misplaced (0.554%) (OBJECT_MISPLACED)
|