Project

General

Profile

Bug #5084 » ceph.log

Faidon Liambotis, 05/16/2013 11:26 AM

 
2013-05-16 18:17:23.183835 mon.0 10.64.0.167:6789/0 456532 : [INF] pgmap v7224934: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 44644B/s wr, 5op/s
2013-05-16 18:17:24.482454 mon.0 10.64.0.167:6789/0 456533 : [INF] pgmap v7224935: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 65085B/s wr, 15op/s
2013-05-16 18:17:26.375477 mon.0 10.64.0.167:6789/0 456534 : [INF] pgmap v7224936: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1508B/s wr, 8op/s
2013-05-16 18:17:27.663376 mon.0 10.64.0.167:6789/0 456535 : [INF] pgmap v7224937: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 5725KB/s wr, 33op/s
2013-05-16 18:17:29.503594 mon.0 10.64.0.167:6789/0 456536 : [INF] pgmap v7224938: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail
2013-05-16 18:17:30.750172 mon.0 10.64.0.167:6789/0 456537 : [INF] pgmap v7224939: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 199KB/s wr, 48op/s
2013-05-16 18:17:32.683935 mon.0 10.64.0.167:6789/0 456538 : [INF] pgmap v7224940: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 532KB/s wr, 17op/s
2013-05-16 18:17:33.938597 mon.0 10.64.0.167:6789/0 456539 : [INF] pgmap v7224941: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 103KB/s wr, 30op/s
2013-05-16 18:17:35.813620 mon.0 10.64.0.167:6789/0 456540 : [INF] pgmap v7224942: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 100KB/s wr, 16op/s
2013-05-16 18:17:37.018934 mon.0 10.64.0.167:6789/0 456541 : [INF] pgmap v7224943: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 816KB/s wr, 66op/s
2013-05-16 18:17:39.156243 mon.0 10.64.0.167:6789/0 456542 : [INF] pgmap v7224944: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 27458B/s wr, 28op/s
2013-05-16 18:17:40.361214 mon.0 10.64.0.167:6789/0 456543 : [INF] pgmap v7224945: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 5923B/s wr, 31op/s
2013-05-16 18:17:42.351600 mon.0 10.64.0.167:6789/0 456544 : [INF] pgmap v7224946: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 95750B/s wr, 24op/s
2013-05-16 18:17:43.603848 mon.0 10.64.0.167:6789/0 456545 : [INF] pgmap v7224947: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 412KB/s wr, 69op/s
2013-05-16 18:17:45.516720 mon.0 10.64.0.167:6789/0 456546 : [INF] pgmap v7224948: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3827B/s wr, 11op/s
2013-05-16 18:17:45.869639 mon.0 10.64.0.167:6789/0 456547 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.135 10.64.0.178:6827/3796
2013-05-16 18:17:45.869740 mon.0 10.64.0.167:6789/0 456548 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.131 10.64.32.16:6854/1276
2013-05-16 18:17:45.931463 mon.0 10.64.0.167:6789/0 456549 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.65 10.64.32.11:6838/9550
2013-05-16 18:17:46.304759 mon.0 10.64.0.167:6789/0 456550 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.24 10.64.0.175:6810/8333
2013-05-16 18:17:46.838388 mon.0 10.64.0.167:6789/0 456551 : [INF] pgmap v7224949: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 373KB/s wr, 37op/s
2013-05-16 18:17:47.057457 mon.0 10.64.0.167:6789/0 456552 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.72 10.64.32.12:6801/3348
2013-05-16 18:17:47.422346 mon.0 10.64.0.167:6789/0 456553 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.52 10.64.32.10:6840/17776
2013-05-16 18:17:47.458768 mon.0 10.64.0.167:6789/0 456554 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.67 10.64.32.11:6846/11672
2013-05-16 18:17:47.561745 mon.0 10.64.0.167:6789/0 456555 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.102 10.64.32.14:6800/24561
2013-05-16 18:17:47.756620 mon.0 10.64.0.167:6789/0 456556 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.71 10.64.32.11:6862/12441
2013-05-16 18:17:48.029370 mon.0 10.64.0.167:6789/0 456557 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.31 10.64.0.175:6857/25576
2013-05-16 18:17:48.123981 mon.0 10.64.0.167:6789/0 456558 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.113 10.64.32.15:6844/28543
2013-05-16 18:17:48.170036 mon.0 10.64.0.167:6789/0 456559 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.54 10.64.32.10:6848/17827
2013-05-16 18:17:48.188081 mon.0 10.64.0.167:6789/0 456560 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.143 10.64.0.178:6853/3994
2013-05-16 18:17:48.728337 mon.0 10.64.0.167:6789/0 456561 : [INF] pgmap v7224950: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 405KB/s wr, 10op/s
2013-05-16 18:17:49.133983 mon.0 10.64.0.167:6789/0 456562 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.35 10.64.0.175:6869/27247
2013-05-16 18:17:49.134031 mon.0 10.64.0.167:6789/0 456563 : [INF] osd.0 10.64.0.173:6807/13276 failed (14 reports from 14 peers after 2013-05-16 18:18:09.133969 >= grace 20.000000)
2013-05-16 18:17:49.134234 mon.0 10.64.0.167:6789/0 456564 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.48 10.64.32.10:6800/17615
2013-05-16 18:17:49.134319 mon.0 10.64.0.167:6789/0 456565 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.117 10.64.32.15:6860/28820
2013-05-16 18:17:49.134434 mon.0 10.64.0.167:6789/0 456566 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.112 10.64.32.15:6838/28519
2013-05-16 18:17:49.134549 mon.0 10.64.0.167:6789/0 456567 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.42 10.64.0.176:6840/22347
2013-05-16 18:17:49.134630 mon.0 10.64.0.167:6789/0 456568 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.77 10.64.32.12:6845/3604
2013-05-16 18:17:49.134714 mon.0 10.64.0.167:6789/0 456569 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.137 10.64.0.178:6835/3868
2013-05-16 18:17:49.134860 mon.0 10.64.0.167:6789/0 456570 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.58 10.64.32.10:6865/18149
2013-05-16 18:17:49.135339 mon.0 10.64.0.167:6789/0 456571 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.119 10.64.32.15:6869/30101
2013-05-16 18:17:49.135450 mon.0 10.64.0.167:6789/0 456572 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.101 10.64.32.14:6818/2810
2013-05-16 18:17:49.135642 mon.0 10.64.0.167:6789/0 456573 : [DBG] osd.0 10.64.0.173:6807/13276 reported failed by osd.44 10.64.0.176:6847/22395
2013-05-16 18:17:50.445028 mon.0 10.64.0.167:6789/0 456574 : [INF] pgmap v7224951: 16760 pgs: 16752 active+clean, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2831KB/s wr, 39op/s
2013-05-16 18:17:51.111626 mon.0 10.64.0.167:6789/0 456575 : [INF] osdmap e180322: 144 osds: 142 up, 143 in
2013-05-16 18:17:52.795649 mon.0 10.64.0.167:6789/0 456576 : [INF] pgmap v7224952: 16760 pgs: 16751 active+clean, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1041KB/s wr, 67op/s
2013-05-16 18:17:53.342993 mon.0 10.64.0.167:6789/0 456577 : [INF] osdmap e180323: 144 osds: 142 up, 143 in
2013-05-16 18:17:55.021177 mon.0 10.64.0.167:6789/0 456578 : [INF] pgmap v7224953: 16760 pgs: 16744 active+clean, 7 peering, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3734KB/s wr, 43op/s
2013-05-16 18:17:55.726707 mon.0 10.64.0.167:6789/0 456579 : [INF] osdmap e180324: 144 osds: 142 up, 143 in
2013-05-16 18:17:57.345823 mon.0 10.64.0.167:6789/0 456580 : [INF] pgmap v7224954: 16760 pgs: 16722 active+clean, 29 peering, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 438KB/s wr, 3op/s
2013-05-16 18:17:57.960474 mon.0 10.64.0.167:6789/0 456581 : [INF] osdmap e180325: 144 osds: 142 up, 143 in
2013-05-16 18:17:59.664497 mon.0 10.64.0.167:6789/0 456582 : [INF] pgmap v7224955: 16760 pgs: 16704 active+clean, 47 peering, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 0B/s wr, 4op/s
2013-05-16 18:18:00.444411 mon.0 10.64.0.167:6789/0 456583 : [INF] osdmap e180326: 144 osds: 143 up, 143 in
2013-05-16 18:18:00.446307 mon.0 10.64.0.167:6789/0 456584 : [INF] osd.0 10.64.0.173:6822/19826 boot
2013-05-16 18:18:02.283393 mon.0 10.64.0.167:6789/0 456585 : [INF] pgmap v7224956: 16760 pgs: 16580 active+clean, 143 peering, 28 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1330B/s wr, 3op/s; 442926/806455712 degraded (0.055%)
2013-05-16 18:18:03.203951 mon.0 10.64.0.167:6789/0 456586 : [INF] osdmap e180327: 144 osds: 143 up, 143 in
2013-05-16 18:18:01.887877 osd.38 10.64.0.176:6831/22256 1580 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.250386 secs
2013-05-16 18:18:01.887934 osd.38 10.64.0.176:6831/22256 1581 : [WRN] slow request 30.250386 seconds old, received at 2013-05-16 18:17:31.637450: osd_op(client.27297.0:50128581 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently reached pg
2013-05-16 18:18:04.935214 mon.0 10.64.0.167:6789/0 456587 : [INF] pgmap v7224957: 16760 pgs: 16463 active+clean, 229 peering, 59 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 951095/806455712 degraded (0.118%)
2013-05-16 18:18:05.854149 mon.0 10.64.0.167:6789/0 456588 : [INF] osdmap e180328: 144 osds: 143 up, 143 in
2013-05-16 18:18:07.493089 mon.0 10.64.0.167:6789/0 456589 : [INF] pgmap v7224958: 16760 pgs: 16420 active+clean, 262 peering, 70 active+degraded, 8 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 160KB/s wr, 0op/s; 1131204/806455715 degraded (0.140%)
2013-05-16 18:18:08.506091 mon.0 10.64.0.167:6789/0 456590 : [INF] osdmap e180329: 144 osds: 143 up, 143 in
2013-05-16 18:18:10.161182 mon.0 10.64.0.167:6789/0 456591 : [INF] pgmap v7224959: 16760 pgs: 11 active, 16415 active+clean, 247 peering, 78 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1246030/806455715 degraded (0.155%)
2013-05-16 18:18:11.076930 mon.0 10.64.0.167:6789/0 456592 : [INF] osdmap e180330: 144 osds: 143 up, 143 in
2013-05-16 18:18:13.074499 mon.0 10.64.0.167:6789/0 456593 : [INF] pgmap v7224960: 16760 pgs: 51 active, 16409 active+clean, 219 peering, 72 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 22123B/s wr, 5op/s; 1148132/806455721 degraded (0.142%)
2013-05-16 18:18:13.990737 mon.0 10.64.0.167:6789/0 456594 : [INF] osdmap e180331: 144 osds: 143 up, 143 in
2013-05-16 18:18:15.592408 mon.0 10.64.0.167:6789/0 456595 : [INF] pgmap v7224961: 16760 pgs: 116 active, 16402 active+clean, 182 peering, 51 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 44707B/s wr, 12op/s; 803374/806455733 degraded (0.100%)
2013-05-16 18:18:16.964958 mon.0 10.64.0.167:6789/0 456596 : [INF] pgmap v7224962: 16760 pgs: 116 active, 16402 active+clean, 182 peering, 51 active+degraded, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 803374/806455733 degraded (0.100%)
2013-05-16 18:18:18.835676 mon.0 10.64.0.167:6789/0 456597 : [INF] pgmap v7224963: 16760 pgs: 223 active, 16398 active+clean, 130 peering, 9 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 866KB/s wr, 56op/s
2013-05-16 18:18:20.965188 mon.0 10.64.0.167:6789/0 456598 : [INF] pgmap v7224964: 16760 pgs: 244 active, 16397 active+clean, 109 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 331KB/s wr, 21op/s
2013-05-16 18:18:22.553644 mon.0 10.64.0.167:6789/0 456599 : [INF] pgmap v7224965: 16760 pgs: 255 active, 16397 active+clean, 98 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 1066KB/s wr, 45op/s
2013-05-16 18:18:24.009484 mon.0 10.64.0.167:6789/0 456600 : [INF] pgmap v7224966: 16760 pgs: 267 active, 16397 active+clean, 86 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 6459KB/s wr, 112op/s
2013-05-16 18:18:25.689352 mon.0 10.64.0.167:6789/0 456601 : [INF] pgmap v7224967: 16760 pgs: 270 active, 16397 active+clean, 83 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 612KB/s wr, 58op/s
2013-05-16 18:18:27.046053 mon.0 10.64.0.167:6789/0 456602 : [INF] pgmap v7224968: 16760 pgs: 271 active, 16397 active+clean, 82 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 399KB/s wr, 45op/s
2013-05-16 18:18:28.593797 mon.0 10.64.0.167:6789/0 456603 : [INF] pgmap v7224969: 16760 pgs: 272 active, 16397 active+clean, 81 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4524B/s wr, 8op/s
2013-05-16 18:18:29.792217 mon.0 10.64.0.167:6789/0 456604 : [INF] pgmap v7224970: 16760 pgs: 281 active, 16397 active+clean, 72 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 44376B/s wr, 28op/s
2013-05-16 18:18:31.240252 mon.0 10.64.0.167:6789/0 456605 : [INF] pgmap v7224971: 16760 pgs: 281 active, 16397 active+clean, 72 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 77098B/s wr, 22op/s; 2/806455982 degraded (0.000%)
2013-05-16 18:18:32.305320 mon.0 10.64.0.167:6789/0 456606 : [INF] pgmap v7224972: 16760 pgs: 281 active, 16397 active+clean, 72 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 17796B/s wr, 20op/s; 2/806455991 degraded (0.000%)
2013-05-16 18:18:33.778228 mon.0 10.64.0.167:6789/0 456607 : [INF] pgmap v7224973: 16760 pgs: 281 active, 16397 active+clean, 72 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 24786B/s wr, 15op/s; 2/806455997 degraded (0.000%)
2013-05-16 18:18:35.101227 mon.0 10.64.0.167:6789/0 456608 : [INF] pgmap v7224974: 16760 pgs: 281 active, 16397 active+clean, 72 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 408KB/s wr, 8op/s; 2/806456003 degraded (0.000%)
2013-05-16 18:18:32.447202 osd.38 10.64.0.176:6831/22256 1582 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.809696 secs
2013-05-16 18:18:32.447211 osd.38 10.64.0.176:6831/22256 1583 : [WRN] slow request 60.809696 seconds old, received at 2013-05-16 18:17:31.637450: osd_op(client.27297.0:50128581 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently reached pg
2013-05-16 18:18:36.699530 mon.0 10.64.0.167:6789/0 456609 : [INF] pgmap v7224975: 16760 pgs: 292 active, 16397 active+clean, 61 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3182KB/s wr, 21op/s; 2/806456015 degraded (0.000%)
2013-05-16 18:18:38.056449 mon.0 10.64.0.167:6789/0 456610 : [INF] pgmap v7224976: 16760 pgs: 292 active, 16397 active+clean, 61 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 639KB/s wr, 16op/s; 2/806456024 degraded (0.000%)
2013-05-16 18:18:31.159119 osd.0 10.64.0.173:6822/19826 1 : [WRN] 2 slow requests, 2 included below; oldest blocked for > 30.483520 secs
2013-05-16 18:18:31.159127 osd.0 10.64.0.173:6822/19826 2 : [WRN] slow request 30.483520 seconds old, received at 2013-05-16 18:18:00.675520: osd_op(client.27495.0:5293708 21406.1730_840e203401db4651ca004b8a69f87bb5.map [getxattrs,stat,read 0~524288] 3.61c06ff RETRY) v4 currently reached pg
2013-05-16 18:18:31.159137 osd.0 10.64.0.173:6822/19826 3 : [WRN] slow request 30.483437 seconds old, received at 2013-05-16 18:18:00.675603: osd_op(client.27495.0:5293813 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712 RETRY) v4 currently reached pg
2013-05-16 18:18:32.159361 osd.0 10.64.0.173:6822/19826 4 : [WRN] 5 slow requests, 3 included below; oldest blocked for > 31.483764 secs
2013-05-16 18:18:32.159371 osd.0 10.64.0.173:6822/19826 5 : [WRN] slow request 30.914339 seconds old, received at 2013-05-16 18:18:01.244945: osd_op(client.27152.0:11789081 4465.12__shadow__FKwBY42rmiouowW2P86x8aciYYL8lJC_1 [read 0~524288] 3.48f0c29f RETRY) v4 currently reached pg
2013-05-16 18:18:32.159378 osd.0 10.64.0.173:6822/19826 6 : [WRN] slow request 30.851448 seconds old, received at 2013-05-16 18:18:01.307836: osd_op(client.27297.0:50128650 21406.4154_fa0f71d4f31f46823f6fa811cccc70f2.map [getxattrs,stat,read 0~524288] 3.4fb02a80 RETRY) v4 currently reached pg
2013-05-16 18:18:32.159387 osd.0 10.64.0.173:6822/19826 7 : [WRN] slow request 30.851296 seconds old, received at 2013-05-16 18:18:01.307988: osd_op(client.27297.0:50128744 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8 RETRY) v4 currently reached pg
2013-05-16 18:18:33.159846 osd.0 10.64.0.173:6822/19826 8 : [WRN] 7 slow requests, 2 included below; oldest blocked for > 32.484265 secs
2013-05-16 18:18:33.159857 osd.0 10.64.0.173:6822/19826 9 : [WRN] slow request 30.293001 seconds old, received at 2013-05-16 18:18:02.866784: osd_op(client.27495.0:5294566 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently reached pg
2013-05-16 18:18:33.159863 osd.0 10.64.0.173:6822/19826 10 : [WRN] slow request 30.153631 seconds old, received at 2013-05-16 18:18:03.006154: osd_op(client.25068.0:11834698 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently reached pg
2013-05-16 18:18:35.160188 osd.0 10.64.0.173:6822/19826 11 : [WRN] 8 slow requests, 1 included below; oldest blocked for > 34.484619 secs
2013-05-16 18:18:35.160194 osd.0 10.64.0.173:6822/19826 12 : [WRN] slow request 30.733095 seconds old, received at 2013-05-16 18:18:04.427044: osd_op(client.27297.0:50129588 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently reached pg
2013-05-16 18:18:38.160861 osd.0 10.64.0.173:6822/19826 13 : [WRN] 9 slow requests, 1 included below; oldest blocked for > 37.485293 secs
2013-05-16 18:18:38.160872 osd.0 10.64.0.173:6822/19826 14 : [WRN] slow request 30.643246 seconds old, received at 2013-05-16 18:18:07.517567: osd_op(client.27152.0:11789626 21406.4154_fa0f71d4f31f46823f6fa811cccc70f2.map [getxattrs,stat,read 0~524288] 3.4fb02a80) v4 currently no flag points reached
2013-05-16 18:18:39.161169 osd.0 10.64.0.173:6822/19826 15 : [WRN] 12 slow requests, 3 included below; oldest blocked for > 38.485574 secs
2013-05-16 18:18:39.161184 osd.0 10.64.0.173:6822/19826 16 : [WRN] slow request 30.269819 seconds old, received at 2013-05-16 18:18:08.891275: osd_op(client.25068.0:11834840 6900.1942_7/3/9/image_fcbc2674_7397949ca06e5c96.png [getxattrs,stat,read 0~524288] 3.224e627a) v4 currently wait for new map
2013-05-16 18:18:39.161191 osd.0 10.64.0.173:6822/19826 17 : [WRN] slow request 30.295716 seconds old, received at 2013-05-16 18:18:08.865378: osd_op(client.27495.0:5294688 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently wait for new map
2013-05-16 18:18:39.161199 osd.0 10.64.0.173:6822/19826 18 : [WRN] slow request 30.166780 seconds old, received at 2013-05-16 18:18:08.994314: osd_op(client.25068.0:11834841 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently wait for new map
2013-05-16 18:18:39.450859 mon.0 10.64.0.167:6789/0 456611 : [INF] pgmap v7224977: 16760 pgs: 292 active, 16397 active+clean, 61 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 166KB/s wr, 35op/s; 2/806456042 degraded (0.000%)
2013-05-16 18:18:40.877969 mon.0 10.64.0.167:6789/0 456612 : [INF] pgmap v7224978: 16760 pgs: 298 active, 16397 active+clean, 55 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1456B/s wr, 8op/s; 2/806456045 degraded (0.000%)
2013-05-16 18:18:42.388218 mon.0 10.64.0.167:6789/0 456613 : [INF] pgmap v7224979: 16760 pgs: 298 active, 16397 active+clean, 55 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 102KB/s wr, 19op/s; 2/806456057 degraded (0.000%)
2013-05-16 18:18:43.927580 mon.0 10.64.0.167:6789/0 456614 : [INF] pgmap v7224980: 16760 pgs: 296 active, 16399 active+clean, 55 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 169KB/s wr, 13op/s; 2/806456066 degraded (0.000%)
2013-05-16 18:18:45.563883 mon.0 10.64.0.167:6789/0 456615 : [INF] pgmap v7224981: 16760 pgs: 289 active, 16406 active+clean, 55 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 29253B/s wr, 18op/s; 2/806456078 degraded (0.000%)
2013-05-16 18:18:46.995220 mon.0 10.64.0.167:6789/0 456616 : [INF] pgmap v7224982: 16760 pgs: 319 active, 16413 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 917KB/s wr, 46op/s; 2/806456108 degraded (0.000%)
2013-05-16 18:18:48.976384 mon.0 10.64.0.167:6789/0 456617 : [INF] pgmap v7224983: 16760 pgs: 312 active, 16420 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 373KB/s wr, 14op/s; 2/806456117 degraded (0.000%)
2013-05-16 18:18:46.162483 osd.0 10.64.0.173:6822/19826 19 : [WRN] 10 slow requests, 1 included below; oldest blocked for > 45.486907 secs
2013-05-16 18:18:46.162495 osd.0 10.64.0.173:6822/19826 20 : [WRN] slow request 30.353871 seconds old, received at 2013-05-16 18:18:15.808556: osd_op(client.27495.0:5294799 18499.46_3/39/Phantasm_II.jpg/293px-Phantasm_II.jpg [getxattrs,stat,read 0~524288] 3.e61713d6) v4 currently no flag points reached
2013-05-16 18:18:50.416063 mon.0 10.64.0.167:6789/0 456618 : [INF] pgmap v7224984: 16760 pgs: 304 active, 16428 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 132KB/s wr, 17op/s; 2/806456132 degraded (0.000%)
2013-05-16 18:18:51.965581 mon.0 10.64.0.167:6789/0 456619 : [INF] pgmap v7224985: 16760 pgs: 302 active, 16430 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 345KB/s wr, 24op/s; 2/806456147 degraded (0.000%)
2013-05-16 18:18:53.179648 mon.0 10.64.0.167:6789/0 456620 : [INF] pgmap v7224986: 16760 pgs: 301 active, 16431 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 141KB/s wr, 13op/s; 2/806456156 degraded (0.000%)
2013-05-16 18:18:49.593678 osd.56 10.64.32.10:6857/17945 1781 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.815016 secs
2013-05-16 18:18:49.593702 osd.56 10.64.32.10:6857/17945 1782 : [WRN] slow request 30.815016 seconds old, received at 2013-05-16 18:18:18.778569: osd_op(client.27152.0:11789995 10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png [create 0~0,delete,writefull 0~37036,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (42),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.8552d551) v4 currently waiting for subops from [37,0]
2013-05-16 18:18:51.812529 osd.50 10.64.32.10:6833/17728 1959 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.594860 secs
2013-05-16 18:18:51.812565 osd.50 10.64.32.10:6833/17728 1960 : [WRN] slow request 30.594860 seconds old, received at 2013-05-16 18:18:21.217583: osd_op(client.27297.0:50130173 10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg [create 0~0,delete,writefull 0~3329,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (97),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.abdaea43) v4 currently waiting for subops from [0,100]
2013-05-16 18:18:54.810490 mon.0 10.64.0.167:6789/0 456621 : [INF] pgmap v7224987: 16760 pgs: 298 active, 16434 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4368KB/s wr, 32op/s; 2/806456174 degraded (0.000%)
2013-05-16 18:18:52.451480 osd.38 10.64.0.176:6831/22256 1584 : [WRN] 3 slow requests, 2 included below; oldest blocked for > 80.813967 secs
2013-05-16 18:18:52.451488 osd.38 10.64.0.176:6831/22256 1585 : [WRN] slow request 30.724024 seconds old, received at 2013-05-16 18:18:21.727393: osd_op(client.27297.0:50130229 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat,read 0~524288] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:18:52.451528 osd.38 10.64.0.176:6831/22256 1586 : [WRN] slow request 30.723535 seconds old, received at 2013-05-16 18:18:21.727882: osd_op(client.25068.0:11835229 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat,read 0~524288] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:18:56.241468 mon.0 10.64.0.167:6789/0 456622 : [INF] pgmap v7224988: 16760 pgs: 296 active, 16436 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 90788B/s wr, 8op/s; 2/806456180 degraded (0.000%)
2013-05-16 18:18:57.814349 mon.0 10.64.0.167:6789/0 456623 : [INF] pgmap v7224989: 16760 pgs: 295 active, 16437 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 41726B/s wr, 18op/s; 2/806456192 degraded (0.000%)
2013-05-16 18:18:59.262419 mon.0 10.64.0.167:6789/0 456624 : [INF] pgmap v7224990: 16760 pgs: 294 active, 16438 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 295KB/s wr, 39op/s; 2/806456219 degraded (0.000%)
2013-05-16 18:19:00.760559 mon.0 10.64.0.167:6789/0 456625 : [INF] pgmap v7224991: 16760 pgs: 292 active, 16440 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 318KB/s wr, 34op/s; 2/806456240 degraded (0.000%)
2013-05-16 18:19:01.975288 mon.0 10.64.0.167:6789/0 456626 : [INF] pgmap v7224992: 16760 pgs: 290 active, 16442 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 241KB/s wr, 18op/s; 2/806456252 degraded (0.000%)
2013-05-16 18:19:03.533027 mon.0 10.64.0.167:6789/0 456627 : [INF] pgmap v7224993: 16760 pgs: 287 active, 16445 active+clean, 18 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 96256B/s wr, 17op/s; 2/806456261 degraded (0.000%)
2013-05-16 18:19:04.837297 osd.2 10.64.0.173:6819/13006 530 : [INF] 3.23f3 deep-scrub ok
2013-05-16 18:19:04.905951 mon.0 10.64.0.167:6789/0 456628 : [INF] pgmap v7224994: 16760 pgs: 291 active, 16451 active+clean, 8 peering, 10 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 29428B/s wr, 8op/s; 2/806456267 degraded (0.000%)
2013-05-16 18:19:06.478263 mon.0 10.64.0.167:6789/0 456629 : [INF] pgmap v7224995: 16760 pgs: 290 active, 16451 active+clean, 8 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 122KB/s wr, 5op/s; 2/806456270 degraded (0.000%)
2013-05-16 18:19:07.825891 mon.0 10.64.0.167:6789/0 456630 : [INF] pgmap v7224996: 16760 pgs: 288 active, 16453 active+clean, 8 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 15547B/s wr, 13op/s; 2/806456279 degraded (0.000%)
2013-05-16 18:19:01.514855 osd.0 10.64.0.173:6822/19826 21 : [WRN] 12 slow requests, 6 included below; oldest blocked for > 60.839246 secs
2013-05-16 18:19:01.514868 osd.0 10.64.0.173:6822/19826 22 : [WRN] slow request 60.839246 seconds old, received at 2013-05-16 18:18:00.675520: osd_op(client.27495.0:5293708 21406.1730_840e203401db4651ca004b8a69f87bb5.map [getxattrs,stat,read 0~524288] 3.61c06ff RETRY) v4 currently reached pg
2013-05-16 18:19:01.514875 osd.0 10.64.0.173:6822/19826 23 : [WRN] slow request 60.839163 seconds old, received at 2013-05-16 18:18:00.675603: osd_op(client.27495.0:5293813 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712 RETRY) v4 currently reached pg
2013-05-16 18:19:01.514882 osd.0 10.64.0.173:6822/19826 24 : [WRN] slow request 60.269821 seconds old, received at 2013-05-16 18:18:01.244945: osd_op(client.27152.0:11789081 4465.12__shadow__FKwBY42rmiouowW2P86x8aciYYL8lJC_1 [read 0~524288] 3.48f0c29f RETRY) v4 currently reached pg
2013-05-16 18:19:01.514887 osd.0 10.64.0.173:6822/19826 25 : [WRN] slow request 60.206778 seconds old, received at 2013-05-16 18:18:01.307988: osd_op(client.27297.0:50128744 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8 RETRY) v4 currently reached pg
2013-05-16 18:19:01.514895 osd.0 10.64.0.173:6822/19826 26 : [WRN] slow request 45.071794 seconds old, received at 2013-05-16 18:18:16.442972: osd_op(client.27152.0:11789852 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:02.515179 osd.0 10.64.0.173:6822/19826 27 : [WRN] 17 slow requests, 6 included below; oldest blocked for > 61.839539 secs
2013-05-16 18:19:02.515192 osd.0 10.64.0.173:6822/19826 28 : [WRN] slow request 45.688723 seconds old, received at 2013-05-16 18:18:16.826336: osd_op(client.25068.0:11835001 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:02.515201 osd.0 10.64.0.173:6822/19826 29 : [WRN] slow request 42.946706 seconds old, received at 2013-05-16 18:18:19.568353: osd_op(client.27297.0:50129950 21406.4154_69439ed9dd5cbc2402e9e2c8c62587b5.map [getxattrs,stat,read 0~524288] 3.3cc8f6fc) v4 currently no flag points reached
2013-05-16 18:19:02.515207 osd.0 10.64.0.173:6822/19826 30 : [WRN] slow request 39.611243 seconds old, received at 2013-05-16 18:18:22.903816: osd_op(client.27152.0:11790463 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:19:02.515212 osd.0 10.64.0.173:6822/19826 31 : [WRN] slow request 37.766860 seconds old, received at 2013-05-16 18:18:24.748199: osd_op(client.27152.0:11790516 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:02.515219 osd.0 10.64.0.173:6822/19826 32 : [WRN] slow request 41.296231 seconds old, received at 2013-05-16 18:18:21.218828: osd_sub_op(client.27297.0:50130173 3.2a43 abdaea43/10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/head//3 [] v 180331'22072 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:19:03.515465 osd.0 10.64.0.173:6822/19826 33 : [WRN] 21 slow requests, 6 included below; oldest blocked for > 62.839836 secs
2013-05-16 18:19:03.515476 osd.0 10.64.0.173:6822/19826 34 : [WRN] slow request 60.509202 seconds old, received at 2013-05-16 18:18:03.006154: osd_op(client.25068.0:11834698 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently reached pg
2013-05-16 18:19:03.515483 osd.0 10.64.0.173:6822/19826 35 : [WRN] slow request 44.734512 seconds old, received at 2013-05-16 18:18:18.780844: osd_sub_op(client.27152.0:11789995 3.1551 8552d551/10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png/head//3 [] v 180331'19700 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:19:03.515490 osd.0 10.64.0.173:6822/19826 36 : [WRN] slow request 36.288975 seconds old, received at 2013-05-16 18:18:27.226381: osd_op(client.25068.0:11835632 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:03.515495 osd.0 10.64.0.173:6822/19826 37 : [WRN] slow request 35.849865 seconds old, received at 2013-05-16 18:18:27.665491: osd_op(client.25068.0:11835683 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:03.515502 osd.0 10.64.0.173:6822/19826 38 : [WRN] slow request 35.528957 seconds old, received at 2013-05-16 18:18:27.986399: osd_op(client.27297.0:50130639 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:04.515749 osd.0 10.64.0.173:6822/19826 39 : [WRN] 25 slow requests, 6 included below; oldest blocked for > 63.840130 secs
2013-05-16 18:19:04.515761 osd.0 10.64.0.173:6822/19826 40 : [WRN] slow request 60.088606 seconds old, received at 2013-05-16 18:18:04.427044: osd_op(client.27297.0:50129588 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently reached pg
2013-05-16 18:19:04.515770 osd.0 10.64.0.173:6822/19826 41 : [WRN] slow request 33.147309 seconds old, received at 2013-05-16 18:18:31.368341: osd_op(client.25068.0:11835805 21406.4154_7f677c7a48599cecce1055205e123bae.map [getxattrs,stat,read 0~524288] 3.38c4bd65) v4 currently no flag points reached
2013-05-16 18:19:04.515779 osd.0 10.64.0.173:6822/19826 42 : [WRN] slow request 31.622764 seconds old, received at 2013-05-16 18:18:32.892886: osd_op(client.27297.0:50130711 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:04.515788 osd.0 10.64.0.173:6822/19826 43 : [WRN] slow request 31.475519 seconds old, received at 2013-05-16 18:18:33.040131: osd_op(client.27152.0:11791365 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:19:04.515803 osd.0 10.64.0.173:6822/19826 44 : [WRN] slow request 30.999420 seconds old, received at 2013-05-16 18:18:33.516230: osd_op(client.25068.0:11835885 6900.1942_7/7/1/image_61addc6d_7719271e4e57f271.png [getxattrs,stat,read 0~524288] 3.3d01c249) v4 currently no flag points reached
2013-05-16 18:19:05.516054 osd.0 10.64.0.173:6822/19826 45 : [WRN] 27 slow requests, 3 included below; oldest blocked for > 64.840431 secs
2013-05-16 18:19:05.516066 osd.0 10.64.0.173:6822/19826 46 : [WRN] slow request 31.067885 seconds old, received at 2013-05-16 18:18:34.448066: osd_op(client.27297.0:50130760 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:19:05.516077 osd.0 10.64.0.173:6822/19826 47 : [WRN] slow request 31.033881 seconds old, received at 2013-05-16 18:18:34.482070: osd_op(client.27297.0:50130763 .dir.10267.441 [call rgw.bucket_prepare_op] 3.c47fb39d) v4 currently no flag points reached
2013-05-16 18:19:05.516083 osd.0 10.64.0.173:6822/19826 48 : [WRN] slow request 30.085923 seconds old, received at 2013-05-16 18:18:35.430028: osd_op(client.27297.0:50130785 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:06.995083 osd.0 10.64.0.173:6822/19826 49 : [WRN] 30 slow requests, 3 included below; oldest blocked for > 66.319496 secs
2013-05-16 18:19:06.995090 osd.0 10.64.0.173:6822/19826 50 : [WRN] slow request 31.441331 seconds old, received at 2013-05-16 18:18:35.553685: osd_op(client.25068.0:11835934 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:06.995134 osd.0 10.64.0.173:6822/19826 51 : [WRN] slow request 30.692755 seconds old, received at 2013-05-16 18:18:36.302261: osd_op(client.27152.0:11791499 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:06.995140 osd.0 10.64.0.173:6822/19826 52 : [WRN] slow request 30.410158 seconds old, received at 2013-05-16 18:18:36.584858: osd_op(client.27495.0:5295666 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:19:07.995358 osd.0 10.64.0.173:6822/19826 53 : [WRN] 31 slow requests, 2 included below; oldest blocked for > 67.319760 secs
2013-05-16 18:19:07.995377 osd.0 10.64.0.173:6822/19826 54 : [WRN] slow request 30.153072 seconds old, received at 2013-05-16 18:18:37.842208: osd_op(client.25068.0:11835984 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:07.995384 osd.0 10.64.0.173:6822/19826 55 : [WRN] slow request 30.127952 seconds old, received at 2013-05-16 18:18:37.867328: osd_op(client.25068.0:11835986 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:08.995638 osd.0 10.64.0.173:6822/19826 56 : [WRN] 33 slow requests, 5 included below; oldest blocked for > 68.320015 secs
2013-05-16 18:19:08.995648 osd.0 10.64.0.173:6822/19826 57 : [WRN] slow request 60.104260 seconds old, received at 2013-05-16 18:18:08.891275: osd_op(client.25068.0:11834840 6900.1942_7/3/9/image_fcbc2674_7397949ca06e5c96.png [getxattrs,stat,read 0~524288] 3.224e627a) v4 currently reached pg
2013-05-16 18:19:08.995658 osd.0 10.64.0.173:6822/19826 58 : [WRN] slow request 60.130157 seconds old, received at 2013-05-16 18:18:08.865378: osd_op(client.27495.0:5294688 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently wait for new map
2013-05-16 18:19:08.995668 osd.0 10.64.0.173:6822/19826 59 : [WRN] slow request 60.001221 seconds old, received at 2013-05-16 18:18:08.994314: osd_op(client.25068.0:11834841 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently wait for new map
2013-05-16 18:19:08.995674 osd.0 10.64.0.173:6822/19826 60 : [WRN] slow request 30.774880 seconds old, received at 2013-05-16 18:18:38.220655: osd_op(client.25068.0:11835994 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:19:08.995680 osd.0 10.64.0.173:6822/19826 61 : [WRN] slow request 30.765899 seconds old, received at 2013-05-16 18:18:38.229636: osd_op(client.27495.0:5295709 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:19:09.182281 mon.0 10.64.0.167:6789/0 456631 : [INF] pgmap v7224997: 16760 pgs: 287 active, 16453 active+clean, 8 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 54018B/s wr, 20op/s; 2/806456291 degraded (0.000%)
2013-05-16 18:19:10.355341 mon.0 10.64.0.167:6789/0 456632 : [INF] pgmap v7224998: 16760 pgs: 290 active, 16455 active+clean, 4 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 217KB/s wr, 40op/s; 2/806456312 degraded (0.000%)
2013-05-16 18:19:11.736835 mon.0 10.64.0.167:6789/0 456633 : [INF] pgmap v7224999: 16760 pgs: 288 active, 16457 active+clean, 4 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 955KB/s wr, 37op/s; 2/806456330 degraded (0.000%)
2013-05-16 18:19:13.143643 mon.0 10.64.0.167:6789/0 456634 : [INF] pgmap v7225000: 16760 pgs: 288 active, 16457 active+clean, 4 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 130KB/s wr, 11op/s; 2/806456336 degraded (0.000%)
2013-05-16 18:19:14.608382 mon.0 10.64.0.167:6789/0 456635 : [INF] pgmap v7225001: 16760 pgs: 286 active, 16459 active+clean, 4 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4181KB/s wr, 19op/s; 2/806456351 degraded (0.000%)
2013-05-16 18:19:15.714250 mon.0 10.64.0.167:6789/0 456636 : [INF] pgmap v7225002: 16760 pgs: 287 active, 16460 active+clean, 2 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2797KB/s wr, 24op/s; 2/806456366 degraded (0.000%)
2013-05-16 18:19:17.145612 mon.0 10.64.0.167:6789/0 456637 : [INF] pgmap v7225003: 16760 pgs: 285 active, 16462 active+clean, 2 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 153KB/s wr, 29op/s; 2/806456381 degraded (0.000%)
2013-05-16 18:19:18.760551 mon.0 10.64.0.167:6789/0 456638 : [INF] pgmap v7225004: 16760 pgs: 284 active, 16463 active+clean, 2 peering, 11 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 49389B/s wr, 5op/s; 2/806456384 degraded (0.000%)
2013-05-16 18:19:10.360826 osd.0 10.64.0.173:6822/19826 62 : [WRN] 36 slow requests, 3 included below; oldest blocked for > 69.685211 secs
2013-05-16 18:19:10.360838 osd.0 10.64.0.173:6822/19826 63 : [WRN] slow request 31.162496 seconds old, received at 2013-05-16 18:18:39.198235: osd_op(client.27495.0:5295737 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:10.360845 osd.0 10.64.0.173:6822/19826 64 : [WRN] slow request 31.160948 seconds old, received at 2013-05-16 18:18:39.199783: osd_op(client.27297.0:50130869 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:10.360853 osd.0 10.64.0.173:6822/19826 65 : [WRN] slow request 30.772968 seconds old, received at 2013-05-16 18:18:39.587763: osd_op(client.25068.0:11836048 21406.10171_da635d1178177192feeca19830761c79.map [getxattrs,stat,read 0~524288] 3.11457cb) v4 currently no flag points reached
2013-05-16 18:19:11.361088 osd.0 10.64.0.173:6822/19826 66 : [WRN] 38 slow requests, 2 included below; oldest blocked for > 70.685500 secs
2013-05-16 18:19:11.361099 osd.0 10.64.0.173:6822/19826 67 : [WRN] slow request 30.358408 seconds old, received at 2013-05-16 18:18:41.002612: osd_op(client.25068.0:11836114 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:19:11.361106 osd.0 10.64.0.173:6822/19826 68 : [WRN] slow request 30.235897 seconds old, received at 2013-05-16 18:18:41.125123: osd_op(client.27152.0:11791706 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:13.361605 osd.0 10.64.0.173:6822/19826 69 : [WRN] 40 slow requests, 2 included below; oldest blocked for > 72.686015 secs
2013-05-16 18:19:13.361617 osd.0 10.64.0.173:6822/19826 70 : [WRN] slow request 30.697560 seconds old, received at 2013-05-16 18:18:42.663975: osd_op(client.27495.0:5295831 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:19:13.361623 osd.0 10.64.0.173:6822/19826 71 : [WRN] slow request 30.688120 seconds old, received at 2013-05-16 18:18:42.673415: osd_op(client.25068.0:11836173 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:14.372511 osd.0 10.64.0.173:6822/19826 72 : [WRN] 43 slow requests, 3 included below; oldest blocked for > 73.696898 secs
2013-05-16 18:19:14.372525 osd.0 10.64.0.173:6822/19826 73 : [WRN] slow request 30.911177 seconds old, received at 2013-05-16 18:18:43.461241: osd_op(client.27495.0:5295844 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:19:14.372532 osd.0 10.64.0.173:6822/19826 74 : [WRN] slow request 30.668230 seconds old, received at 2013-05-16 18:18:43.704188: osd_op(client.27495.0:5295854 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:14.372537 osd.0 10.64.0.173:6822/19826 75 : [WRN] slow request 30.584587 seconds old, received at 2013-05-16 18:18:43.787831: osd_op(client.27152.0:11791795 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:16.380646 osd.0 10.64.0.173:6822/19826 76 : [WRN] 44 slow requests, 2 included below; oldest blocked for > 75.705049 secs
2013-05-16 18:19:16.380656 osd.0 10.64.0.173:6822/19826 77 : [WRN] slow request 60.572013 seconds old, received at 2013-05-16 18:18:15.808556: osd_op(client.27495.0:5294799 18499.46_3/39/Phantasm_II.jpg/293px-Phantasm_II.jpg [getxattrs,stat,read 0~524288] 3.e61713d6) v4 currently no flag points reached
2013-05-16 18:19:16.380667 osd.0 10.64.0.173:6822/19826 78 : [WRN] slow request 30.063170 seconds old, received at 2013-05-16 18:18:46.317399: osd_op(client.27152.0:11791973 4143.177_archive/3/36/20081029174247!Wappen_Ringsheim.png [getxattrs,stat,read 0~524288] 3.e7843420) v4 currently no flag points reached
2013-05-16 18:19:17.380984 osd.0 10.64.0.173:6822/19826 79 : [WRN] 46 slow requests, 4 included below; oldest blocked for > 76.705364 secs
2013-05-16 18:19:17.380994 osd.0 10.64.0.173:6822/19826 80 : [WRN] slow request 60.937912 seconds old, received at 2013-05-16 18:18:16.442972: osd_op(client.27152.0:11789852 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:17.381001 osd.0 10.64.0.173:6822/19826 81 : [WRN] slow request 60.554548 seconds old, received at 2013-05-16 18:18:16.826336: osd_op(client.25068.0:11835001 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:17.381007 osd.0 10.64.0.173:6822/19826 82 : [WRN] slow request 30.900499 seconds old, received at 2013-05-16 18:18:46.480385: osd_op(client.27152.0:11791990 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:17.381012 osd.0 10.64.0.173:6822/19826 83 : [WRN] slow request 30.336838 seconds old, received at 2013-05-16 18:18:47.044046: osd_op(client.27297.0:50131293 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:18.381216 osd.0 10.64.0.173:6822/19826 84 : [WRN] 47 slow requests, 1 included below; oldest blocked for > 77.705638 secs
2013-05-16 18:19:18.381229 osd.0 10.64.0.173:6822/19826 85 : [WRN] slow request 30.419263 seconds old, received at 2013-05-16 18:18:47.961895: osd_op(client.27495.0:5296228 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:19.381412 osd.0 10.64.0.173:6822/19826 86 : [WRN] 47 slow requests, 1 included below; oldest blocked for > 78.705846 secs
2013-05-16 18:19:19.381423 osd.0 10.64.0.173:6822/19826 87 : [WRN] slow request 60.600522 seconds old, received at 2013-05-16 18:18:18.780844: osd_sub_op(client.27152.0:11789995 3.1551 8552d551/10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png/head//3 [] v 180331'19700 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:19:20.271301 mon.0 10.64.0.167:6789/0 456639 : [INF] pgmap v7225005: 16760 pgs: 280 active, 16467 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 159KB/s wr, 7op/s; 2/806456390 degraded (0.000%)
2013-05-16 18:19:21.527340 mon.0 10.64.0.167:6789/0 456640 : [INF] pgmap v7225006: 16760 pgs: 280 active, 16467 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 58047B/s wr, 10op/s; 2/806456396 degraded (0.000%)
2013-05-16 18:19:23.099795 mon.0 10.64.0.167:6789/0 456641 : [INF] pgmap v7225007: 16760 pgs: 280 active, 16467 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 209KB/s wr, 22op/s; 2/806456408 degraded (0.000%)
2013-05-16 18:19:19.598631 osd.56 10.64.32.10:6857/17945 1783 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.819919 secs
2013-05-16 18:19:19.598700 osd.56 10.64.32.10:6857/17945 1784 : [WRN] slow request 60.819919 seconds old, received at 2013-05-16 18:18:18.778569: osd_op(client.27152.0:11789995 10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png [create 0~0,delete,writefull 0~37036,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (42),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.8552d551) v4 currently waiting for subops from [37,0]
2013-05-16 18:19:21.818336 osd.50 10.64.32.10:6833/17728 1961 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.600690 secs
2013-05-16 18:19:21.818364 osd.50 10.64.32.10:6833/17728 1962 : [WRN] slow request 60.600690 seconds old, received at 2013-05-16 18:18:21.217583: osd_op(client.27297.0:50130173 10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg [create 0~0,delete,writefull 0~3329,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (97),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.abdaea43) v4 currently waiting for subops from [0,100]
2013-05-16 18:19:24.585896 mon.0 10.64.0.167:6789/0 456642 : [INF] pgmap v7225008: 16760 pgs: 279 active, 16468 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 224KB/s wr, 31op/s; 2/806456429 degraded (0.000%)
2013-05-16 18:19:22.457495 osd.38 10.64.0.176:6831/22256 1587 : [WRN] 5 slow requests, 4 included below; oldest blocked for > 110.819955 secs
2013-05-16 18:19:22.457511 osd.38 10.64.0.176:6831/22256 1588 : [WRN] slow request 60.730012 seconds old, received at 2013-05-16 18:18:21.727393: osd_op(client.27297.0:50130229 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat,read 0~524288] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:22.457517 osd.38 10.64.0.176:6831/22256 1589 : [WRN] slow request 60.729523 seconds old, received at 2013-05-16 18:18:21.727882: osd_op(client.25068.0:11835229 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat,read 0~524288] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:22.457524 osd.38 10.64.0.176:6831/22256 1590 : [WRN] slow request 30.272250 seconds old, received at 2013-05-16 18:18:52.185155: osd_op(client.27495.0:5296321 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:22.457529 osd.38 10.64.0.176:6831/22256 1591 : [WRN] slow request 30.182975 seconds old, received at 2013-05-16 18:18:52.274430: osd_op(client.25068.0:11836405 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:26.117526 mon.0 10.64.0.167:6789/0 456643 : [INF] pgmap v7225009: 16760 pgs: 279 active, 16468 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 98720B/s wr, 36op/s; 2/806456453 degraded (0.000%)
2013-05-16 18:19:27.307422 mon.0 10.64.0.167:6789/0 456644 : [INF] pgmap v7225010: 16760 pgs: 276 active, 16471 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1427KB/s wr, 26op/s; 2/806456471 degraded (0.000%)
2013-05-16 18:19:28.648864 mon.0 10.64.0.167:6789/0 456645 : [INF] pgmap v7225011: 16760 pgs: 270 active, 16477 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 78900B/s wr, 45op/s; 2/806456495 degraded (0.000%)
2013-05-16 18:19:20.381656 osd.0 10.64.0.173:6822/19826 88 : [WRN] 47 slow requests, 1 included below; oldest blocked for > 79.706066 secs
2013-05-16 18:19:20.381666 osd.0 10.64.0.173:6822/19826 89 : [WRN] slow request 60.813233 seconds old, received at 2013-05-16 18:18:19.568353: osd_op(client.27297.0:50129950 21406.4154_69439ed9dd5cbc2402e9e2c8c62587b5.map [getxattrs,stat,read 0~524288] 3.3cc8f6fc) v4 currently no flag points reached
2013-05-16 18:19:21.381881 osd.0 10.64.0.173:6822/19826 90 : [WRN] 48 slow requests, 2 included below; oldest blocked for > 80.706298 secs
2013-05-16 18:19:21.381890 osd.0 10.64.0.173:6822/19826 91 : [WRN] slow request 60.162990 seconds old, received at 2013-05-16 18:18:21.218828: osd_sub_op(client.27297.0:50130173 3.2a43 abdaea43/10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/head//3 [] v 180331'22072 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:19:21.381894 osd.0 10.64.0.173:6822/19826 92 : [WRN] slow request 30.777696 seconds old, received at 2013-05-16 18:18:50.604122: osd_op(client.27152.0:11792263 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:22.382102 osd.0 10.64.0.173:6822/19826 93 : [WRN] 49 slow requests, 1 included below; oldest blocked for > 81.706519 secs
2013-05-16 18:19:22.382112 osd.0 10.64.0.173:6822/19826 94 : [WRN] slow request 30.956360 seconds old, received at 2013-05-16 18:18:51.425679: osd_op(client.27495.0:5296308 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:23.420451 osd.0 10.64.0.173:6822/19826 95 : [WRN] 50 slow requests, 2 included below; oldest blocked for > 82.744846 secs
2013-05-16 18:19:23.420463 osd.0 10.64.0.173:6822/19826 96 : [WRN] slow request 60.516550 seconds old, received at 2013-05-16 18:18:22.903816: osd_op(client.27152.0:11790463 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:19:23.420472 osd.0 10.64.0.173:6822/19826 97 : [WRN] slow request 30.477561 seconds old, received at 2013-05-16 18:18:52.942805: osd_op(client.27152.0:11792304 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:19:25.641529 osd.0 10.64.0.173:6822/19826 98 : [WRN] 50 slow requests, 1 included below; oldest blocked for > 84.965934 secs
2013-05-16 18:19:25.641539 osd.0 10.64.0.173:6822/19826 99 : [WRN] slow request 60.893255 seconds old, received at 2013-05-16 18:18:24.748199: osd_op(client.27152.0:11790516 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:27.970710 osd.0 10.64.0.173:6822/19826 100 : [WRN] 50 slow requests, 2 included below; oldest blocked for > 87.295099 secs
2013-05-16 18:19:27.970719 osd.0 10.64.0.173:6822/19826 101 : [WRN] slow request 60.744238 seconds old, received at 2013-05-16 18:18:27.226381: osd_op(client.25068.0:11835632 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:27.970729 osd.0 10.64.0.173:6822/19826 102 : [WRN] slow request 60.305128 seconds old, received at 2013-05-16 18:18:27.665491: osd_op(client.25068.0:11835683 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:28.970935 osd.0 10.64.0.173:6822/19826 103 : [WRN] 50 slow requests, 1 included below; oldest blocked for > 88.295359 secs
2013-05-16 18:19:28.970941 osd.0 10.64.0.173:6822/19826 104 : [WRN] slow request 60.984480 seconds old, received at 2013-05-16 18:18:27.986399: osd_op(client.27297.0:50130639 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:29.953135 mon.0 10.64.0.167:6789/0 456646 : [INF] pgmap v7225012: 16760 pgs: 269 active, 16478 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2550KB/s wr, 15op/s; 2/806456504 degraded (0.000%)
2013-05-16 18:19:31.368691 mon.0 10.64.0.167:6789/0 456647 : [INF] pgmap v7225013: 16760 pgs: 269 active, 16478 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3190KB/s wr, 16op/s; 2/806456513 degraded (0.000%)
2013-05-16 18:19:32.708085 mon.0 10.64.0.167:6789/0 456648 : [INF] pgmap v7225014: 16760 pgs: 269 active, 16478 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 526KB/s wr, 27op/s; 2/806456531 degraded (0.000%)
2013-05-16 18:19:34.247154 mon.0 10.64.0.167:6789/0 456649 : [INF] pgmap v7225015: 16760 pgs: 265 active, 16482 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3786KB/s wr, 29op/s; 2/806456546 degraded (0.000%)
2013-05-16 18:19:32.459606 osd.38 10.64.0.176:6831/22256 1592 : [WRN] 5 slow requests, 1 included below; oldest blocked for > 120.822100 secs
2013-05-16 18:19:32.459616 osd.38 10.64.0.176:6831/22256 1593 : [WRN] slow request 120.822100 seconds old, received at 2013-05-16 18:17:31.637450: osd_op(client.27297.0:50128581 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently reached pg
2013-05-16 18:19:35.537133 mon.0 10.64.0.167:6789/0 456650 : [INF] pgmap v7225016: 16760 pgs: 263 active, 16484 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s wr, 3op/s; 2/806456546 degraded (0.000%)
2013-05-16 18:19:37.043478 mon.0 10.64.0.167:6789/0 456651 : [INF] pgmap v7225017: 16760 pgs: 260 active, 16487 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 92474B/s wr, 21op/s; 2/806456558 degraded (0.000%)
2013-05-16 18:19:38.349428 mon.0 10.64.0.167:6789/0 456652 : [INF] pgmap v7225018: 16760 pgs: 259 active, 16488 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 267KB/s wr, 16op/s; 2/806456567 degraded (0.000%)
2013-05-16 18:19:31.153417 osd.0 10.64.0.173:6822/19826 105 : [WRN] 51 slow requests, 1 included below; oldest blocked for > 90.477856 secs
2013-05-16 18:19:31.153426 osd.0 10.64.0.173:6822/19826 106 : [WRN] slow request 30.201462 seconds old, received at 2013-05-16 18:19:00.951914: osd_op(client.27152.0:11792707 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:32.153632 osd.0 10.64.0.173:6822/19826 107 : [WRN] 51 slow requests, 1 included below; oldest blocked for > 91.478044 secs
2013-05-16 18:19:32.153644 osd.0 10.64.0.173:6822/19826 108 : [WRN] slow request 60.785223 seconds old, received at 2013-05-16 18:18:31.368341: osd_op(client.25068.0:11835805 21406.4154_7f677c7a48599cecce1055205e123bae.map [getxattrs,stat,read 0~524288] 3.38c4bd65) v4 currently no flag points reached
2013-05-16 18:19:33.153906 osd.0 10.64.0.173:6822/19826 109 : [WRN] 52 slow requests, 3 included below; oldest blocked for > 92.478274 secs
2013-05-16 18:19:33.153916 osd.0 10.64.0.173:6822/19826 110 : [WRN] slow request 60.260908 seconds old, received at 2013-05-16 18:18:32.892886: osd_op(client.27297.0:50130711 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:33.153923 osd.0 10.64.0.173:6822/19826 111 : [WRN] slow request 60.113663 seconds old, received at 2013-05-16 18:18:33.040131: osd_op(client.27152.0:11791365 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:19:33.153929 osd.0 10.64.0.173:6822/19826 112 : [WRN] slow request 30.561380 seconds old, received at 2013-05-16 18:19:02.592414: osd_op(client.27297.0:50132075 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:19:34.241403 osd.0 10.64.0.173:6822/19826 113 : [WRN] 53 slow requests, 2 included below; oldest blocked for > 93.565789 secs
2013-05-16 18:19:34.241414 osd.0 10.64.0.173:6822/19826 114 : [WRN] slow request 60.725079 seconds old, received at 2013-05-16 18:18:33.516230: osd_op(client.25068.0:11835885 6900.1942_7/7/1/image_61addc6d_7719271e4e57f271.png [getxattrs,stat,read 0~524288] 3.3d01c249) v4 currently no flag points reached
2013-05-16 18:19:34.241421 osd.0 10.64.0.173:6822/19826 115 : [WRN] slow request 30.958095 seconds old, received at 2013-05-16 18:19:03.283214: osd_op(client.27152.0:11792771 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:35.241686 osd.0 10.64.0.173:6822/19826 116 : [WRN] 54 slow requests, 3 included below; oldest blocked for > 94.566063 secs
2013-05-16 18:19:35.241695 osd.0 10.64.0.173:6822/19826 117 : [WRN] slow request 60.793517 seconds old, received at 2013-05-16 18:18:34.448066: osd_op(client.27297.0:50130760 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:19:35.241701 osd.0 10.64.0.173:6822/19826 118 : [WRN] slow request 60.759513 seconds old, received at 2013-05-16 18:18:34.482070: osd_op(client.27297.0:50130763 .dir.10267.441 [call rgw.bucket_prepare_op] 3.c47fb39d) v4 currently no flag points reached
2013-05-16 18:19:35.241708 osd.0 10.64.0.173:6822/19826 119 : [WRN] slow request 30.753697 seconds old, received at 2013-05-16 18:19:04.487886: osd_op(client.25068.0:11837242 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:19:36.241944 osd.0 10.64.0.173:6822/19826 120 : [WRN] 54 slow requests, 2 included below; oldest blocked for > 95.566349 secs
2013-05-16 18:19:36.241954 osd.0 10.64.0.173:6822/19826 121 : [WRN] slow request 60.811841 seconds old, received at 2013-05-16 18:18:35.430028: osd_op(client.27297.0:50130785 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:36.241962 osd.0 10.64.0.173:6822/19826 122 : [WRN] slow request 60.688184 seconds old, received at 2013-05-16 18:18:35.553685: osd_op(client.25068.0:11835934 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:37.492340 osd.0 10.64.0.173:6822/19826 123 : [WRN] 56 slow requests, 4 included below; oldest blocked for > 96.816719 secs
2013-05-16 18:19:37.492350 osd.0 10.64.0.173:6822/19826 124 : [WRN] slow request 61.189978 seconds old, received at 2013-05-16 18:18:36.302261: osd_op(client.27152.0:11791499 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:37.492356 osd.0 10.64.0.173:6822/19826 125 : [WRN] slow request 60.907381 seconds old, received at 2013-05-16 18:18:36.584858: osd_op(client.27495.0:5295666 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:19:37.492364 osd.0 10.64.0.173:6822/19826 126 : [WRN] slow request 31.153697 seconds old, received at 2013-05-16 18:19:06.338542: osd_op(client.27297.0:50132218 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:37.492370 osd.0 10.64.0.173:6822/19826 127 : [WRN] slow request 30.871475 seconds old, received at 2013-05-16 18:19:06.620764: osd_op(client.25068.0:11837301 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:19:38.492590 osd.0 10.64.0.173:6822/19826 128 : [WRN] 58 slow requests, 6 included below; oldest blocked for > 97.816975 secs
2013-05-16 18:19:38.492600 osd.0 10.64.0.173:6822/19826 129 : [WRN] slow request 60.650287 seconds old, received at 2013-05-16 18:18:37.842208: osd_op(client.25068.0:11835984 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:38.492607 osd.0 10.64.0.173:6822/19826 130 : [WRN] slow request 60.625167 seconds old, received at 2013-05-16 18:18:37.867328: osd_op(client.25068.0:11835986 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:38.492613 osd.0 10.64.0.173:6822/19826 131 : [WRN] slow request 60.271840 seconds old, received at 2013-05-16 18:18:38.220655: osd_op(client.25068.0:11835994 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:19:38.492620 osd.0 10.64.0.173:6822/19826 132 : [WRN] slow request 60.262859 seconds old, received at 2013-05-16 18:18:38.229636: osd_op(client.27495.0:5295709 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:19:38.492629 osd.0 10.64.0.173:6822/19826 133 : [WRN] slow request 30.917409 seconds old, received at 2013-05-16 18:19:07.575086: osd_op(client.27152.0:11792858 10267.620_9/94/Bloomberg_tower.jpg/212px-Bloomberg_tower.jpg [getxattrs,stat,read 0~524288] 3.dd653d65) v4 currently no flag points reached
2013-05-16 18:19:39.492972 osd.0 10.64.0.173:6822/19826 134 : [WRN] 61 slow requests, 6 included below; oldest blocked for > 98.817348 secs
2013-05-16 18:19:39.492983 osd.0 10.64.0.173:6822/19826 135 : [WRN] slow request 60.294633 seconds old, received at 2013-05-16 18:18:39.198235: osd_op(client.27495.0:5295737 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:39.492991 osd.0 10.64.0.173:6822/19826 136 : [WRN] slow request 60.293085 seconds old, received at 2013-05-16 18:18:39.199783: osd_op(client.27297.0:50130869 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:39.493000 osd.0 10.64.0.173:6822/19826 137 : [WRN] slow request 31.614338 seconds old, received at 2013-05-16 18:19:07.878530: osd_op(client.27297.0:50132256 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:39.493006 osd.0 10.64.0.173:6822/19826 138 : [WRN] slow request 31.591820 seconds old, received at 2013-05-16 18:19:07.901048: osd_op(client.27297.0:50132258 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:39.493012 osd.0 10.64.0.173:6822/19826 139 : [WRN] slow request 31.320252 seconds old, received at 2013-05-16 18:19:08.172616: osd_op(client.25068.0:11837335 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:19:39.855731 mon.0 10.64.0.167:6789/0 456653 : [INF] pgmap v7225019: 16760 pgs: 259 active, 16488 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 106KB/s wr, 4op/s; 2/806456570 degraded (0.000%)
2013-05-16 18:19:41.185155 mon.0 10.64.0.167:6789/0 456654 : [INF] pgmap v7225020: 16760 pgs: 254 active, 16493 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 513KB/s wr, 40op/s; 2/806456591 degraded (0.000%)
2013-05-16 18:19:42.758044 mon.0 10.64.0.167:6789/0 456655 : [INF] pgmap v7225021: 16760 pgs: 249 active, 16498 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 54114B/s wr, 15op/s; 2/806456600 degraded (0.000%)
2013-05-16 18:19:44.239414 mon.0 10.64.0.167:6789/0 456656 : [INF] pgmap v7225022: 16760 pgs: 246 active, 16501 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 4445B/s wr, 9op/s; 2/806456603 degraded (0.000%)
2013-05-16 18:19:45.762351 mon.0 10.64.0.167:6789/0 456657 : [INF] pgmap v7225023: 16760 pgs: 246 active, 16501 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2851KB/s wr, 38op/s; 2/806456627 degraded (0.000%)
2013-05-16 18:19:38.460911 osd.38 10.64.0.176:6831/22256 1594 : [WRN] 6 slow requests, 1 included below; oldest blocked for > 126.823402 secs
2013-05-16 18:19:38.460921 osd.38 10.64.0.176:6831/22256 1595 : [WRN] slow request 30.781973 seconds old, received at 2013-05-16 18:19:07.678879: osd_op(client.27152.0:11792859 .dir.10267.498 [call rgw.bucket_list] 3.a2457d08) v4 currently reached pg
2013-05-16 18:19:47.171771 mon.0 10.64.0.167:6789/0 456658 : [INF] pgmap v7225024: 16760 pgs: 242 active, 16505 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 65614B/s wr, 18op/s; 2/806456639 degraded (0.000%)
2013-05-16 18:19:48.749239 mon.0 10.64.0.167:6789/0 456659 : [INF] pgmap v7225025: 16760 pgs: 237 active, 16510 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 96400B/s wr, 33op/s; 2/806456660 degraded (0.000%)
2013-05-16 18:19:40.900227 osd.0 10.64.0.173:6822/19826 140 : [WRN] 62 slow requests, 3 included below; oldest blocked for > 100.224592 secs
2013-05-16 18:19:40.900235 osd.0 10.64.0.173:6822/19826 141 : [WRN] slow request 61.312349 seconds old, received at 2013-05-16 18:18:39.587763: osd_op(client.25068.0:11836048 21406.10171_da635d1178177192feeca19830761c79.map [getxattrs,stat,read 0~524288] 3.11457cb) v4 currently no flag points reached
2013-05-16 18:19:40.900241 osd.0 10.64.0.173:6822/19826 142 : [WRN] slow request 32.642829 seconds old, received at 2013-05-16 18:19:08.257283: osd_op(client.25068.0:11837336 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:19:40.900246 osd.0 10.64.0.173:6822/19826 143 : [WRN] slow request 32.632773 seconds old, received at 2013-05-16 18:19:08.267339: osd_op(client.27495.0:5296678 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:19:41.985938 osd.0 10.64.0.173:6822/19826 144 : [WRN] 66 slow requests, 6 included below; oldest blocked for > 101.310310 secs
2013-05-16 18:19:41.985947 osd.0 10.64.0.173:6822/19826 145 : [WRN] slow request 60.983218 seconds old, received at 2013-05-16 18:18:41.002612: osd_op(client.25068.0:11836114 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:19:41.985953 osd.0 10.64.0.173:6822/19826 146 : [WRN] slow request 60.860707 seconds old, received at 2013-05-16 18:18:41.125123: osd_op(client.27152.0:11791706 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:41.985958 osd.0 10.64.0.173:6822/19826 147 : [WRN] slow request 30.947260 seconds old, received at 2013-05-16 18:19:11.038570: osd_op(client.25068.0:11837372 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:19:41.985963 osd.0 10.64.0.173:6822/19826 148 : [WRN] slow request 30.465094 seconds old, received at 2013-05-16 18:19:11.520736: osd_op(client.27297.0:50132407 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:19:41.985970 osd.0 10.64.0.173:6822/19826 149 : [WRN] slow request 30.461746 seconds old, received at 2013-05-16 18:19:11.524084: osd_op(client.25068.0:11837390 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:19:42.986195 osd.0 10.64.0.173:6822/19826 150 : [WRN] 69 slow requests, 6 included below; oldest blocked for > 102.310577 secs
2013-05-16 18:19:42.986208 osd.0 10.64.0.173:6822/19826 151 : [WRN] slow request 60.322122 seconds old, received at 2013-05-16 18:18:42.663975: osd_op(client.27495.0:5295831 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:19:42.986215 osd.0 10.64.0.173:6822/19826 152 : [WRN] slow request 60.312682 seconds old, received at 2013-05-16 18:18:42.673415: osd_op(client.25068.0:11836173 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:19:42.986219 osd.0 10.64.0.173:6822/19826 153 : [WRN] slow request 31.438900 seconds old, received at 2013-05-16 18:19:11.547197: osd_op(client.27297.0:50132408 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:42.986225 osd.0 10.64.0.173:6822/19826 154 : [WRN] slow request 30.754268 seconds old, received at 2013-05-16 18:19:12.231829: osd_op(client.27297.0:50132445 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:42.986235 osd.0 10.64.0.173:6822/19826 155 : [WRN] slow request 30.286080 seconds old, received at 2013-05-16 18:19:12.700017: osd_op(client.27297.0:50132459 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:19:43.986412 osd.0 10.64.0.173:6822/19826 156 : [WRN] 70 slow requests, 5 included below; oldest blocked for > 103.310823 secs
2013-05-16 18:19:43.986418 osd.0 10.64.0.173:6822/19826 157 : [WRN] slow request 60.525102 seconds old, received at 2013-05-16 18:18:43.461241: osd_op(client.27495.0:5295844 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:19:43.986423 osd.0 10.64.0.173:6822/19826 158 : [WRN] slow request 60.282155 seconds old, received at 2013-05-16 18:18:43.704188: osd_op(client.27495.0:5295854 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:43.986426 osd.0 10.64.0.173:6822/19826 159 : [WRN] slow request 60.198512 seconds old, received at 2013-05-16 18:18:43.787831: osd_op(client.27152.0:11791795 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:43.986431 osd.0 10.64.0.173:6822/19826 160 : [WRN] slow request 31.147744 seconds old, received at 2013-05-16 18:19:12.838599: osd_op(client.27297.0:50132463 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:43.986434 osd.0 10.64.0.173:6822/19826 161 : [WRN] slow request 30.928417 seconds old, received at 2013-05-16 18:19:13.057926: osd_op(client.25068.0:11837432 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:19:44.986617 osd.0 10.64.0.173:6822/19826 162 : [WRN] 71 slow requests, 1 included below; oldest blocked for > 104.311052 secs
2013-05-16 18:19:44.986623 osd.0 10.64.0.173:6822/19826 163 : [WRN] slow request 30.821011 seconds old, received at 2013-05-16 18:19:14.165561: osd_op(client.27152.0:11792997 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:19:47.078451 osd.0 10.64.0.173:6822/19826 164 : [WRN] 71 slow requests, 3 included below; oldest blocked for > 106.402835 secs
2013-05-16 18:19:47.078466 osd.0 10.64.0.173:6822/19826 165 : [WRN] slow request 60.760956 seconds old, received at 2013-05-16 18:18:46.317399: osd_op(client.27152.0:11791973 4143.177_archive/3/36/20081029174247!Wappen_Ringsheim.png [getxattrs,stat,read 0~524288] 3.e7843420) v4 currently no flag points reached
2013-05-16 18:19:47.078474 osd.0 10.64.0.173:6822/19826 166 : [WRN] slow request 60.597970 seconds old, received at 2013-05-16 18:18:46.480385: osd_op(client.27152.0:11791990 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:47.078479 osd.0 10.64.0.173:6822/19826 167 : [WRN] slow request 60.034309 seconds old, received at 2013-05-16 18:18:47.044046: osd_op(client.27297.0:50131293 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:19:48.078702 osd.0 10.64.0.173:6822/19826 168 : [WRN] 72 slow requests, 2 included below; oldest blocked for > 107.403114 secs
2013-05-16 18:19:48.078714 osd.0 10.64.0.173:6822/19826 169 : [WRN] slow request 60.116739 seconds old, received at 2013-05-16 18:18:47.961895: osd_op(client.27495.0:5296228 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:19:48.078723 osd.0 10.64.0.173:6822/19826 170 : [WRN] slow request 30.906776 seconds old, received at 2013-05-16 18:19:17.171858: osd_op(client.27152.0:11793045 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:49.078934 osd.0 10.64.0.173:6822/19826 171 : [WRN] 73 slow requests, 1 included below; oldest blocked for > 108.403350 secs
2013-05-16 18:19:49.078953 osd.0 10.64.0.173:6822/19826 172 : [WRN] slow request 30.331037 seconds old, received at 2013-05-16 18:19:18.747833: osd_op(client.27495.0:5296990 6900.31_7/79/Birch_House_Farm_-_geograph.org.uk_-_1204925.jpg [getxattrs,stat,read 0~524288] 3.efdae9e0) v4 currently no flag points reached
2013-05-16 18:19:50.118103 mon.0 10.64.0.167:6789/0 456660 : [INF] pgmap v7225026: 16760 pgs: 236 active, 16511 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1581KB/s wr, 30op/s; 2/806456684 degraded (0.000%)
2013-05-16 18:19:51.695050 mon.0 10.64.0.167:6789/0 456661 : [INF] pgmap v7225027: 16760 pgs: 235 active, 16512 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 767KB/s wr, 37op/s; 2/806456705 degraded (0.000%)
2013-05-16 18:19:53.098094 mon.0 10.64.0.167:6789/0 456662 : [INF] pgmap v7225028: 16760 pgs: 235 active, 16512 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1152KB/s wr, 37op/s; 2/806456729 degraded (0.000%)
2013-05-16 18:19:54.688152 mon.0 10.64.0.167:6789/0 456663 : [INF] pgmap v7225029: 16760 pgs: 230 active, 16517 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 833KB/s wr, 17op/s; 2/806456744 degraded (0.000%)
2013-05-16 18:19:52.463897 osd.38 10.64.0.176:6831/22256 1596 : [WRN] 6 slow requests, 2 included below; oldest blocked for > 140.826373 secs
2013-05-16 18:19:52.463908 osd.38 10.64.0.176:6831/22256 1597 : [WRN] slow request 60.278668 seconds old, received at 2013-05-16 18:18:52.185155: osd_op(client.27495.0:5296321 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:52.463914 osd.38 10.64.0.176:6831/22256 1598 : [WRN] slow request 60.189393 seconds old, received at 2013-05-16 18:18:52.274430: osd_op(client.25068.0:11836405 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [getxattrs,stat] 3.59e4fd08) v4 currently reached pg
2013-05-16 18:19:56.080045 mon.0 10.64.0.167:6789/0 456664 : [INF] pgmap v7225030: 16760 pgs: 229 active, 16518 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3472KB/s wr, 42op/s; 2/806456774 degraded (0.000%)
2013-05-16 18:19:57.627422 mon.0 10.64.0.167:6789/0 456665 : [INF] pgmap v7225031: 16760 pgs: 226 active, 16521 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1326KB/s wr, 22op/s; 2/806456789 degraded (0.000%)
2013-05-16 18:19:50.994222 osd.101 10.64.32.14:6818/2810 1852 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.549394 secs
2013-05-16 18:19:50.994246 osd.101 10.64.32.14:6818/2810 1853 : [WRN] slow request 30.549394 seconds old, received at 2013-05-16 18:19:20.444711: osd_op(client.25068.0:11837568 10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg [create 0~0,delete,writefull 0~2097,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (87),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.e5cd201a) v4 currently waiting for subops from [0,25]
2013-05-16 18:19:59.115127 mon.0 10.64.0.167:6789/0 456666 : [INF] pgmap v7225032: 16760 pgs: 226 active, 16521 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4271KB/s wr, 46op/s; 2/806456819 degraded (0.000%)
2013-05-16 18:19:51.293233 osd.0 10.64.0.173:6822/19826 173 : [WRN] 75 slow requests, 3 included below; oldest blocked for > 110.617604 secs
2013-05-16 18:19:51.293244 osd.0 10.64.0.173:6822/19826 174 : [WRN] slow request 60.689002 seconds old, received at 2013-05-16 18:18:50.604122: osd_op(client.27152.0:11792263 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:51.293250 osd.0 10.64.0.173:6822/19826 175 : [WRN] slow request 30.847329 seconds old, received at 2013-05-16 18:19:20.445795: osd_sub_op(client.25068.0:11837568 3.201a e5cd201a/10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/head//3 [] v 180331'21662 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:19:51.293255 osd.0 10.64.0.173:6822/19826 176 : [WRN] slow request 30.653723 seconds old, received at 2013-05-16 18:19:20.639401: osd_op(client.27297.0:50132610 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:19:52.293473 osd.0 10.64.0.173:6822/19826 177 : [WRN] 76 slow requests, 2 included below; oldest blocked for > 111.617878 secs
2013-05-16 18:19:52.293483 osd.0 10.64.0.173:6822/19826 178 : [WRN] slow request 60.867719 seconds old, received at 2013-05-16 18:18:51.425679: osd_op(client.27495.0:5296308 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:52.293489 osd.0 10.64.0.173:6822/19826 179 : [WRN] slow request 30.831731 seconds old, received at 2013-05-16 18:19:21.461667: osd_op(client.27152.0:11793151 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:19:53.293708 osd.0 10.64.0.173:6822/19826 180 : [WRN] 77 slow requests, 2 included below; oldest blocked for > 112.618100 secs
2013-05-16 18:19:53.293715 osd.0 10.64.0.173:6822/19826 181 : [WRN] slow request 60.350815 seconds old, received at 2013-05-16 18:18:52.942805: osd_op(client.27152.0:11792304 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:19:53.293720 osd.0 10.64.0.173:6822/19826 182 : [WRN] slow request 30.439928 seconds old, received at 2013-05-16 18:19:22.853692: osd_op(client.27495.0:5297108 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:19:55.879665 osd.0 10.64.0.173:6822/19826 183 : [WRN] 79 slow requests, 2 included below; oldest blocked for > 115.204060 secs
2013-05-16 18:19:55.879678 osd.0 10.64.0.173:6822/19826 184 : [WRN] slow request 30.974040 seconds old, received at 2013-05-16 18:19:24.905540: osd_op(client.25068.0:11837653 6900.1942_3/9/3/image_3336f9d6_3938ed8917ea182e.png [getxattrs,stat,read 0~524288] 3.1507b6c9) v4 currently no flag points reached
2013-05-16 18:19:55.879684 osd.0 10.64.0.173:6822/19826 185 : [WRN] slow request 30.803869 seconds old, received at 2013-05-16 18:19:25.075711: osd_op(client.27152.0:11793231 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat,read 0~524288] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:19:56.879893 osd.0 10.64.0.173:6822/19826 186 : [WRN] 80 slow requests, 1 included below; oldest blocked for > 116.204313 secs
2013-05-16 18:19:56.879899 osd.0 10.64.0.173:6822/19826 187 : [WRN] slow request 30.457656 seconds old, received at 2013-05-16 18:19:26.422177: osd_op(client.27297.0:50132717 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:00.606304 mon.0 10.64.0.167:6789/0 456667 : [INF] pgmap v7225033: 16760 pgs: 223 active, 16524 active+clean, 1 peering, 12 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 710KB/s wr, 20op/s; 2/806456834 degraded (0.000%)
2013-05-16 18:20:01.979512 mon.0 10.64.0.167:6789/0 456668 : [INF] pgmap v7225034: 16760 pgs: 219 active, 16527 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 1466KB/s wr, 32op/s; 2/806456855 degraded (0.000%)
2013-05-16 18:20:03.485223 mon.0 10.64.0.167:6789/0 456669 : [INF] pgmap v7225035: 16760 pgs: 219 active, 16527 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3871KB/s wr, 28op/s; 2/806456870 degraded (0.000%)
2013-05-16 18:20:04.866617 mon.0 10.64.0.167:6789/0 456670 : [INF] pgmap v7225036: 16760 pgs: 215 active, 16531 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 5797KB/s wr, 42op/s; 2/806456891 degraded (0.000%)
2013-05-16 18:20:06.347941 mon.0 10.64.0.167:6789/0 456671 : [INF] pgmap v7225037: 16760 pgs: 213 active, 16533 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 7283B/s wr, 15op/s; 2/806456891 degraded (0.000%)
2013-05-16 18:20:07.821671 mon.0 10.64.0.167:6789/0 456672 : [INF] pgmap v7225038: 16760 pgs: 212 active, 16534 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 160KB/s wr, 45op/s; 2/806456909 degraded (0.000%)
2013-05-16 18:20:09.261384 mon.0 10.64.0.167:6789/0 456673 : [INF] pgmap v7225039: 16760 pgs: 209 active, 16537 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 124KB/s wr, 33op/s; 2/806456921 degraded (0.000%)
2013-05-16 18:20:01.144557 osd.0 10.64.0.173:6822/19826 188 : [WRN] 81 slow requests, 4 included below; oldest blocked for > 120.468944 secs
2013-05-16 18:20:01.144570 osd.0 10.64.0.173:6822/19826 189 : [WRN] slow request 120.468944 seconds old, received at 2013-05-16 18:18:00.675520: osd_op(client.27495.0:5293708 21406.1730_840e203401db4651ca004b8a69f87bb5.map [getxattrs,stat,read 0~524288] 3.61c06ff RETRY) v4 currently reached pg
2013-05-16 18:20:01.144584 osd.0 10.64.0.173:6822/19826 190 : [WRN] slow request 120.468861 seconds old, received at 2013-05-16 18:18:00.675603: osd_op(client.27495.0:5293813 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712 RETRY) v4 currently reached pg
2013-05-16 18:20:01.144590 osd.0 10.64.0.173:6822/19826 191 : [WRN] slow request 60.192550 seconds old, received at 2013-05-16 18:19:00.951914: osd_op(client.27152.0:11792707 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:01.144620 osd.0 10.64.0.173:6822/19826 192 : [WRN] slow request 30.047630 seconds old, received at 2013-05-16 18:19:31.096834: osd_op(client.27152.0:11793342 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:20:02.412521 osd.0 10.64.0.173:6822/19826 193 : [WRN] 84 slow requests, 4 included below; oldest blocked for > 121.736893 secs
2013-05-16 18:20:02.412529 osd.0 10.64.0.173:6822/19826 194 : [WRN] slow request 121.104425 seconds old, received at 2013-05-16 18:18:01.307988: osd_op(client.27297.0:50128744 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8 RETRY) v4 currently reached pg
2013-05-16 18:20:02.412533 osd.0 10.64.0.173:6822/19826 195 : [WRN] slow request 30.514319 seconds old, received at 2013-05-16 18:19:31.898094: osd_op(client.27495.0:5297521 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:02.412537 osd.0 10.64.0.173:6822/19826 196 : [WRN] slow request 30.509088 seconds old, received at 2013-05-16 18:19:31.903325: osd_op(client.27297.0:50133033 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:02.412543 osd.0 10.64.0.173:6822/19826 197 : [WRN] slow request 30.295732 seconds old, received at 2013-05-16 18:19:32.116681: osd_op(client.27297.0:50133037 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:20:03.412810 osd.0 10.64.0.173:6822/19826 198 : [WRN] 85 slow requests, 4 included below; oldest blocked for > 122.737186 secs
2013-05-16 18:20:03.412819 osd.0 10.64.0.173:6822/19826 199 : [WRN] slow request 120.406552 seconds old, received at 2013-05-16 18:18:03.006154: osd_op(client.25068.0:11834698 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently reached pg
2013-05-16 18:20:03.412839 osd.0 10.64.0.173:6822/19826 200 : [WRN] slow request 60.820292 seconds old, received at 2013-05-16 18:19:02.592414: osd_op(client.27297.0:50132075 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:20:03.412845 osd.0 10.64.0.173:6822/19826 201 : [WRN] slow request 60.129492 seconds old, received at 2013-05-16 18:19:03.283214: osd_op(client.27152.0:11792771 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:03.412851 osd.0 10.64.0.173:6822/19826 202 : [WRN] slow request 30.784930 seconds old, received at 2013-05-16 18:19:32.627776: osd_op(client.27495.0:5297532 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:20:05.413246 osd.0 10.64.0.173:6822/19826 203 : [WRN] 86 slow requests, 3 included below; oldest blocked for > 124.737641 secs
2013-05-16 18:20:05.413254 osd.0 10.64.0.173:6822/19826 204 : [WRN] slow request 120.986117 seconds old, received at 2013-05-16 18:18:04.427044: osd_op(client.27297.0:50129588 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently reached pg
2013-05-16 18:20:05.413264 osd.0 10.64.0.173:6822/19826 205 : [WRN] slow request 60.925275 seconds old, received at 2013-05-16 18:19:04.487886: osd_op(client.25068.0:11837242 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:20:05.413270 osd.0 10.64.0.173:6822/19826 206 : [WRN] slow request 30.428689 seconds old, received at 2013-05-16 18:19:34.984472: osd_op(client.27152.0:11793433 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:20:06.413504 osd.0 10.64.0.173:6822/19826 207 : [WRN] 87 slow requests, 2 included below; oldest blocked for > 125.737909 secs
2013-05-16 18:20:06.413515 osd.0 10.64.0.173:6822/19826 208 : [WRN] slow request 60.074887 seconds old, received at 2013-05-16 18:19:06.338542: osd_op(client.27297.0:50132218 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:20:06.413521 osd.0 10.64.0.173:6822/19826 209 : [WRN] slow request 30.707495 seconds old, received at 2013-05-16 18:19:35.705934: osd_op(client.27152.0:11793455 21406.1730_d5d29fdc7a4f92c12cd935e043c6f46a.map [getxattrs,stat,read 0~524288] 3.c4a5e27a) v4 currently no flag points reached
2013-05-16 18:20:07.413713 osd.0 10.64.0.173:6822/19826 210 : [WRN] 87 slow requests, 1 included below; oldest blocked for > 126.738128 secs
2013-05-16 18:20:07.413723 osd.0 10.64.0.173:6822/19826 211 : [WRN] slow request 60.792884 seconds old, received at 2013-05-16 18:19:06.620764: osd_op(client.25068.0:11837301 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:08.414010 osd.0 10.64.0.173:6822/19826 212 : [WRN] 62 slow requests, 6 included below; oldest blocked for > 127.738377 secs
2013-05-16 18:20:08.414021 osd.0 10.64.0.173:6822/19826 213 : [WRN] slow request 60.838811 seconds old, received at 2013-05-16 18:19:07.575086: osd_op(client.27152.0:11792858 10267.620_9/94/Bloomberg_tower.jpg/212px-Bloomberg_tower.jpg [getxattrs,stat,read 0~524288] 3.dd653d65) v4 currently no flag points reached
2013-05-16 18:20:08.414035 osd.0 10.64.0.173:6822/19826 214 : [WRN] slow request 60.535367 seconds old, received at 2013-05-16 18:19:07.878530: osd_op(client.27297.0:50132256 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:08.414040 osd.0 10.64.0.173:6822/19826 215 : [WRN] slow request 60.512849 seconds old, received at 2013-05-16 18:19:07.901048: osd_op(client.27297.0:50132258 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:08.414046 osd.0 10.64.0.173:6822/19826 216 : [WRN] slow request 60.241281 seconds old, received at 2013-05-16 18:19:08.172616: osd_op(client.25068.0:11837335 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:20:08.414051 osd.0 10.64.0.173:6822/19826 217 : [WRN] slow request 60.156614 seconds old, received at 2013-05-16 18:19:08.257283: osd_op(client.25068.0:11837336 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:09.414296 osd.0 10.64.0.173:6822/19826 218 : [WRN] 89 slow requests, 6 included below; oldest blocked for > 128.738666 secs
2013-05-16 18:20:09.414305 osd.0 10.64.0.173:6822/19826 219 : [WRN] slow request 120.522911 seconds old, received at 2013-05-16 18:18:08.891275: osd_op(client.25068.0:11834840 6900.1942_7/3/9/image_fcbc2674_7397949ca06e5c96.png [getxattrs,stat,read 0~524288] 3.224e627a) v4 currently reached pg
2013-05-16 18:20:09.414313 osd.0 10.64.0.173:6822/19826 220 : [WRN] slow request 120.548808 seconds old, received at 2013-05-16 18:18:08.865378: osd_op(client.27495.0:5294688 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently wait for new map
2013-05-16 18:20:09.414320 osd.0 10.64.0.173:6822/19826 221 : [WRN] slow request 120.419872 seconds old, received at 2013-05-16 18:18:08.994314: osd_op(client.25068.0:11834841 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently wait for new map
2013-05-16 18:20:09.414329 osd.0 10.64.0.173:6822/19826 222 : [WRN] slow request 61.146847 seconds old, received at 2013-05-16 18:19:08.267339: osd_op(client.27495.0:5296678 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:20:09.414335 osd.0 10.64.0.173:6822/19826 223 : [WRN] slow request 31.748432 seconds old, received at 2013-05-16 18:19:37.665754: osd_op(client.27152.0:11793522 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:20:10.684992 mon.0 10.64.0.167:6789/0 456674 : [INF] pgmap v7225040: 16760 pgs: 207 active, 16539 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 331KB/s wr, 39op/s; 2/806456945 degraded (0.000%)
2013-05-16 18:20:12.005748 mon.0 10.64.0.167:6789/0 456675 : [INF] pgmap v7225041: 16760 pgs: 206 active, 16540 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1447B/s wr, 4op/s; 2/806456948 degraded (0.000%)
2013-05-16 18:20:13.328878 mon.0 10.64.0.167:6789/0 456676 : [INF] pgmap v7225042: 16760 pgs: 206 active, 16540 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 32091B/s wr, 15op/s; 2/806456957 degraded (0.000%)
2013-05-16 18:20:14.629179 mon.0 10.64.0.167:6789/0 456677 : [INF] pgmap v7225043: 16760 pgs: 205 active, 16541 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4611KB/s wr, 31op/s; 2/806456975 degraded (0.000%)
2013-05-16 18:20:08.467127 osd.38 10.64.0.176:6831/22256 1599 : [WRN] 6 slow requests, 1 included below; oldest blocked for > 156.829616 secs
2013-05-16 18:20:08.467137 osd.38 10.64.0.176:6831/22256 1600 : [WRN] slow request 60.788187 seconds old, received at 2013-05-16 18:19:07.678879: osd_op(client.27152.0:11792859 .dir.10267.498 [call rgw.bucket_list] 3.a2457d08) v4 currently reached pg
2013-05-16 18:20:13.468146 osd.38 10.64.0.176:6831/22256 1601 : [WRN] 7 slow requests, 1 included below; oldest blocked for > 161.830617 secs
2013-05-16 18:20:13.468176 osd.38 10.64.0.176:6831/22256 1602 : [WRN] slow request 30.032567 seconds old, received at 2013-05-16 18:19:43.435500: osd_op(client.27495.0:5297872 .dir.10267.498 [call rgw.bucket_list] 3.a2457d08) v4 currently reached pg
2013-05-16 18:20:16.177786 mon.0 10.64.0.167:6789/0 456678 : [INF] pgmap v7225044: 16760 pgs: 202 active, 16544 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3135B/s wr, 10op/s; 2/806456981 degraded (0.000%)
2013-05-16 18:20:17.714183 mon.0 10.64.0.167:6789/0 456679 : [INF] pgmap v7225045: 16760 pgs: 201 active, 16545 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 740KB/s wr, 10op/s; 2/806456990 degraded (0.000%)
2013-05-16 18:20:19.312152 mon.0 10.64.0.167:6789/0 456680 : [INF] pgmap v7225046: 16760 pgs: 199 active, 16547 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 363KB/s wr, 25op/s; 2/806457002 degraded (0.000%)
2013-05-16 18:20:10.414606 osd.0 10.64.0.173:6822/19826 224 : [WRN] 94 slow requests, 6 included below; oldest blocked for > 129.738972 secs
2013-05-16 18:20:10.414616 osd.0 10.64.0.173:6822/19826 225 : [WRN] slow request 32.631294 seconds old, received at 2013-05-16 18:19:37.783198: osd_op(client.27495.0:5297687 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:10.414627 osd.0 10.64.0.173:6822/19826 226 : [WRN] slow request 32.208141 seconds old, received at 2013-05-16 18:19:38.206351: osd_op(client.25068.0:11837884 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:20:10.414635 osd.0 10.64.0.173:6822/19826 227 : [WRN] slow request 32.182052 seconds old, received at 2013-05-16 18:19:38.232440: osd_op(client.27495.0:5297690 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:10.414640 osd.0 10.64.0.173:6822/19826 228 : [WRN] slow request 32.111024 seconds old, received at 2013-05-16 18:19:38.303468: osd_op(client.27495.0:5297693 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:10.414655 osd.0 10.64.0.173:6822/19826 229 : [WRN] slow request 30.577904 seconds old, received at 2013-05-16 18:19:39.836588: osd_op(client.27297.0:50133216 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:20:11.615180 osd.0 10.64.0.173:6822/19826 230 : [WRN] 95 slow requests, 6 included below; oldest blocked for > 130.939543 secs
2013-05-16 18:20:11.615190 osd.0 10.64.0.173:6822/19826 231 : [WRN] slow request 60.576493 seconds old, received at 2013-05-16 18:19:11.038570: osd_op(client.25068.0:11837372 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:20:11.615197 osd.0 10.64.0.173:6822/19826 232 : [WRN] slow request 60.094327 seconds old, received at 2013-05-16 18:19:11.520736: osd_op(client.27297.0:50132407 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:11.615203 osd.0 10.64.0.173:6822/19826 233 : [WRN] slow request 60.090979 seconds old, received at 2013-05-16 18:19:11.524084: osd_op(client.25068.0:11837390 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:11.615208 osd.0 10.64.0.173:6822/19826 234 : [WRN] slow request 60.067866 seconds old, received at 2013-05-16 18:19:11.547197: osd_op(client.27297.0:50132408 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:11.615214 osd.0 10.64.0.173:6822/19826 235 : [WRN] slow request 31.630263 seconds old, received at 2013-05-16 18:19:39.984800: osd_op(client.27152.0:11793566 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat,read 0~524288] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:20:12.615433 osd.0 10.64.0.173:6822/19826 236 : [WRN] 97 slow requests, 4 included below; oldest blocked for > 131.939830 secs
2013-05-16 18:20:12.615443 osd.0 10.64.0.173:6822/19826 237 : [WRN] slow request 60.383521 seconds old, received at 2013-05-16 18:19:12.231829: osd_op(client.27297.0:50132445 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:12.615448 osd.0 10.64.0.173:6822/19826 238 : [WRN] slow request 32.616068 seconds old, received at 2013-05-16 18:19:39.999282: osd_op(client.27495.0:5297745 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:20:12.615452 osd.0 10.64.0.173:6822/19826 239 : [WRN] slow request 31.058434 seconds old, received at 2013-05-16 18:19:41.556916: osd_op(client.27152.0:11793588 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:12.615456 osd.0 10.64.0.173:6822/19826 240 : [WRN] slow request 31.055661 seconds old, received at 2013-05-16 18:19:41.559689: osd_op(client.27495.0:5297789 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:13.615642 osd.0 10.64.0.173:6822/19826 241 : [WRN] 100 slow requests, 6 included below; oldest blocked for > 132.940050 secs
2013-05-16 18:20:13.615654 osd.0 10.64.0.173:6822/19826 242 : [WRN] slow request 60.915553 seconds old, received at 2013-05-16 18:19:12.700017: osd_op(client.27297.0:50132459 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:20:13.615659 osd.0 10.64.0.173:6822/19826 243 : [WRN] slow request 60.776971 seconds old, received at 2013-05-16 18:19:12.838599: osd_op(client.27297.0:50132463 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:13.615664 osd.0 10.64.0.173:6822/19826 244 : [WRN] slow request 60.557644 seconds old, received at 2013-05-16 18:19:13.057926: osd_op(client.25068.0:11837432 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:13.615667 osd.0 10.64.0.173:6822/19826 245 : [WRN] slow request 30.742970 seconds old, received at 2013-05-16 18:19:42.872600: osd_op(client.27495.0:5297853 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:13.615671 osd.0 10.64.0.173:6822/19826 246 : [WRN] slow request 30.519988 seconds old, received at 2013-05-16 18:19:43.095582: osd_op(client.25068.0:11838000 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:14.736692 osd.0 10.64.0.173:6822/19826 247 : [WRN] 101 slow requests, 3 included below; oldest blocked for > 134.061057 secs
2013-05-16 18:20:14.736703 osd.0 10.64.0.173:6822/19826 248 : [WRN] slow request 60.571016 seconds old, received at 2013-05-16 18:19:14.165561: osd_op(client.27152.0:11792997 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:14.736710 osd.0 10.64.0.173:6822/19826 249 : [WRN] slow request 31.195364 seconds old, received at 2013-05-16 18:19:43.541213: osd_op(client.27152.0:11793646 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:14.736716 osd.0 10.64.0.173:6822/19826 250 : [WRN] slow request 30.291119 seconds old, received at 2013-05-16 18:19:44.445458: osd_sub_op(client.25068.0:11838030 3.2d0b 92652d0b/4143.159_9/99/CFZ_PRESS02.jpg/head//3 [] v 180331'104505 snapset=0=[]:[] snapc=0=[]) v7 currently commit sent
2013-05-16 18:20:16.864126 osd.0 10.64.0.173:6822/19826 251 : [WRN] 104 slow requests, 6 included below; oldest blocked for > 136.188478 secs
2013-05-16 18:20:16.864137 osd.0 10.64.0.173:6822/19826 252 : [WRN] slow request 121.055442 seconds old, received at 2013-05-16 18:18:15.808556: osd_op(client.27495.0:5294799 18499.46_3/39/Phantasm_II.jpg/293px-Phantasm_II.jpg [getxattrs,stat,read 0~524288] 3.e61713d6) v4 currently no flag points reached
2013-05-16 18:20:16.864146 osd.0 10.64.0.173:6822/19826 253 : [WRN] slow request 120.421026 seconds old, received at 2013-05-16 18:18:16.442972: osd_op(client.27152.0:11789852 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:16.864152 osd.0 10.64.0.173:6822/19826 254 : [WRN] slow request 120.037662 seconds old, received at 2013-05-16 18:18:16.826336: osd_op(client.25068.0:11835001 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:16.864158 osd.0 10.64.0.173:6822/19826 255 : [WRN] slow request 30.782723 seconds old, received at 2013-05-16 18:19:46.081275: osd_op(client.27152.0:11793704 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:16.864164 osd.0 10.64.0.173:6822/19826 256 : [WRN] slow request 30.437770 seconds old, received at 2013-05-16 18:19:46.426228: osd_op(client.27495.0:5297990 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:20:17.864386 osd.0 10.64.0.173:6822/19826 257 : [WRN] 105 slow requests, 3 included below; oldest blocked for > 137.188794 secs
2013-05-16 18:20:17.864400 osd.0 10.64.0.173:6822/19826 258 : [WRN] slow request 60.692456 seconds old, received at 2013-05-16 18:19:17.171858: osd_op(client.27152.0:11793045 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:17.864404 osd.0 10.64.0.173:6822/19826 259 : [WRN] slow request 31.358266 seconds old, received at 2013-05-16 18:19:46.506048: osd_op(client.25068.0:11838079 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:17.864411 osd.0 10.64.0.173:6822/19826 260 : [WRN] slow request 31.343368 seconds old, received at 2013-05-16 18:19:46.520946: osd_op(client.27152.0:11793718 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:18.864672 osd.0 10.64.0.173:6822/19826 261 : [WRN] 107 slow requests, 4 included below; oldest blocked for > 138.189050 secs
2013-05-16 18:20:18.864683 osd.0 10.64.0.173:6822/19826 262 : [WRN] slow request 120.083726 seconds old, received at 2013-05-16 18:18:18.780844: osd_sub_op(client.27152.0:11789995 3.1551 8552d551/10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png/head//3 [] v 180331'19700 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:20:18.864689 osd.0 10.64.0.173:6822/19826 263 : [WRN] slow request 60.116737 seconds old, received at 2013-05-16 18:19:18.747833: osd_op(client.27495.0:5296990 6900.31_7/79/Birch_House_Farm_-_geograph.org.uk_-_1204925.jpg [getxattrs,stat,read 0~524288] 3.efdae9e0) v4 currently no flag points reached
2013-05-16 18:20:18.864695 osd.0 10.64.0.173:6822/19826 264 : [WRN] slow request 30.808953 seconds old, received at 2013-05-16 18:19:48.055617: osd_op(client.27495.0:5298102 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:18.864701 osd.0 10.64.0.173:6822/19826 265 : [WRN] slow request 30.647634 seconds old, received at 2013-05-16 18:19:48.216936: osd_sub_op(client.27152.0:11793771 3.3363 62347363/10267.605_9/9a/Tribuna_di_galileo,_medaglioni_05.JPG/75px-Tribuna_di_galileo,_medaglioni_05.JPG/head//3 [] v 180331'19844 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:20.810126 mon.0 10.64.0.167:6789/0 456681 : [INF] pgmap v7225047: 16760 pgs: 198 active, 16548 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 44809B/s wr, 12op/s; 2/806457011 degraded (0.000%)
2013-05-16 18:20:19.177998 osd.74 10.64.32.12:6814/3431 8872 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.963142 secs
2013-05-16 18:20:19.178048 osd.74 10.64.32.12:6814/3431 8873 : [WRN] slow request 30.963142 seconds old, received at 2013-05-16 18:19:48.214765: osd_op(client.27152.0:11793771 10267.605_9/9a/Tribuna_di_galileo,_medaglioni_05.JPG/75px-Tribuna_di_galileo,_medaglioni_05.JPG [create 0~0,delete,writefull 0~3003,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (64),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.62347363) v4 currently waiting for subops from [24,0]
2013-05-16 18:20:22.037322 osd.28 10.64.0.175:6803/12389 693 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.518882 secs
2013-05-16 18:20:22.037339 osd.28 10.64.0.175:6803/12389 694 : [WRN] slow request 30.518882 seconds old, received at 2013-05-16 18:19:51.518362: osd_op(client.27297.0:50133628 .dir.4320.35 [call rgw.bucket_prepare_op] 3.a0666da3) v4 currently waiting for subops from [0,125]
2013-05-16 18:20:22.191604 mon.0 10.64.0.167:6789/0 456682 : [INF] pgmap v7225048: 16760 pgs: 197 active, 16549 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 850KB/s wr, 23op/s; 2/806457023 degraded (0.000%)
2013-05-16 18:20:23.647988 mon.0 10.64.0.167:6789/0 456683 : [INF] pgmap v7225049: 16760 pgs: 196 active, 16550 active+clean, 1 peering, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 125KB/s wr, 20op/s; 2/806457035 degraded (0.000%)
2013-05-16 18:20:19.609286 osd.56 10.64.32.10:6857/17945 1785 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 120.830640 secs
2013-05-16 18:20:19.609299 osd.56 10.64.32.10:6857/17945 1786 : [WRN] slow request 120.830640 seconds old, received at 2013-05-16 18:18:18.778569: osd_op(client.27152.0:11789995 10267.453_1/1e/Cyprus-Paphos.png/420px-Cyprus-Paphos.png [create 0~0,delete,writefull 0~37036,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (42),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.8552d551) v4 currently waiting for subops from [37,0]
2013-05-16 18:20:21.830120 osd.50 10.64.32.10:6833/17728 1963 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 120.612471 secs
2013-05-16 18:20:21.830135 osd.50 10.64.32.10:6833/17728 1964 : [WRN] slow request 120.612471 seconds old, received at 2013-05-16 18:18:21.217583: osd_op(client.27297.0:50130173 10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg [create 0~0,delete,writefull 0~3329,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (97),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.abdaea43) v4 currently waiting for subops from [0,100]
2013-05-16 18:20:25.046008 mon.0 10.64.0.167:6789/0 456684 : [INF] pgmap v7225050: 16760 pgs: 196 active, 16550 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 256KB/s wr, 17op/s; 3/806457044 degraded (0.000%); recovering 0 o/s, 0B/s, 14386 key/s
2013-05-16 18:20:26.568516 mon.0 10.64.0.167:6789/0 456685 : [INF] pgmap v7225051: 16760 pgs: 192 active, 16554 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3639B/s wr, 4op/s; 3/806457047 degraded (0.000%)
2013-05-16 18:20:28.034482 mon.0 10.64.0.167:6789/0 456686 : [INF] pgmap v7225052: 16760 pgs: 191 active, 16555 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 22188B/s wr, 13op/s; 3/806457056 degraded (0.000%)
2013-05-16 18:20:19.864912 osd.0 10.64.0.173:6822/19826 266 : [WRN] 108 slow requests, 2 included below; oldest blocked for > 139.189318 secs
2013-05-16 18:20:19.864921 osd.0 10.64.0.173:6822/19826 267 : [WRN] slow request 120.296485 seconds old, received at 2013-05-16 18:18:19.568353: osd_op(client.27297.0:50129950 21406.4154_69439ed9dd5cbc2402e9e2c8c62587b5.map [getxattrs,stat,read 0~524288] 3.3cc8f6fc) v4 currently no flag points reached
2013-05-16 18:20:19.864928 osd.0 10.64.0.173:6822/19826 268 : [WRN] slow request 30.075602 seconds old, received at 2013-05-16 18:19:49.789236: osd_op(client.27495.0:5298177 6900.1954_3/f/f/3ff46dd4c09b416a7d57c7457c0a2cb9.png [getxattrs,stat,read 0~524288] 3.e9a57159) v4 currently no flag points reached
2013-05-16 18:20:20.865175 osd.0 10.64.0.173:6822/19826 269 : [WRN] 109 slow requests, 3 included below; oldest blocked for > 140.189565 secs
2013-05-16 18:20:20.865185 osd.0 10.64.0.173:6822/19826 270 : [WRN] slow request 60.419290 seconds old, received at 2013-05-16 18:19:20.445795: osd_sub_op(client.25068.0:11837568 3.201a e5cd201a/10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/head//3 [] v 180331'21662 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:20:20.865189 osd.0 10.64.0.173:6822/19826 271 : [WRN] slow request 60.225684 seconds old, received at 2013-05-16 18:19:20.639401: osd_op(client.27297.0:50132610 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:20:20.865193 osd.0 10.64.0.173:6822/19826 272 : [WRN] slow request 30.881717 seconds old, received at 2013-05-16 18:19:49.983368: osd_op(client.27495.0:5298184 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat,read 0~524288] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:20:21.397946 osd.101 10.64.32.14:6818/2810 1854 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.953176 secs
2013-05-16 18:20:21.397956 osd.101 10.64.32.14:6818/2810 1855 : [WRN] slow request 60.953176 seconds old, received at 2013-05-16 18:19:20.444711: osd_op(client.25068.0:11837568 10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg [create 0~0,delete,writefull 0~2097,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (87),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.e5cd201a) v4 currently waiting for subops from [0,25]
2013-05-16 18:20:21.922798 osd.0 10.64.0.173:6822/19826 273 : [WRN] 110 slow requests, 3 included below; oldest blocked for > 141.247170 secs
2013-05-16 18:20:21.922812 osd.0 10.64.0.173:6822/19826 274 : [WRN] slow request 120.703862 seconds old, received at 2013-05-16 18:18:21.218828: osd_sub_op(client.27297.0:50130173 3.2a43 abdaea43/10267.620_9/94/Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/110px-Saddle_south_of_the_Worcestershire_Beacon_-_geograph.org.uk_-_645983.jpg/head//3 [] v 180331'22072 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:20:21.922818 osd.0 10.64.0.173:6822/19826 275 : [WRN] slow request 60.461023 seconds old, received at 2013-05-16 18:19:21.461667: osd_op(client.27152.0:11793151 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:21.922824 osd.0 10.64.0.173:6822/19826 276 : [WRN] slow request 30.365528 seconds old, received at 2013-05-16 18:19:51.557162: osd_sub_op(client.27297.0:50133628 3.2da3 a0666da3/.dir.4320.35/head//3 [] v 180331'182424 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:22.923069 osd.0 10.64.0.173:6822/19826 277 : [WRN] 112 slow requests, 4 included below; oldest blocked for > 142.247445 secs
2013-05-16 18:20:22.923081 osd.0 10.64.0.173:6822/19826 278 : [WRN] slow request 120.019149 seconds old, received at 2013-05-16 18:18:22.903816: osd_op(client.27152.0:11790463 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:20:22.923089 osd.0 10.64.0.173:6822/19826 279 : [WRN] slow request 60.069273 seconds old, received at 2013-05-16 18:19:22.853692: osd_op(client.27495.0:5297108 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:22.923096 osd.0 10.64.0.173:6822/19826 280 : [WRN] slow request 30.693266 seconds old, received at 2013-05-16 18:19:52.229699: osd_op(client.27152.0:11793912 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:22.923101 osd.0 10.64.0.173:6822/19826 281 : [WRN] slow request 30.034524 seconds old, received at 2013-05-16 18:19:52.888441: osd_op(client.27297.0:50133691 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:24.100264 osd.0 10.64.0.173:6822/19826 282 : [WRN] 113 slow requests, 1 included below; oldest blocked for > 143.424664 secs
2013-05-16 18:20:24.100274 osd.0 10.64.0.173:6822/19826 283 : [WRN] slow request 30.600670 seconds old, received at 2013-05-16 18:19:53.499514: osd_op(client.27297.0:50133705 10267.524_b/bb/Pat_Madden's_Plaque_-_geograph.org.uk_-_1658210.jpg/640px-Pat_Madden's_Plaque_-_geograph.org.uk_-_1658210.jpg [getxattrs,stat,read 0~524288] 3.9ba8ea80) v4 currently no flag points reached
2013-05-16 18:20:25.100502 osd.0 10.64.0.173:6822/19826 284 : [WRN] 113 slow requests, 3 included below; oldest blocked for > 144.424897 secs
2013-05-16 18:20:25.100511 osd.0 10.64.0.173:6822/19826 285 : [WRN] slow request 120.352218 seconds old, received at 2013-05-16 18:18:24.748199: osd_op(client.27152.0:11790516 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:25.100523 osd.0 10.64.0.173:6822/19826 286 : [WRN] slow request 60.194877 seconds old, received at 2013-05-16 18:19:24.905540: osd_op(client.25068.0:11837653 6900.1942_3/9/3/image_3336f9d6_3938ed8917ea182e.png [getxattrs,stat,read 0~524288] 3.1507b6c9) v4 currently no flag points reached
2013-05-16 18:20:25.100529 osd.0 10.64.0.173:6822/19826 287 : [WRN] slow request 60.024706 seconds old, received at 2013-05-16 18:19:25.075711: osd_op(client.27152.0:11793231 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat,read 0~524288] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:20:26.100688 osd.0 10.64.0.173:6822/19826 288 : [WRN] 114 slow requests, 1 included below; oldest blocked for > 145.425125 secs
2013-05-16 18:20:26.100695 osd.0 10.64.0.173:6822/19826 289 : [WRN] slow request 30.565683 seconds old, received at 2013-05-16 18:19:55.534962: osd_op(client.25068.0:11838444 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:20:27.100915 osd.0 10.64.0.173:6822/19826 290 : [WRN] 115 slow requests, 2 included below; oldest blocked for > 146.425315 secs
2013-05-16 18:20:27.100925 osd.0 10.64.0.173:6822/19826 291 : [WRN] slow request 60.678658 seconds old, received at 2013-05-16 18:19:26.422177: osd_op(client.27297.0:50132717 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:27.100932 osd.0 10.64.0.173:6822/19826 292 : [WRN] slow request 30.670754 seconds old, received at 2013-05-16 18:19:56.430081: osd_op(client.27152.0:11794035 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:28.101163 osd.0 10.64.0.173:6822/19826 293 : [WRN] 116 slow requests, 4 included below; oldest blocked for > 147.425548 secs
2013-05-16 18:20:28.101176 osd.0 10.64.0.173:6822/19826 294 : [WRN] slow request 120.874687 seconds old, received at 2013-05-16 18:18:27.226381: osd_op(client.25068.0:11835632 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:28.101183 osd.0 10.64.0.173:6822/19826 295 : [WRN] slow request 120.435577 seconds old, received at 2013-05-16 18:18:27.665491: osd_op(client.25068.0:11835683 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:28.101188 osd.0 10.64.0.173:6822/19826 296 : [WRN] slow request 120.114669 seconds old, received at 2013-05-16 18:18:27.986399: osd_op(client.27297.0:50130639 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:28.101196 osd.0 10.64.0.173:6822/19826 297 : [WRN] slow request 30.886221 seconds old, received at 2013-05-16 18:19:57.214847: osd_op(client.27495.0:5298387 21406.1730_e8c95a401e29cd7e9c06e276e65a45cb.err [getxattrs,stat,read 0~524288] 3.bc51352f) v4 currently no flag points reached
2013-05-16 18:20:29.101429 osd.0 10.64.0.173:6822/19826 298 : [WRN] 118 slow requests, 2 included below; oldest blocked for > 148.425826 secs
2013-05-16 18:20:29.101440 osd.0 10.64.0.173:6822/19826 299 : [WRN] slow request 30.861335 seconds old, received at 2013-05-16 18:19:58.240011: osd_op(client.25068.0:11838607 6900.1991_b/8/3/b8375e70b66148be8f6c766462fd4b37.png [getxattrs,stat,read 0~524288] 3.a1768795) v4 currently no flag points reached
2013-05-16 18:20:29.101455 osd.0 10.64.0.173:6822/19826 300 : [WRN] slow request 30.061692 seconds old, received at 2013-05-16 18:19:59.039654: osd_op(client.25068.0:11838665 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:29.666173 mon.0 10.64.0.167:6789/0 456687 : [INF] pgmap v7225053: 16760 pgs: 189 active, 16557 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 36761B/s wr, 30op/s; 3/806457065 degraded (0.000%)
2013-05-16 18:20:31.089354 mon.0 10.64.0.167:6789/0 456688 : [INF] pgmap v7225054: 16760 pgs: 184 active, 16562 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1892B/s wr, 4op/s; 3/806457068 degraded (0.000%)
2013-05-16 18:20:32.793142 mon.0 10.64.0.167:6789/0 456689 : [INF] pgmap v7225055: 16760 pgs: 181 active, 16565 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 394KB/s wr, 36op/s; 3/806457086 degraded (0.000%)
2013-05-16 18:20:34.307924 mon.0 10.64.0.167:6789/0 456690 : [INF] pgmap v7225056: 16760 pgs: 178 active, 16568 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3458KB/s wr, 28op/s; 3/806457110 degraded (0.000%)
2013-05-16 18:20:35.825869 mon.0 10.64.0.167:6789/0 456691 : [INF] pgmap v7225057: 16760 pgs: 178 active, 16568 active+clean, 1 active+recovery_wait, 13 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 23774B/s wr, 8op/s; 3/806457113 degraded (0.000%)
2013-05-16 18:20:37.157207 mon.0 10.64.0.167:6789/0 456692 : [INF] pgmap v7225058: 16760 pgs: 176 active, 16569 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 83581B/s wr, 13op/s; 3/806457122 degraded (0.000%)
2013-05-16 18:20:38.755852 mon.0 10.64.0.167:6789/0 456693 : [INF] pgmap v7225059: 16760 pgs: 172 active, 16573 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3512KB/s wr, 37op/s; 3/806457143 degraded (0.000%)
2013-05-16 18:20:30.101689 osd.0 10.64.0.173:6822/19826 301 : [WRN] 119 slow requests, 1 included below; oldest blocked for > 149.426100 secs
2013-05-16 18:20:30.101701 osd.0 10.64.0.173:6822/19826 302 : [WRN] slow request 30.584608 seconds old, received at 2013-05-16 18:19:59.517012: osd_op(client.27495.0:5298586 21406.4154_a8f01675114bec2866ba345f6bc2ec58.err [getxattrs,stat,read 0~524288] 3.3eddfb97) v4 currently no flag points reached
2013-05-16 18:20:31.101968 osd.0 10.64.0.173:6822/19826 303 : [WRN] 119 slow requests, 1 included below; oldest blocked for > 150.426364 secs
2013-05-16 18:20:31.101978 osd.0 10.64.0.173:6822/19826 304 : [WRN] slow request 60.005050 seconds old, received at 2013-05-16 18:19:31.096834: osd_op(client.27152.0:11793342 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:20:32.102239 osd.0 10.64.0.173:6822/19826 305 : [WRN] 121 slow requests, 5 included below; oldest blocked for > 151.426610 secs
2013-05-16 18:20:32.102249 osd.0 10.64.0.173:6822/19826 306 : [WRN] slow request 120.733789 seconds old, received at 2013-05-16 18:18:31.368341: osd_op(client.25068.0:11835805 21406.4154_7f677c7a48599cecce1055205e123bae.map [getxattrs,stat,read 0~524288] 3.38c4bd65) v4 currently no flag points reached
2013-05-16 18:20:32.102255 osd.0 10.64.0.173:6822/19826 307 : [WRN] slow request 60.204036 seconds old, received at 2013-05-16 18:19:31.898094: osd_op(client.27495.0:5297521 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:32.102260 osd.0 10.64.0.173:6822/19826 308 : [WRN] slow request 60.198805 seconds old, received at 2013-05-16 18:19:31.903325: osd_op(client.27297.0:50133033 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:32.102267 osd.0 10.64.0.173:6822/19826 309 : [WRN] slow request 30.969244 seconds old, received at 2013-05-16 18:20:01.132886: osd_op(client.27297.0:50134156 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:20:32.102272 osd.0 10.64.0.173:6822/19826 310 : [WRN] slow request 30.791805 seconds old, received at 2013-05-16 18:20:01.310325: osd_op(client.27152.0:11794583 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:33.102530 osd.0 10.64.0.173:6822/19826 311 : [WRN] 122 slow requests, 5 included below; oldest blocked for > 152.426906 secs
2013-05-16 18:20:33.102547 osd.0 10.64.0.173:6822/19826 312 : [WRN] slow request 120.209540 seconds old, received at 2013-05-16 18:18:32.892886: osd_op(client.27297.0:50130711 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:33.102554 osd.0 10.64.0.173:6822/19826 313 : [WRN] slow request 120.062295 seconds old, received at 2013-05-16 18:18:33.040131: osd_op(client.27152.0:11791365 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:20:33.102571 osd.0 10.64.0.173:6822/19826 314 : [WRN] slow request 60.985745 seconds old, received at 2013-05-16 18:19:32.116681: osd_op(client.27297.0:50133037 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:20:33.102575 osd.0 10.64.0.173:6822/19826 315 : [WRN] slow request 60.474650 seconds old, received at 2013-05-16 18:19:32.627776: osd_op(client.27495.0:5297532 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:20:33.102579 osd.0 10.64.0.173:6822/19826 316 : [WRN] slow request 30.608170 seconds old, received at 2013-05-16 18:20:02.494256: osd_op(client.25068.0:11838882 .dir.10267.604 [call rgw.bucket_prepare_op] 3.7d9ce9e0) v4 currently no flag points reached
2013-05-16 18:20:34.102809 osd.0 10.64.0.173:6822/19826 317 : [WRN] 124 slow requests, 3 included below; oldest blocked for > 153.427201 secs
2013-05-16 18:20:34.102819 osd.0 10.64.0.173:6822/19826 318 : [WRN] slow request 120.586491 seconds old, received at 2013-05-16 18:18:33.516230: osd_op(client.25068.0:11835885 6900.1942_7/7/1/image_61addc6d_7719271e4e57f271.png [getxattrs,stat,read 0~524288] 3.3d01c249) v4 currently no flag points reached
2013-05-16 18:20:34.102826 osd.0 10.64.0.173:6822/19826 319 : [WRN] slow request 30.679956 seconds old, received at 2013-05-16 18:20:03.422765: osd_op(client.25068.0:11838982 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:20:34.102832 osd.0 10.64.0.173:6822/19826 320 : [WRN] slow request 30.485389 seconds old, received at 2013-05-16 18:20:03.617332: osd_op(client.25068.0:11839003 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:35.225313 osd.0 10.64.0.173:6822/19826 321 : [WRN] 125 slow requests, 4 included below; oldest blocked for > 154.549718 secs
2013-05-16 18:20:35.225321 osd.0 10.64.0.173:6822/19826 322 : [WRN] slow request 120.777172 seconds old, received at 2013-05-16 18:18:34.448066: osd_op(client.27297.0:50130760 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:20:35.225324 osd.0 10.64.0.173:6822/19826 323 : [WRN] slow request 120.743168 seconds old, received at 2013-05-16 18:18:34.482070: osd_op(client.27297.0:50130763 .dir.10267.441 [call rgw.bucket_prepare_op] 3.c47fb39d) v4 currently no flag points reached
2013-05-16 18:20:35.225329 osd.0 10.64.0.173:6822/19826 324 : [WRN] slow request 60.240766 seconds old, received at 2013-05-16 18:19:34.984472: osd_op(client.27152.0:11793433 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:20:35.225335 osd.0 10.64.0.173:6822/19826 325 : [WRN] slow request 31.075505 seconds old, received at 2013-05-16 18:20:04.149733: osd_op(client.27152.0:11794747 6900.1878_7/4/f/74f765c63701e4a309f1f6213634edc8.png [getxattrs,stat,read 0~524288] 3.4626ce2e) v4 currently no flag points reached
2013-05-16 18:20:36.225675 osd.0 10.64.0.173:6822/19826 326 : [WRN] 127 slow requests, 5 included below; oldest blocked for > 155.550037 secs
2013-05-16 18:20:36.225683 osd.0 10.64.0.173:6822/19826 327 : [WRN] slow request 120.795529 seconds old, received at 2013-05-16 18:18:35.430028: osd_op(client.27297.0:50130785 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:36.225691 osd.0 10.64.0.173:6822/19826 328 : [WRN] slow request 120.671872 seconds old, received at 2013-05-16 18:18:35.553685: osd_op(client.25068.0:11835934 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:36.225698 osd.0 10.64.0.173:6822/19826 329 : [WRN] slow request 60.519623 seconds old, received at 2013-05-16 18:19:35.705934: osd_op(client.27152.0:11793455 21406.1730_d5d29fdc7a4f92c12cd935e043c6f46a.map [getxattrs,stat,read 0~524288] 3.c4a5e27a) v4 currently no flag points reached
2013-05-16 18:20:36.225707 osd.0 10.64.0.173:6822/19826 330 : [WRN] slow request 30.568847 seconds old, received at 2013-05-16 18:20:05.656710: osd_op(client.25068.0:11839063 18499.10_c/cb/JCVD_Poster.jpg/225px-JCVD_Poster.jpg [getxattrs,stat,read 0~524288] 3.4609fc1a) v4 currently no flag points reached
2013-05-16 18:20:36.225713 osd.0 10.64.0.173:6822/19826 331 : [WRN] slow request 30.147419 seconds old, received at 2013-05-16 18:20:06.078138: osd_op(client.25068.0:11839069 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:20:37.222592 osd.126 10.64.32.16:6845/1202 1530 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.615414 secs
2013-05-16 18:20:37.222619 osd.126 10.64.32.16:6845/1202 1531 : [WRN] slow request 30.615414 seconds old, received at 2013-05-16 18:20:06.606696: osd_op(client.27297.0:50134323 10267.473_f/fe/Plan_de_Ayutla_Location.png/180px-Plan_de_Ayutla_Location.png [create 0~0,delete,writefull 0~42937,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (52),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.d5a53b9c) v4 currently waiting for subops from [0,46]
2013-05-16 18:20:37.225941 osd.0 10.64.0.173:6822/19826 332 : [WRN] 128 slow requests, 3 included below; oldest blocked for > 156.550329 secs
2013-05-16 18:20:37.225951 osd.0 10.64.0.173:6822/19826 333 : [WRN] slow request 120.923588 seconds old, received at 2013-05-16 18:18:36.302261: osd_op(client.27152.0:11791499 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:20:37.225958 osd.0 10.64.0.173:6822/19826 334 : [WRN] slow request 120.640991 seconds old, received at 2013-05-16 18:18:36.584858: osd_op(client.27495.0:5295666 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:37.225964 osd.0 10.64.0.173:6822/19826 335 : [WRN] slow request 30.614546 seconds old, received at 2013-05-16 18:20:06.611303: osd_sub_op(client.27297.0:50134323 3.3b9c d5a53b9c/10267.473_f/fe/Plan_de_Ayutla_Location.png/180px-Plan_de_Ayutla_Location.png/head//3 [] v 180331'19652 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:38.226218 osd.0 10.64.0.173:6822/19826 336 : [WRN] 90 slow requests, 6 included below; oldest blocked for > 157.550585 secs
2013-05-16 18:20:38.226228 osd.0 10.64.0.173:6822/19826 337 : [WRN] slow request 120.383897 seconds old, received at 2013-05-16 18:18:37.842208: osd_op(client.25068.0:11835984 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:38.226234 osd.0 10.64.0.173:6822/19826 338 : [WRN] slow request 120.358777 seconds old, received at 2013-05-16 18:18:37.867328: osd_op(client.25068.0:11835986 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:38.226240 osd.0 10.64.0.173:6822/19826 339 : [WRN] slow request 120.005450 seconds old, received at 2013-05-16 18:18:38.220655: osd_op(client.25068.0:11835994 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:38.226246 osd.0 10.64.0.173:6822/19826 340 : [WRN] slow request 60.560351 seconds old, received at 2013-05-16 18:19:37.665754: osd_op(client.27152.0:11793522 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:20:38.226253 osd.0 10.64.0.173:6822/19826 341 : [WRN] slow request 60.442907 seconds old, received at 2013-05-16 18:19:37.783198: osd_op(client.27495.0:5297687 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:39.226478 osd.0 10.64.0.173:6822/19826 342 : [WRN] 92 slow requests, 6 included below; oldest blocked for > 158.550852 secs
2013-05-16 18:20:39.226488 osd.0 10.64.0.173:6822/19826 343 : [WRN] slow request 120.996736 seconds old, received at 2013-05-16 18:18:38.229636: osd_op(client.27495.0:5295709 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:20:39.226497 osd.0 10.64.0.173:6822/19826 344 : [WRN] slow request 120.028137 seconds old, received at 2013-05-16 18:18:39.198235: osd_op(client.27495.0:5295737 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:20:39.226503 osd.0 10.64.0.173:6822/19826 345 : [WRN] slow request 120.026589 seconds old, received at 2013-05-16 18:18:39.199783: osd_op(client.27297.0:50130869 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:20:39.226509 osd.0 10.64.0.173:6822/19826 346 : [WRN] slow request 61.020021 seconds old, received at 2013-05-16 18:19:38.206351: osd_op(client.25068.0:11837884 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:20:39.226514 osd.0 10.64.0.173:6822/19826 347 : [WRN] slow request 60.993932 seconds old, received at 2013-05-16 18:19:38.232440: osd_op(client.27495.0:5297690 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:40.162152 mon.0 10.64.0.167:6789/0 456694 : [INF] pgmap v7225060: 16760 pgs: 169 active, 16576 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 17163B/s wr, 4op/s; 3/806457146 degraded (0.000%)
2013-05-16 18:20:41.643847 mon.0 10.64.0.167:6789/0 456695 : [INF] pgmap v7225061: 16760 pgs: 167 active, 16578 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 387KB/s wr, 17op/s; 3/806457152 degraded (0.000%)
2013-05-16 18:20:43.008593 mon.0 10.64.0.167:6789/0 456696 : [INF] pgmap v7225062: 16760 pgs: 165 active, 16580 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 2171KB/s wr, 36op/s; 3/806457164 degraded (0.000%)
2013-05-16 18:20:44.526009 mon.0 10.64.0.167:6789/0 456697 : [INF] pgmap v7225063: 16760 pgs: 164 active, 16581 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 46974B/s wr, 25op/s; 3/806457179 degraded (0.000%)
2013-05-16 18:20:45.971684 mon.0 10.64.0.167:6789/0 456698 : [INF] pgmap v7225064: 16760 pgs: 164 active, 16581 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3556KB/s wr, 62op/s; 3/806457209 degraded (0.000%)
2013-05-16 18:20:47.462888 mon.0 10.64.0.167:6789/0 456699 : [INF] pgmap v7225065: 16760 pgs: 162 active, 16583 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 140KB/s wr, 19op/s; 3/806457221 degraded (0.000%)
2013-05-16 18:20:40.564633 osd.60 10.64.32.11:6800/8782 1533 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.234023 secs
2013-05-16 18:20:40.564742 osd.60 10.64.32.11:6800/8782 1534 : [WRN] slow request 30.234023 seconds old, received at 2013-05-16 18:20:10.330529: osd_op(client.25068.0:11839182 10267.624_2/2e/Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/120px-Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg [create 0~0,delete,writefull 0~5131,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (92),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.ab888ed0) v4 currently waiting for subops from [133,0]
2013-05-16 18:20:48.893773 mon.0 10.64.0.167:6789/0 456700 : [INF] pgmap v7225066: 16760 pgs: 160 active, 16585 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 258KB/s wr, 23op/s; 3/806457236 degraded (0.000%)
2013-05-16 18:20:40.226770 osd.0 10.64.0.173:6822/19826 348 : [WRN] 129 slow requests, 6 included below; oldest blocked for > 159.551145 secs
2013-05-16 18:20:40.226780 osd.0 10.64.0.173:6822/19826 349 : [WRN] slow request 120.638902 seconds old, received at 2013-05-16 18:18:39.587763: osd_op(client.25068.0:11836048 21406.10171_da635d1178177192feeca19830761c79.map [getxattrs,stat,read 0~524288] 3.11457cb) v4 currently no flag points reached
2013-05-16 18:20:40.226790 osd.0 10.64.0.173:6822/19826 350 : [WRN] slow request 61.923197 seconds old, received at 2013-05-16 18:19:38.303468: osd_op(client.27495.0:5297693 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:40.226807 osd.0 10.64.0.173:6822/19826 351 : [WRN] slow request 60.390077 seconds old, received at 2013-05-16 18:19:39.836588: osd_op(client.27297.0:50133216 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:20:40.226812 osd.0 10.64.0.173:6822/19826 352 : [WRN] slow request 60.241865 seconds old, received at 2013-05-16 18:19:39.984800: osd_op(client.27152.0:11793566 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat,read 0~524288] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:20:40.226816 osd.0 10.64.0.173:6822/19826 353 : [WRN] slow request 60.227383 seconds old, received at 2013-05-16 18:19:39.999282: osd_op(client.27495.0:5297745 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:20:41.227141 osd.0 10.64.0.173:6822/19826 354 : [WRN] 132 slow requests, 6 included below; oldest blocked for > 160.551512 secs
2013-05-16 18:20:41.227154 osd.0 10.64.0.173:6822/19826 355 : [WRN] slow request 120.224420 seconds old, received at 2013-05-16 18:18:41.002612: osd_op(client.25068.0:11836114 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:20:41.227161 osd.0 10.64.0.173:6822/19826 356 : [WRN] slow request 120.101909 seconds old, received at 2013-05-16 18:18:41.125123: osd_op(client.27152.0:11791706 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:41.227170 osd.0 10.64.0.173:6822/19826 357 : [WRN] slow request 33.795697 seconds old, received at 2013-05-16 18:20:07.431335: osd_op(client.27152.0:11794857 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:20:41.227175 osd.0 10.64.0.173:6822/19826 358 : [WRN] slow request 30.894865 seconds old, received at 2013-05-16 18:20:10.332167: osd_sub_op(client.25068.0:11839182 3.ed0 ab888ed0/10267.624_2/2e/Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/120px-Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/head//3 [] v 180331'20235 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:41.227180 osd.0 10.64.0.173:6822/19826 359 : [WRN] slow request 30.646838 seconds old, received at 2013-05-16 18:20:10.580194: osd_op(client.27297.0:50134411 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:20:42.505144 osd.0 10.64.0.173:6822/19826 360 : [WRN] 134 slow requests, 5 included below; oldest blocked for > 161.829495 secs
2013-05-16 18:20:42.505158 osd.0 10.64.0.173:6822/19826 361 : [WRN] slow request 60.948099 seconds old, received at 2013-05-16 18:19:41.556916: osd_op(client.27152.0:11793588 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:42.505165 osd.0 10.64.0.173:6822/19826 362 : [WRN] slow request 60.945326 seconds old, received at 2013-05-16 18:19:41.559689: osd_op(client.27495.0:5297789 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:42.505170 osd.0 10.64.0.173:6822/19826 363 : [WRN] slow request 31.916477 seconds old, received at 2013-05-16 18:20:10.588538: osd_op(client.25068.0:11839187 .dir.10267.604 [call rgw.bucket_prepare_op] 3.7d9ce9e0) v4 currently no flag points reached
2013-05-16 18:20:42.505180 osd.0 10.64.0.173:6822/19826 364 : [WRN] slow request 31.391247 seconds old, received at 2013-05-16 18:20:11.113768: osd_op(client.27495.0:5299302 21406.4154_a6d4ddc2fb4e8341ab71859f41ecbf68.map [getxattrs,stat,read 0~524288] 3.a39ee7de) v4 currently no flag points reached
2013-05-16 18:20:42.505186 osd.0 10.64.0.173:6822/19826 365 : [WRN] slow request 30.986349 seconds old, received at 2013-05-16 18:20:11.518666: osd_op(client.27152.0:11794987 18499.51_6/63/INTI_International_University_Logo.png/545px-INTI_International_University_Logo.png [getxattrs,stat,read 0~524288] 3.cc04ae7d) v4 currently no flag points reached
2013-05-16 18:20:43.505423 osd.0 10.64.0.173:6822/19826 366 : [WRN] 135 slow requests, 6 included below; oldest blocked for > 162.829796 secs
2013-05-16 18:20:43.505432 osd.0 10.64.0.173:6822/19826 367 : [WRN] slow request 120.841341 seconds old, received at 2013-05-16 18:18:42.663975: osd_op(client.27495.0:5295831 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:20:43.505439 osd.0 10.64.0.173:6822/19826 368 : [WRN] slow request 120.831901 seconds old, received at 2013-05-16 18:18:42.673415: osd_op(client.25068.0:11836173 10267.546_e/e4/RodaliesBarcelona_L7_Logo.PNG/130px-RodaliesBarcelona_L7_Logo.PNG [getxattrs,stat,read 0~524288] 3.cc34712) v4 currently no flag points reached
2013-05-16 18:20:43.505445 osd.0 10.64.0.173:6822/19826 369 : [WRN] slow request 120.044075 seconds old, received at 2013-05-16 18:18:43.461241: osd_op(client.27495.0:5295844 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:20:43.505451 osd.0 10.64.0.173:6822/19826 370 : [WRN] slow request 60.632716 seconds old, received at 2013-05-16 18:19:42.872600: osd_op(client.27495.0:5297853 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:43.505457 osd.0 10.64.0.173:6822/19826 371 : [WRN] slow request 60.409734 seconds old, received at 2013-05-16 18:19:43.095582: osd_op(client.25068.0:11838000 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:44.505684 osd.0 10.64.0.173:6822/19826 372 : [WRN] 136 slow requests, 6 included below; oldest blocked for > 163.830070 secs
2013-05-16 18:20:44.505692 osd.0 10.64.0.173:6822/19826 373 : [WRN] slow request 120.801402 seconds old, received at 2013-05-16 18:18:43.704188: osd_op(client.27495.0:5295854 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:44.505696 osd.0 10.64.0.173:6822/19826 374 : [WRN] slow request 120.717759 seconds old, received at 2013-05-16 18:18:43.787831: osd_op(client.27152.0:11791795 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:44.505701 osd.0 10.64.0.173:6822/19826 375 : [WRN] slow request 60.964377 seconds old, received at 2013-05-16 18:19:43.541213: osd_op(client.27152.0:11793646 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:44.505704 osd.0 10.64.0.173:6822/19826 376 : [WRN] slow request 60.060132 seconds old, received at 2013-05-16 18:19:44.445458: osd_sub_op(client.25068.0:11838030 3.2d0b 92652d0b/4143.159_9/99/CFZ_PRESS02.jpg/head//3 [] v 180331'104505 snapset=0=[]:[] snapc=0=[]) v7 currently commit sent
2013-05-16 18:20:44.505707 osd.0 10.64.0.173:6822/19826 377 : [WRN] slow request 31.834245 seconds old, received at 2013-05-16 18:20:12.671345: osd_op(client.27495.0:5299348 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:45.964207 osd.0 10.64.0.173:6822/19826 378 : [WRN] 139 slow requests, 4 included below; oldest blocked for > 165.288572 secs
2013-05-16 18:20:45.964218 osd.0 10.64.0.173:6822/19826 379 : [WRN] slow request 33.261717 seconds old, received at 2013-05-16 18:20:12.702375: osd_op(client.25068.0:11839326 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:45.964226 osd.0 10.64.0.173:6822/19826 380 : [WRN] slow request 32.711661 seconds old, received at 2013-05-16 18:20:13.252431: osd_op(client.27152.0:11795042 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:45.964232 osd.0 10.64.0.173:6822/19826 381 : [WRN] slow request 32.221524 seconds old, received at 2013-05-16 18:20:13.742568: osd_op(client.27495.0:5299368 21406.4154_13741766be7abb8a96999f31219228db.map [getxattrs,stat,read 0~524288] 3.a8b03c1a) v4 currently no flag points reached
2013-05-16 18:20:45.964238 osd.0 10.64.0.173:6822/19826 382 : [WRN] slow request 31.910143 seconds old, received at 2013-05-16 18:20:14.053949: osd_op(client.25068.0:11839469 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:47.049120 osd.0 10.64.0.173:6822/19826 383 : [WRN] 104 slow requests, 6 included below; oldest blocked for > 166.373485 secs
2013-05-16 18:20:47.049130 osd.0 10.64.0.173:6822/19826 384 : [WRN] slow request 120.731606 seconds old, received at 2013-05-16 18:18:46.317399: osd_op(client.27152.0:11791973 4143.177_archive/3/36/20081029174247!Wappen_Ringsheim.png [getxattrs,stat,read 0~524288] 3.e7843420) v4 currently no flag points reached
2013-05-16 18:20:47.049137 osd.0 10.64.0.173:6822/19826 385 : [WRN] slow request 120.568620 seconds old, received at 2013-05-16 18:18:46.480385: osd_op(client.27152.0:11791990 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:47.049143 osd.0 10.64.0.173:6822/19826 386 : [WRN] slow request 120.004959 seconds old, received at 2013-05-16 18:18:47.044046: osd_op(client.27297.0:50131293 .dir.10267.540 [call rgw.bucket_prepare_op] 3.1d4d6874) v4 currently no flag points reached
2013-05-16 18:20:47.049150 osd.0 10.64.0.173:6822/19826 387 : [WRN] slow request 60.967730 seconds old, received at 2013-05-16 18:19:46.081275: osd_op(client.27152.0:11793704 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:47.049156 osd.0 10.64.0.173:6822/19826 388 : [WRN] slow request 60.622777 seconds old, received at 2013-05-16 18:19:46.426228: osd_op(client.27495.0:5297990 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:20:48.238953 osd.0 10.64.0.173:6822/19826 389 : [WRN] 140 slow requests, 6 included below; oldest blocked for > 167.563292 secs
2013-05-16 18:20:48.238963 osd.0 10.64.0.173:6822/19826 390 : [WRN] slow request 120.276917 seconds old, received at 2013-05-16 18:18:47.961895: osd_op(client.27495.0:5296228 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:20:48.238971 osd.0 10.64.0.173:6822/19826 391 : [WRN] slow request 61.732764 seconds old, received at 2013-05-16 18:19:46.506048: osd_op(client.25068.0:11838079 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:48.238981 osd.0 10.64.0.173:6822/19826 392 : [WRN] slow request 61.717866 seconds old, received at 2013-05-16 18:19:46.520946: osd_op(client.27152.0:11793718 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:48.238988 osd.0 10.64.0.173:6822/19826 393 : [WRN] slow request 60.183195 seconds old, received at 2013-05-16 18:19:48.055617: osd_op(client.27495.0:5298102 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:48.238995 osd.0 10.64.0.173:6822/19826 394 : [WRN] slow request 60.021876 seconds old, received at 2013-05-16 18:19:48.216936: osd_sub_op(client.27152.0:11793771 3.3363 62347363/10267.605_9/9a/Tribuna_di_galileo,_medaglioni_05.JPG/75px-Tribuna_di_galileo,_medaglioni_05.JPG/head//3 [] v 180331'19844 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:48.893773 mon.0 10.64.0.167:6789/0 456700 : [INF] pgmap v7225066: 16760 pgs: 160 active, 16585 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 258KB/s wr, 23op/s; 3/806457236 degraded (0.000%)
2013-05-16 18:20:49.440689 osd.0 10.64.0.173:6822/19826 395 : [WRN] 143 slow requests, 5 included below; oldest blocked for > 168.765078 secs
2013-05-16 18:20:49.440697 osd.0 10.64.0.173:6822/19826 396 : [WRN] slow request 33.358837 seconds old, received at 2013-05-16 18:20:16.081761: osd_op(client.27297.0:50134562 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:20:49.440701 osd.0 10.64.0.173:6822/19826 397 : [WRN] slow request 32.945723 seconds old, received at 2013-05-16 18:20:16.494875: osd_op(client.25068.0:11839712 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:49.440711 osd.0 10.64.0.173:6822/19826 398 : [WRN] slow request 32.897936 seconds old, received at 2013-05-16 18:20:16.542662: osd_op(client.27495.0:5299471 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:49.440715 osd.0 10.64.0.173:6822/19826 399 : [WRN] slow request 32.885368 seconds old, received at 2013-05-16 18:20:16.555230: osd_op(client.27152.0:11795158 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:20:49.440720 osd.0 10.64.0.173:6822/19826 400 : [WRN] slow request 31.352034 seconds old, received at 2013-05-16 18:20:18.088564: osd_op(client.27297.0:50134619 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:50.465861 mon.0 10.64.0.167:6789/0 456701 : [INF] pgmap v7225067: 16760 pgs: 158 active, 16587 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 46249B/s wr, 9op/s; 3/806457242 degraded (0.000%)
2013-05-16 18:20:51.897522 mon.0 10.64.0.167:6789/0 456702 : [INF] pgmap v7225068: 16760 pgs: 156 active, 16589 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 2039KB/s wr, 37op/s; 3/806457263 degraded (0.000%)
2013-05-16 18:20:52.042462 osd.28 10.64.0.175:6803/12389 695 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.524047 secs
2013-05-16 18:20:52.042471 osd.28 10.64.0.175:6803/12389 696 : [WRN] slow request 60.524047 seconds old, received at 2013-05-16 18:19:51.518362: osd_op(client.27297.0:50133628 .dir.4320.35 [call rgw.bucket_prepare_op] 3.a0666da3) v4 currently waiting for subops from [0,125]
2013-05-16 18:20:49.184575 osd.74 10.64.32.12:6814/3431 8874 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.969742 secs
2013-05-16 18:20:49.184619 osd.74 10.64.32.12:6814/3431 8875 : [WRN] slow request 60.969742 seconds old, received at 2013-05-16 18:19:48.214765: osd_op(client.27152.0:11793771 10267.605_9/9a/Tribuna_di_galileo,_medaglioni_05.JPG/75px-Tribuna_di_galileo,_medaglioni_05.JPG [create 0~0,delete,writefull 0~3003,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (64),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.62347363) v4 currently waiting for subops from [24,0]
2013-05-16 18:20:53.503987 mon.0 10.64.0.167:6789/0 456703 : [INF] pgmap v7225069: 16760 pgs: 155 active, 16590 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3494KB/s wr, 21op/s; 3/806457278 degraded (0.000%)
2013-05-16 18:20:54.868137 mon.0 10.64.0.167:6789/0 456704 : [INF] pgmap v7225070: 16760 pgs: 153 active, 16592 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 431KB/s wr, 25op/s; 3/806457290 degraded (0.000%)
2013-05-16 18:20:50.476042 osd.38 10.64.0.176:6831/22256 1603 : [WRN] 3 slow requests, 2 included below; oldest blocked for > 198.838505 secs
2013-05-16 18:20:50.476059 osd.38 10.64.0.176:6831/22256 1604 : [WRN] slow request 30.658324 seconds old, received at 2013-05-16 18:20:19.817631: osd_op(client.27495.0:5299567 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [create 0~0,delete,writefull 0~1632,setxattr user.rgw.acl (123),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.59e4fd08) v4 currently waiting for subops from [56,0]
2013-05-16 18:20:50.476077 osd.38 10.64.0.176:6831/22256 1605 : [WRN] slow request 30.364193 seconds old, received at 2013-05-16 18:20:20.111762: osd_op(client.25068.0:11839883 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [create 0~0,delete,writefull 0~1632,setxattr user.rgw.acl (123),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.59e4fd08) v4 currently waiting for subops from [56,0]
2013-05-16 18:20:51.476340 osd.38 10.64.0.176:6831/22256 1606 : [WRN] 7 slow requests, 4 included below; oldest blocked for > 199.838798 secs
2013-05-16 18:20:51.476349 osd.38 10.64.0.176:6831/22256 1607 : [WRN] slow request 30.837185 seconds old, received at 2013-05-16 18:20:20.639063: osd_op(client.27297.0:50134711 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:51.476356 osd.38 10.64.0.176:6831/22256 1608 : [WRN] slow request 30.828428 seconds old, received at 2013-05-16 18:20:20.647820: osd_op(client.25068.0:11839898 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:51.476362 osd.38 10.64.0.176:6831/22256 1609 : [WRN] slow request 30.804807 seconds old, received at 2013-05-16 18:20:20.671441: osd_op(client.27297.0:50134713 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:51.476368 osd.38 10.64.0.176:6831/22256 1610 : [WRN] slow request 30.704890 seconds old, received at 2013-05-16 18:20:20.771358: osd_op(client.25068.0:11839902 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:56.349973 mon.0 10.64.0.167:6789/0 456705 : [INF] pgmap v7225071: 16760 pgs: 150 active, 16595 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 109KB/s wr, 31op/s; 3/806457308 degraded (0.000%)
2013-05-16 18:20:56.477525 osd.38 10.64.0.176:6831/22256 1611 : [WRN] 9 slow requests, 2 included below; oldest blocked for > 204.840003 secs
2013-05-16 18:20:56.477534 osd.38 10.64.0.176:6831/22256 1612 : [WRN] slow request 30.223710 seconds old, received at 2013-05-16 18:20:26.253743: osd_op(client.27495.0:5299738 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:56.477541 osd.38 10.64.0.176:6831/22256 1613 : [WRN] slow request 30.210356 seconds old, received at 2013-05-16 18:20:26.267097: osd_op(client.25068.0:11840100 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:20:57.839758 mon.0 10.64.0.167:6789/0 456706 : [INF] pgmap v7225072: 16760 pgs: 146 active, 16599 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 140KB/s wr, 18op/s; 1/806457320 degraded (0.000%)
2013-05-16 18:20:59.313354 mon.0 10.64.0.167:6789/0 456707 : [INF] pgmap v7225073: 16760 pgs: 143 active, 16601 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 179KB/s wr, 22op/s; 1/806457335 degraded (0.000%)
2013-05-16 18:20:50.441008 osd.0 10.64.0.173:6822/19826 401 : [WRN] 147 slow requests, 6 included below; oldest blocked for > 169.765361 secs
2013-05-16 18:20:50.441018 osd.0 10.64.0.173:6822/19826 402 : [WRN] slow request 60.651645 seconds old, received at 2013-05-16 18:19:49.789236: osd_op(client.27495.0:5298177 6900.1954_3/f/f/3ff46dd4c09b416a7d57c7457c0a2cb9.png [getxattrs,stat,read 0~524288] 3.e9a57159) v4 currently no flag points reached
2013-05-16 18:20:50.441030 osd.0 10.64.0.173:6822/19826 403 : [WRN] slow request 60.457513 seconds old, received at 2013-05-16 18:19:49.983368: osd_op(client.27495.0:5298184 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat,read 0~524288] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:20:50.441038 osd.0 10.64.0.173:6822/19826 404 : [WRN] slow request 30.340765 seconds old, received at 2013-05-16 18:20:20.100116: osd_sub_op(osd.38.0:9042983 3.3d08 a2457d08/.dir.10267.498/head//3 [push] v 180321'1594145 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:50.441049 osd.0 10.64.0.173:6822/19826 405 : [WRN] slow request 30.211916 seconds old, received at 2013-05-16 18:20:20.228965: osd_sub_op(client.27495.0:5299567 3.3d08 59e4fd08/6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png/head//3 [] v 180331'1594146 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:50.441059 osd.0 10.64.0.173:6822/19826 406 : [WRN] slow request 30.211819 seconds old, received at 2013-05-16 18:20:20.229062: osd_sub_op(client.25068.0:11839883 3.3d08 59e4fd08/6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png/head//3 [] v 180331'1594147 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:51.441258 osd.0 10.64.0.173:6822/19826 407 : [WRN] 149 slow requests, 5 included below; oldest blocked for > 170.765654 secs
2013-05-16 18:20:51.441265 osd.0 10.64.0.173:6822/19826 408 : [WRN] slow request 120.837052 seconds old, received at 2013-05-16 18:18:50.604122: osd_op(client.27152.0:11792263 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:20:51.441270 osd.0 10.64.0.173:6822/19826 409 : [WRN] slow request 120.015495 seconds old, received at 2013-05-16 18:18:51.425679: osd_op(client.27495.0:5296308 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:51.441273 osd.0 10.64.0.173:6822/19826 410 : [WRN] slow request 31.188572 seconds old, received at 2013-05-16 18:20:20.252602: osd_op(client.27152.0:11795306 6900.1954_3/f/f/3ff46dd4c09b416a7d57c7457c0a2cb9.png [getxattrs,stat] 3.e9a57159) v4 currently no flag points reached
2013-05-16 18:20:51.441279 osd.0 10.64.0.173:6822/19826 411 : [WRN] slow request 31.000410 seconds old, received at 2013-05-16 18:20:20.440764: osd_op(client.25068.0:11839892 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:20:51.441284 osd.0 10.64.0.173:6822/19826 412 : [WRN] slow request 30.774664 seconds old, received at 2013-05-16 18:20:20.666510: osd_op(client.27495.0:5299594 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:20:52.552250 osd.0 10.64.0.173:6822/19826 413 : [WRN] 150 slow requests, 3 included below; oldest blocked for > 171.876614 secs
2013-05-16 18:20:52.552262 osd.0 10.64.0.173:6822/19826 414 : [WRN] slow request 60.994972 seconds old, received at 2013-05-16 18:19:51.557162: osd_sub_op(client.27297.0:50133628 3.2da3 a0666da3/.dir.4320.35/head//3 [] v 180331'182424 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:20:52.552269 osd.0 10.64.0.173:6822/19826 415 : [WRN] slow request 60.322435 seconds old, received at 2013-05-16 18:19:52.229699: osd_op(client.27152.0:11793912 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:52.552292 osd.0 10.64.0.173:6822/19826 416 : [WRN] slow request 31.037906 seconds old, received at 2013-05-16 18:20:21.514228: osd_op(client.27297.0:50134778 18499.51_6/63/INTI_International_University_Logo.png/545px-INTI_International_University_Logo.png [getxattrs,stat,read 0~524288] 3.cc04ae7d) v4 currently no flag points reached
2013-05-16 18:20:53.552571 osd.0 10.64.0.173:6822/19826 417 : [WRN] 153 slow requests, 6 included below; oldest blocked for > 172.876921 secs
2013-05-16 18:20:53.552582 osd.0 10.64.0.173:6822/19826 418 : [WRN] slow request 120.609636 seconds old, received at 2013-05-16 18:18:52.942805: osd_op(client.27152.0:11792304 21406.10171_d92569b0c031d0765579382ea913ea2e.err [getxattrs,stat,read 0~524288] 3.f3e36929) v4 currently no flag points reached
2013-05-16 18:20:53.552589 osd.0 10.64.0.173:6822/19826 419 : [WRN] slow request 60.664000 seconds old, received at 2013-05-16 18:19:52.888441: osd_op(client.27297.0:50133691 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:53.552595 osd.0 10.64.0.173:6822/19826 420 : [WRN] slow request 60.052927 seconds old, received at 2013-05-16 18:19:53.499514: osd_op(client.27297.0:50133705 10267.524_b/bb/Pat_Madden's_Plaque_-_geograph.org.uk_-_1658210.jpg/640px-Pat_Madden's_Plaque_-_geograph.org.uk_-_1658210.jpg [getxattrs,stat,read 0~524288] 3.9ba8ea80) v4 currently no flag points reached
2013-05-16 18:20:53.552602 osd.0 10.64.0.173:6822/19826 421 : [WRN] slow request 30.457469 seconds old, received at 2013-05-16 18:20:23.094972: osd_op(client.25068.0:11840011 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:20:53.552609 osd.0 10.64.0.173:6822/19826 422 : [WRN] slow request 30.285065 seconds old, received at 2013-05-16 18:20:23.267376: osd_op(client.27495.0:5299649 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat,read 0~524288] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:20:54.552842 osd.0 10.64.0.173:6822/19826 423 : [WRN] 154 slow requests, 2 included below; oldest blocked for > 173.877231 secs
2013-05-16 18:20:54.552855 osd.0 10.64.0.173:6822/19826 424 : [WRN] slow request 31.265944 seconds old, received at 2013-05-16 18:20:23.286807: osd_op(client.27297.0:50134961 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:20:54.552861 osd.0 10.64.0.173:6822/19826 425 : [WRN] slow request 30.187225 seconds old, received at 2013-05-16 18:20:24.365526: osd_op(client.27495.0:5299670 4320.11_c/ce/Rimbaud_in_Museum.JPG [getxattrs,stat,read 0~524288] 3.597aa411) v4 currently no flag points reached
2013-05-16 18:20:55.553079 osd.0 10.64.0.173:6822/19826 426 : [WRN] 155 slow requests, 2 included below; oldest blocked for > 174.877471 secs
2013-05-16 18:20:55.553088 osd.0 10.64.0.173:6822/19826 427 : [WRN] slow request 60.018029 seconds old, received at 2013-05-16 18:19:55.534962: osd_op(client.25068.0:11838444 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:20:55.553097 osd.0 10.64.0.173:6822/19826 428 : [WRN] slow request 30.361496 seconds old, received at 2013-05-16 18:20:25.191495: osd_op(client.27152.0:11795436 21406.14196_0/01/Ctchannel3.png/720px-Ctchannel3.png [getxattrs,stat,read 0~524288] 3.c21c6411) v4 currently no flag points reached
2013-05-16 18:20:56.655461 osd.0 10.64.0.173:6822/19826 429 : [WRN] 156 slow requests, 2 included below; oldest blocked for > 175.979838 secs
2013-05-16 18:20:56.655473 osd.0 10.64.0.173:6822/19826 430 : [WRN] slow request 60.225277 seconds old, received at 2013-05-16 18:19:56.430081: osd_op(client.27152.0:11794035 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:56.655479 osd.0 10.64.0.173:6822/19826 431 : [WRN] slow request 30.032947 seconds old, received at 2013-05-16 18:20:26.622411: osd_op(client.25068.0:11840106 4320.16_7/7d/Ioannina_orismos_1430.jpg [getxattrs,stat,read 0~524288] 3.371d374b) v4 currently no flag points reached
2013-05-16 18:20:57.655808 osd.0 10.64.0.173:6822/19826 432 : [WRN] 158 slow requests, 3 included below; oldest blocked for > 176.980186 secs
2013-05-16 18:20:57.655819 osd.0 10.64.0.173:6822/19826 433 : [WRN] slow request 60.440859 seconds old, received at 2013-05-16 18:19:57.214847: osd_op(client.27495.0:5298387 21406.1730_e8c95a401e29cd7e9c06e276e65a45cb.err [getxattrs,stat,read 0~524288] 3.bc51352f) v4 currently no flag points reached
2013-05-16 18:20:57.655826 osd.0 10.64.0.173:6822/19826 434 : [WRN] slow request 30.838274 seconds old, received at 2013-05-16 18:20:26.817432: osd_op(client.27297.0:50135129 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:20:57.655833 osd.0 10.64.0.173:6822/19826 435 : [WRN] slow request 30.406351 seconds old, received at 2013-05-16 18:20:27.249355: osd_op(client.25068.0:11840120 21406.1730_e8c95a401e29cd7e9c06e276e65a45cb.err [getxattrs,stat,read 0~524288] 3.bc51352f) v4 currently no flag points reached
2013-05-16 18:20:58.656053 osd.0 10.64.0.173:6822/19826 436 : [WRN] 158 slow requests, 1 included below; oldest blocked for > 177.980452 secs
2013-05-16 18:20:58.656064 osd.0 10.64.0.173:6822/19826 437 : [WRN] slow request 60.415961 seconds old, received at 2013-05-16 18:19:58.240011: osd_op(client.25068.0:11838607 6900.1991_b/8/3/b8375e70b66148be8f6c766462fd4b37.png [getxattrs,stat,read 0~524288] 3.a1768795) v4 currently no flag points reached
2013-05-16 18:21:00.794881 mon.0 10.64.0.167:6789/0 456708 : [INF] pgmap v7225074: 16760 pgs: 143 active, 16601 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 15392B/s wr, 4op/s; 1/806457338 degraded (0.000%)
2013-05-16 18:21:02.351106 mon.0 10.64.0.167:6789/0 456709 : [INF] pgmap v7225075: 16760 pgs: 140 active, 16604 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4119B/s wr, 9op/s; 1/806457344 degraded (0.000%)
2013-05-16 18:21:03.882276 mon.0 10.64.0.167:6789/0 456710 : [INF] pgmap v7225076: 16760 pgs: 139 active, 16605 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 115KB/s wr, 27op/s; 1/806457362 degraded (0.000%)
2013-05-16 18:21:05.443978 mon.0 10.64.0.167:6789/0 456711 : [INF] pgmap v7225077: 16760 pgs: 136 active, 16608 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 35069B/s wr, 18op/s; 1/806457371 degraded (0.000%)
2013-05-16 18:21:05.479464 osd.38 10.64.0.176:6831/22256 1614 : [WRN] 10 slow requests, 1 included below; oldest blocked for > 213.841973 secs
2013-05-16 18:21:05.479470 osd.38 10.64.0.176:6831/22256 1615 : [WRN] slow request 30.852257 seconds old, received at 2013-05-16 18:20:34.627166: osd_op(client.25068.0:11840339 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:06.795460 mon.0 10.64.0.167:6789/0 456712 : [INF] pgmap v7225078: 16760 pgs: 134 active, 16610 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 16483B/s wr, 12op/s; 1/806457380 degraded (0.000%)
2013-05-16 18:21:08.326618 mon.0 10.64.0.167:6789/0 456713 : [INF] pgmap v7225079: 16760 pgs: 131 active, 16613 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 117KB/s wr, 25op/s; 1/806457395 degraded (0.000%)
2013-05-16 18:20:59.961682 osd.0 10.64.0.173:6822/19826 438 : [WRN] 162 slow requests, 6 included below; oldest blocked for > 179.286027 secs
2013-05-16 18:20:59.961701 osd.0 10.64.0.173:6822/19826 439 : [WRN] slow request 60.921893 seconds old, received at 2013-05-16 18:19:59.039654: osd_op(client.25068.0:11838665 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:20:59.961708 osd.0 10.64.0.173:6822/19826 440 : [WRN] slow request 60.444535 seconds old, received at 2013-05-16 18:19:59.517012: osd_op(client.27495.0:5298586 21406.4154_a8f01675114bec2866ba345f6bc2ec58.err [getxattrs,stat,read 0~524288] 3.3eddfb97) v4 currently no flag points reached
2013-05-16 18:20:59.961714 osd.0 10.64.0.173:6822/19826 441 : [WRN] slow request 31.168593 seconds old, received at 2013-05-16 18:20:28.792954: osd_op(client.27152.0:11795607 6900.1991_b/8/3/b8375e70b66148be8f6c766462fd4b37.png [getxattrs,stat] 3.a1768795) v4 currently no flag points reached
2013-05-16 18:20:59.961726 osd.0 10.64.0.173:6822/19826 442 : [WRN] slow request 30.935483 seconds old, received at 2013-05-16 18:20:29.026064: osd_op(client.25068.0:11840169 10267.546_e/e4/Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu/page1-375px-Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu.jpg [getxattrs,stat,read 0~524288] 3.556b58de) v4 currently no flag points reached
2013-05-16 18:20:59.961731 osd.0 10.64.0.173:6822/19826 443 : [WRN] slow request 30.420339 seconds old, received at 2013-05-16 18:20:29.541208: osd_op(client.25068.0:11840187 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:01.016285 osd.0 10.64.0.173:6822/19826 444 : [WRN] 162 slow requests, 2 included below; oldest blocked for > 180.340663 secs
2013-05-16 18:21:01.016305 osd.0 10.64.0.173:6822/19826 445 : [WRN] slow request 120.064269 seconds old, received at 2013-05-16 18:19:00.951914: osd_op(client.27152.0:11792707 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:01.016312 osd.0 10.64.0.173:6822/19826 446 : [WRN] slow request 31.409831 seconds old, received at 2013-05-16 18:20:29.606352: osd_op(client.27297.0:50135253 21406.4154_a8f01675114bec2866ba345f6bc2ec58.err [getxattrs,stat,read 0~524288] 3.3eddfb97) v4 currently no flag points reached
2013-05-16 18:21:02.016492 osd.0 10.64.0.173:6822/19826 447 : [WRN] 164 slow requests, 4 included below; oldest blocked for > 181.340892 secs
2013-05-16 18:21:02.016500 osd.0 10.64.0.173:6822/19826 448 : [WRN] slow request 60.883526 seconds old, received at 2013-05-16 18:20:01.132886: osd_op(client.27297.0:50134156 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:02.016504 osd.0 10.64.0.173:6822/19826 449 : [WRN] slow request 60.706087 seconds old, received at 2013-05-16 18:20:01.310325: osd_op(client.27152.0:11794583 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:02.016509 osd.0 10.64.0.173:6822/19826 450 : [WRN] slow request 30.768430 seconds old, received at 2013-05-16 18:20:31.247982: osd_sub_op(client.27297.0:50135283 3.3c78 7ee67c78/10267.505_8/8e/Coleophora_serenella.JPG/100px-Coleophora_serenella.JPG/head//3 [] v 180331'19701 snapset=0=[]:[] snapc=0=[]) v7 currently commit sent
2013-05-16 18:21:02.016514 osd.0 10.64.0.173:6822/19826 451 : [WRN] slow request 30.646031 seconds old, received at 2013-05-16 18:20:31.370381: osd_op(client.25068.0:11840220 6900.1942_7/f/c/image_abf01d00_7fcc279d4d760833.png [getxattrs,stat,read 0~524288] 3.c1dccee7) v4 currently no flag points reached
2013-05-16 18:21:03.016791 osd.0 10.64.0.173:6822/19826 452 : [WRN] 164 slow requests, 2 included below; oldest blocked for > 182.341184 secs
2013-05-16 18:21:03.016802 osd.0 10.64.0.173:6822/19826 453 : [WRN] slow request 120.424290 seconds old, received at 2013-05-16 18:19:02.592414: osd_op(client.27297.0:50132075 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:21:03.016808 osd.0 10.64.0.173:6822/19826 454 : [WRN] slow request 60.522448 seconds old, received at 2013-05-16 18:20:02.494256: osd_op(client.25068.0:11838882 .dir.10267.604 [call rgw.bucket_prepare_op] 3.7d9ce9e0) v4 currently no flag points reached
2013-05-16 18:21:04.017091 osd.0 10.64.0.173:6822/19826 455 : [WRN] 165 slow requests, 4 included below; oldest blocked for > 183.341449 secs
2013-05-16 18:21:04.017103 osd.0 10.64.0.173:6822/19826 456 : [WRN] slow request 120.733755 seconds old, received at 2013-05-16 18:19:03.283214: osd_op(client.27152.0:11792771 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:04.017118 osd.0 10.64.0.173:6822/19826 457 : [WRN] slow request 60.594204 seconds old, received at 2013-05-16 18:20:03.422765: osd_op(client.25068.0:11838982 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:21:04.017124 osd.0 10.64.0.173:6822/19826 458 : [WRN] slow request 60.399637 seconds old, received at 2013-05-16 18:20:03.617332: osd_op(client.25068.0:11839003 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:04.017130 osd.0 10.64.0.173:6822/19826 459 : [WRN] slow request 30.560097 seconds old, received at 2013-05-16 18:20:33.456872: osd_op(client.27297.0:50135351 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:21:05.017410 osd.0 10.64.0.173:6822/19826 460 : [WRN] 166 slow requests, 3 included below; oldest blocked for > 184.341785 secs
2013-05-16 18:21:05.017429 osd.0 10.64.0.173:6822/19826 461 : [WRN] slow request 120.529419 seconds old, received at 2013-05-16 18:19:04.487886: osd_op(client.25068.0:11837242 21406.4154_2ae09dcdec937314ed1d638afbbc654a.map [getxattrs,stat,read 0~524288] 3.3ff624b6) v4 currently no flag points reached
2013-05-16 18:21:05.017436 osd.0 10.64.0.173:6822/19826 462 : [WRN] slow request 60.867572 seconds old, received at 2013-05-16 18:20:04.149733: osd_op(client.27152.0:11794747 6900.1878_7/4/f/74f765c63701e4a309f1f6213634edc8.png [getxattrs,stat,read 0~524288] 3.4626ce2e) v4 currently no flag points reached
2013-05-16 18:21:05.017442 osd.0 10.64.0.173:6822/19826 463 : [WRN] slow request 30.355803 seconds old, received at 2013-05-16 18:20:34.661502: osd_op(client.25068.0:11840340 6900.1878_7/4/f/74f765c63701e4a309f1f6213634edc8.png [getxattrs,stat] 3.4626ce2e) v4 currently no flag points reached
2013-05-16 18:21:06.085518 osd.0 10.64.0.173:6822/19826 464 : [WRN] 168 slow requests, 4 included below; oldest blocked for > 185.409874 secs
2013-05-16 18:21:06.085527 osd.0 10.64.0.173:6822/19826 465 : [WRN] slow request 60.428684 seconds old, received at 2013-05-16 18:20:05.656710: osd_op(client.25068.0:11839063 18499.10_c/cb/JCVD_Poster.jpg/225px-JCVD_Poster.jpg [getxattrs,stat,read 0~524288] 3.4609fc1a) v4 currently no flag points reached
2013-05-16 18:21:06.085535 osd.0 10.64.0.173:6822/19826 466 : [WRN] slow request 60.007256 seconds old, received at 2013-05-16 18:20:06.078138: osd_op(client.25068.0:11839069 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:06.085540 osd.0 10.64.0.173:6822/19826 467 : [WRN] slow request 30.901330 seconds old, received at 2013-05-16 18:20:35.184064: osd_op(client.27152.0:11795788 21406.14196_0/01/Ctchannel3.png/720px-Ctchannel3.png [getxattrs,stat,read 0~524288] 3.c21c6411) v4 currently no flag points reached
2013-05-16 18:21:06.085546 osd.0 10.64.0.173:6822/19826 468 : [WRN] slow request 30.849982 seconds old, received at 2013-05-16 18:20:35.235412: osd_op(client.27495.0:5300055 21406.4154_d0b3ad5fdb77f600760b613aecdd4e6e.err [getxattrs,stat,read 0~524288] 3.49387b02) v4 currently no flag points reached
2013-05-16 18:21:07.085823 osd.0 10.64.0.173:6822/19826 469 : [WRN] 169 slow requests, 4 included below; oldest blocked for > 186.410159 secs
2013-05-16 18:21:07.085842 osd.0 10.64.0.173:6822/19826 470 : [WRN] slow request 120.747137 seconds old, received at 2013-05-16 18:19:06.338542: osd_op(client.27297.0:50132218 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:21:07.085849 osd.0 10.64.0.173:6822/19826 471 : [WRN] slow request 120.464915 seconds old, received at 2013-05-16 18:19:06.620764: osd_op(client.25068.0:11837301 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:07.085856 osd.0 10.64.0.173:6822/19826 472 : [WRN] slow request 60.474376 seconds old, received at 2013-05-16 18:20:06.611303: osd_sub_op(client.27297.0:50134323 3.3b9c d5a53b9c/10267.473_f/fe/Plan_de_Ayutla_Location.png/180px-Plan_de_Ayutla_Location.png/head//3 [] v 180331'19652 snapset=0=[]:[] snapc=0=[]) v7 currently commit sent
2013-05-16 18:21:07.085870 osd.0 10.64.0.173:6822/19826 473 : [WRN] slow request 30.970957 seconds old, received at 2013-05-16 18:20:36.114722: osd_op(client.27297.0:50135424 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:08.086141 osd.0 10.64.0.173:6822/19826 474 : [WRN] 171 slow requests, 6 included below; oldest blocked for > 187.410500 secs
2013-05-16 18:21:08.086152 osd.0 10.64.0.173:6822/19826 475 : [WRN] slow request 120.510934 seconds old, received at 2013-05-16 18:19:07.575086: osd_op(client.27152.0:11792858 10267.620_9/94/Bloomberg_tower.jpg/212px-Bloomberg_tower.jpg [getxattrs,stat,read 0~524288] 3.dd653d65) v4 currently no flag points reached
2013-05-16 18:21:08.086168 osd.0 10.64.0.173:6822/19826 476 : [WRN] slow request 120.207490 seconds old, received at 2013-05-16 18:19:07.878530: osd_op(client.27297.0:50132256 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:08.086174 osd.0 10.64.0.173:6822/19826 477 : [WRN] slow request 120.184972 seconds old, received at 2013-05-16 18:19:07.901048: osd_op(client.27297.0:50132258 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:08.086189 osd.0 10.64.0.173:6822/19826 478 : [WRN] slow request 60.654685 seconds old, received at 2013-05-16 18:20:07.431335: osd_op(client.27152.0:11794857 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:08.086195 osd.0 10.64.0.173:6822/19826 479 : [WRN] slow request 30.617559 seconds old, received at 2013-05-16 18:20:37.468461: osd_op(client.27152.0:11795843 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:09.302952 osd.0 10.64.0.173:6822/19826 480 : [WRN] 171 slow requests, 4 included below; oldest blocked for > 188.627297 secs
2013-05-16 18:21:09.302962 osd.0 10.64.0.173:6822/19826 481 : [WRN] slow request 121.130201 seconds old, received at 2013-05-16 18:19:08.172616: osd_op(client.25068.0:11837335 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:21:09.302970 osd.0 10.64.0.173:6822/19826 482 : [WRN] slow request 121.045534 seconds old, received at 2013-05-16 18:19:08.257283: osd_op(client.25068.0:11837336 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:09.302976 osd.0 10.64.0.173:6822/19826 483 : [WRN] slow request 121.035478 seconds old, received at 2013-05-16 18:19:08.267339: osd_op(client.27495.0:5296678 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:21:09.302983 osd.0 10.64.0.173:6822/19826 484 : [WRN] slow request 31.482990 seconds old, received at 2013-05-16 18:20:37.819827: osd_op(client.27297.0:50135464 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat,read 0~524288] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:21:09.753586 mon.0 10.64.0.167:6789/0 456714 : [INF] pgmap v7225080: 16760 pgs: 131 active, 16613 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 112KB/s wr, 9op/s; 1/806457401 degraded (0.000%)
2013-05-16 18:21:11.172783 mon.0 10.64.0.167:6789/0 456715 : [INF] pgmap v7225081: 16760 pgs: 130 active, 16614 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 407KB/s wr, 23op/s; 1/806457416 degraded (0.000%)
2013-05-16 18:21:10.059744 osd.3 10.64.0.173:6839/13356 1654 : [INF] 3.1da6 deep-scrub ok
2013-05-16 18:21:12.507355 mon.0 10.64.0.167:6789/0 456716 : [INF] pgmap v7225082: 16760 pgs: 127 active, 16617 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 942KB/s wr, 44op/s; 1/806457440 degraded (0.000%)
2013-05-16 18:21:14.030338 mon.0 10.64.0.167:6789/0 456717 : [INF] pgmap v7225083: 16760 pgs: 125 active, 16619 active+clean, 1 active+recovery_wait, 15 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3031KB/s wr, 11op/s; 1/806457446 degraded (0.000%)
2013-05-16 18:21:15.486243 mon.0 10.64.0.167:6789/0 456718 : [INF] pgmap v7225084: 16760 pgs: 124 active, 16621 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 384KB/s wr, 21op/s; 1/806457455 degraded (0.000%)
2013-05-16 18:21:16.933757 mon.0 10.64.0.167:6789/0 456719 : [INF] pgmap v7225085: 16760 pgs: 120 active, 16625 active+clean, 1 active+recovery_wait, 14 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 1995KB/s wr, 31op/s; 1/806457461 degraded (0.000%)
2013-05-16 18:21:10.570435 osd.60 10.64.32.11:6800/8782 1535 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.239841 secs
2013-05-16 18:21:10.570447 osd.60 10.64.32.11:6800/8782 1536 : [WRN] slow request 60.239841 seconds old, received at 2013-05-16 18:20:10.330529: osd_op(client.25068.0:11839182 10267.624_2/2e/Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/120px-Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg [create 0~0,delete,writefull 0~5131,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (92),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.ab888ed0) v4 currently waiting for subops from [133,0]
2013-05-16 18:21:18.398478 mon.0 10.64.0.167:6789/0 456720 : [INF] pgmap v7225086: 16760 pgs: 119 active, 16624 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3896KB/s wr, 35op/s; 1/806457464 degraded (0.000%)
2013-05-16 18:21:11.303515 osd.0 10.64.0.173:6822/19826 485 : [WRN] 172 slow requests, 6 included below; oldest blocked for > 190.627853 secs
2013-05-16 18:21:11.303526 osd.0 10.64.0.173:6822/19826 486 : [WRN] slow request 120.264803 seconds old, received at 2013-05-16 18:19:11.038570: osd_op(client.25068.0:11837372 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:21:11.303532 osd.0 10.64.0.173:6822/19826 487 : [WRN] slow request 60.971206 seconds old, received at 2013-05-16 18:20:10.332167: osd_sub_op(client.25068.0:11839182 3.ed0 ab888ed0/10267.624_2/2e/Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/120px-Ogród_roślinności_tatrzańskiej_kosodrzewina.jpg/head//3 [] v 180331'20235 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:11.303538 osd.0 10.64.0.173:6822/19826 488 : [WRN] slow request 60.723179 seconds old, received at 2013-05-16 18:20:10.580194: osd_op(client.27297.0:50134411 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:21:11.303544 osd.0 10.64.0.173:6822/19826 489 : [WRN] slow request 60.714835 seconds old, received at 2013-05-16 18:20:10.588538: osd_op(client.25068.0:11839187 .dir.10267.604 [call rgw.bucket_prepare_op] 3.7d9ce9e0) v4 currently no flag points reached
2013-05-16 18:21:11.303554 osd.0 10.64.0.173:6822/19826 490 : [WRN] slow request 60.189605 seconds old, received at 2013-05-16 18:20:11.113768: osd_op(client.27495.0:5299302 21406.4154_a6d4ddc2fb4e8341ab71859f41ecbf68.map [getxattrs,stat,read 0~524288] 3.a39ee7de) v4 currently no flag points reached
2013-05-16 18:21:12.303848 osd.0 10.64.0.173:6822/19826 491 : [WRN] 172 slow requests, 6 included below; oldest blocked for > 191.628181 secs
2013-05-16 18:21:12.303859 osd.0 10.64.0.173:6822/19826 492 : [WRN] slow request 120.782965 seconds old, received at 2013-05-16 18:19:11.520736: osd_op(client.27297.0:50132407 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:12.303866 osd.0 10.64.0.173:6822/19826 493 : [WRN] slow request 120.779617 seconds old, received at 2013-05-16 18:19:11.524084: osd_op(client.25068.0:11837390 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:12.303872 osd.0 10.64.0.173:6822/19826 494 : [WRN] slow request 120.756504 seconds old, received at 2013-05-16 18:19:11.547197: osd_op(client.27297.0:50132408 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:12.303877 osd.0 10.64.0.173:6822/19826 495 : [WRN] slow request 120.071872 seconds old, received at 2013-05-16 18:19:12.231829: osd_op(client.27297.0:50132445 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:12.303885 osd.0 10.64.0.173:6822/19826 496 : [WRN] slow request 60.785035 seconds old, received at 2013-05-16 18:20:11.518666: osd_op(client.27152.0:11794987 18499.51_6/63/INTI_International_University_Logo.png/545px-INTI_International_University_Logo.png [getxattrs,stat,read 0~524288] 3.cc04ae7d) v4 currently no flag points reached
2013-05-16 18:21:13.491472 osd.0 10.64.0.173:6822/19826 497 : [WRN] 136 slow requests, 6 included below; oldest blocked for > 192.815838 secs
2013-05-16 18:21:13.491482 osd.0 10.64.0.173:6822/19826 498 : [WRN] slow request 120.791341 seconds old, received at 2013-05-16 18:19:12.700017: osd_op(client.27297.0:50132459 21406.4154_4f1ecacb3f7ab356a91b8e8b42311d7e.err [getxattrs,stat,read 0~524288] 3.df40aae1) v4 currently no flag points reached
2013-05-16 18:21:13.491489 osd.0 10.64.0.173:6822/19826 499 : [WRN] slow request 120.652759 seconds old, received at 2013-05-16 18:19:12.838599: osd_op(client.27297.0:50132463 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:13.491495 osd.0 10.64.0.173:6822/19826 500 : [WRN] slow request 120.433432 seconds old, received at 2013-05-16 18:19:13.057926: osd_op(client.25068.0:11837432 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:13.491501 osd.0 10.64.0.173:6822/19826 501 : [WRN] slow request 60.820013 seconds old, received at 2013-05-16 18:20:12.671345: osd_op(client.27495.0:5299348 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:13.491511 osd.0 10.64.0.173:6822/19826 502 : [WRN] slow request 60.788983 seconds old, received at 2013-05-16 18:20:12.702375: osd_op(client.25068.0:11839326 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:14.574475 osd.0 10.64.0.173:6822/19826 503 : [WRN] 172 slow requests, 6 included below; oldest blocked for > 193.898808 secs
2013-05-16 18:21:14.574491 osd.0 10.64.0.173:6822/19826 504 : [WRN] slow request 120.408767 seconds old, received at 2013-05-16 18:19:14.165561: osd_op(client.27152.0:11792997 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:14.574499 osd.0 10.64.0.173:6822/19826 505 : [WRN] slow request 61.321897 seconds old, received at 2013-05-16 18:20:13.252431: osd_op(client.27152.0:11795042 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:14.574505 osd.0 10.64.0.173:6822/19826 506 : [WRN] slow request 60.831760 seconds old, received at 2013-05-16 18:20:13.742568: osd_op(client.27495.0:5299368 21406.4154_13741766be7abb8a96999f31219228db.map [getxattrs,stat,read 0~524288] 3.a8b03c1a) v4 currently no flag points reached
2013-05-16 18:21:14.574510 osd.0 10.64.0.173:6822/19826 507 : [WRN] slow request 60.520379 seconds old, received at 2013-05-16 18:20:14.053949: osd_op(client.25068.0:11839469 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:14.574516 osd.0 10.64.0.173:6822/19826 508 : [WRN] slow request 34.028536 seconds old, received at 2013-05-16 18:20:40.545792: osd_op(client.27297.0:50135519 10267.429_5/50/Morshyn_herb2.jpg/110px-Morshyn_herb2.jpg [getxattrs,stat,read 0~524288] 3.9d81346b) v4 currently no flag points reached
2013-05-16 18:21:14.793987 osd.65 10.64.32.11:6838/9550 1846 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.691691 secs
2013-05-16 18:21:14.794005 osd.65 10.64.32.11:6838/9550 1847 : [WRN] slow request 30.691691 seconds old, received at 2013-05-16 18:20:44.102237: osd_op(client.25068.0:11840558 10267.561_e/e9/Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/180px-Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg [setxattr user.rgw.idtag (33),setxattr user.rgw.shadow_name (209),call refcount.put] 3.9132b23a) v4 currently waiting for subops from [98,0]
2013-05-16 18:21:15.574782 osd.0 10.64.0.173:6822/19826 509 : [WRN] 177 slow requests, 6 included below; oldest blocked for > 194.899138 secs
2013-05-16 18:21:15.574793 osd.0 10.64.0.173:6822/19826 510 : [WRN] slow request 34.520117 seconds old, received at 2013-05-16 18:20:41.054541: osd_op(client.25068.0:11840480 21406.4154_970c97357c6f7d1479ecc2c1ae524eab.err [getxattrs,stat,read 0~524288] 3.3e864a98) v4 currently no flag points reached
2013-05-16 18:21:15.574801 osd.0 10.64.0.173:6822/19826 511 : [WRN] slow request 33.241769 seconds old, received at 2013-05-16 18:20:42.332889: osd_op(client.25068.0:11840513 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:15.574807 osd.0 10.64.0.173:6822/19826 512 : [WRN] slow request 33.034671 seconds old, received at 2013-05-16 18:20:42.539987: osd_op(client.27495.0:5300299 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:15.574813 osd.0 10.64.0.173:6822/19826 513 : [WRN] slow request 31.544846 seconds old, received at 2013-05-16 18:20:44.029812: osd_op(client.27297.0:50135674 4143.170_3/3c/Dictionary_of_National_Biography_volume_09.djvu [getxattrs,stat,read 0~524288] 3.f5bd829f) v4 currently no flag points reached
2013-05-16 18:21:15.574819 osd.0 10.64.0.173:6822/19826 514 : [WRN] slow request 31.489282 seconds old, received at 2013-05-16 18:20:44.085376: osd_op(client.25068.0:11840557 21406.10171_7a2ed2ba34ec0f212059f271719ae721.png [getxattrs,stat,read 0~524288] 3.c98c8f5a) v4 currently no flag points reached
2013-05-16 18:21:16.883864 osd.0 10.64.0.173:6822/19826 515 : [WRN] 178 slow requests, 6 included below; oldest blocked for > 196.208197 secs
2013-05-16 18:21:16.883875 osd.0 10.64.0.173:6822/19826 516 : [WRN] slow request 60.801956 seconds old, received at 2013-05-16 18:20:16.081761: osd_op(client.27297.0:50134562 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:16.883881 osd.0 10.64.0.173:6822/19826 517 : [WRN] slow request 60.388842 seconds old, received at 2013-05-16 18:20:16.494875: osd_op(client.25068.0:11839712 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:16.883893 osd.0 10.64.0.173:6822/19826 518 : [WRN] slow request 60.341055 seconds old, received at 2013-05-16 18:20:16.542662: osd_op(client.27495.0:5299471 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:16.883908 osd.0 10.64.0.173:6822/19826 519 : [WRN] slow request 60.328487 seconds old, received at 2013-05-16 18:20:16.555230: osd_op(client.27152.0:11795158 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:16.883923 osd.0 10.64.0.173:6822/19826 520 : [WRN] slow request 32.779848 seconds old, received at 2013-05-16 18:20:44.103869: osd_sub_op(client.25068.0:11840558 3.323a 9132b23a/10267.561_e/e9/Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/180px-Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/head//3 [] v 180331'20978 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:17.884179 osd.0 10.64.0.173:6822/19826 521 : [WRN] 182 slow requests, 6 included below; oldest blocked for > 197.208515 secs
2013-05-16 18:21:17.884191 osd.0 10.64.0.173:6822/19826 522 : [WRN] slow request 120.712177 seconds old, received at 2013-05-16 18:19:17.171858: osd_op(client.27152.0:11793045 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:21:17.884197 osd.0 10.64.0.173:6822/19826 523 : [WRN] slow request 31.767739 seconds old, received at 2013-05-16 18:20:46.116296: osd_op(client.27297.0:50135714 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:17.884206 osd.0 10.64.0.173:6822/19826 524 : [WRN] slow request 31.479851 seconds old, received at 2013-05-16 18:20:46.404184: osd_op(client.25068.0:11840630 .dir.10267.376 [call rgw.bucket_prepare_op] 3.9a9b0184) v4 currently no flag points reached
2013-05-16 18:21:17.884213 osd.0 10.64.0.173:6822/19826 525 : [WRN] slow request 30.782081 seconds old, received at 2013-05-16 18:20:47.101954: osd_sub_op(client.27152.0:11796137 3.3e5f 6a3c3e5f/.dir.4465.36/head//3 [] v 180331'186399 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:17.884219 osd.0 10.64.0.173:6822/19826 526 : [WRN] slow request 30.232576 seconds old, received at 2013-05-16 18:20:47.651459: osd_op(client.27495.0:5300436 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:19.220167 osd.0 10.64.0.173:6822/19826 527 : [WRN] 183 slow requests, 4 included below; oldest blocked for > 198.544520 secs
2013-05-16 18:21:19.220187 osd.0 10.64.0.173:6822/19826 528 : [WRN] slow request 120.472207 seconds old, received at 2013-05-16 18:19:18.747833: osd_op(client.27495.0:5296990 6900.31_7/79/Birch_House_Farm_-_geograph.org.uk_-_1204925.jpg [getxattrs,stat,read 0~524288] 3.efdae9e0) v4 currently no flag points reached
2013-05-16 18:21:19.220194 osd.0 10.64.0.173:6822/19826 529 : [WRN] slow request 61.131476 seconds old, received at 2013-05-16 18:20:18.088564: osd_op(client.27297.0:50134619 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:19.220215 osd.0 10.64.0.173:6822/19826 530 : [WRN] slow request 31.544055 seconds old, received at 2013-05-16 18:20:47.675985: osd_op(client.27297.0:50135743 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:19.220222 osd.0 10.64.0.173:6822/19826 531 : [WRN] slow request 30.048156 seconds old, received at 2013-05-16 18:20:49.171884: osd_op(client.27297.0:50135819 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat,read 0~524288] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:21:19.828596 mon.0 10.64.0.167:6789/0 456721 : [INF] pgmap v7225087: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3327KB/s wr, 38op/s; 1/806457473 degraded (0.000%)
2013-05-16 18:21:19.863975 osd.67 10.64.32.11:6846/11672 1664 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.080097 secs
2013-05-16 18:21:19.864017 osd.67 10.64.32.11:6846/11672 1665 : [WRN] slow request 30.080097 seconds old, received at 2013-05-16 18:20:49.783792: osd_op(client.25068.0:11840770 18499.167_d/de/George_Sandys,_1615.png/774px-George_Sandys,_1615.png [create 0~0,delete,setxattr user.rgw.idtag (33),setxattr user.rgw.shadow_name (91),writefull 0~524288,setxattr user.rgw.manifest (498),setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (50),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.29de9c4d) v4 currently waiting for subops from [38,0]
2013-05-16 18:21:21.010691 mon.0 10.64.0.167:6789/0 456722 : [INF] pgmap v7225088: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 5458KB/s wr, 48op/s; 1/806457497 degraded (0.000%)
2013-05-16 18:21:17.720559 osd.78 10.64.32.12:6849/4095 2408 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.648930 secs
2013-05-16 18:21:17.720569 osd.78 10.64.32.12:6849/4095 2409 : [WRN] slow request 30.648930 seconds old, received at 2013-05-16 18:20:47.071550: osd_op(client.27152.0:11796137 .dir.4465.36 [call rgw.bucket_prepare_op] 3.6a3c3e5f) v4 currently waiting for subops from [0,142]
2013-05-16 18:21:22.517002 mon.0 10.64.0.167:6789/0 456723 : [INF] pgmap v7225089: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 53558B/s wr, 16op/s; 1/806457506 degraded (0.000%)
2013-05-16 18:21:23.998587 mon.0 10.64.0.167:6789/0 456724 : [INF] pgmap v7225090: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 8502B/s wr, 18op/s; 1/806457518 degraded (0.000%)
2013-05-16 18:21:20.482528 osd.38 10.64.0.176:6831/22256 1616 : [WRN] 10 slow requests, 2 included below; oldest blocked for > 228.845013 secs
2013-05-16 18:21:20.482536 osd.38 10.64.0.176:6831/22256 1617 : [WRN] slow request 60.664832 seconds old, received at 2013-05-16 18:20:19.817631: osd_op(client.27495.0:5299567 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [create 0~0,delete,writefull 0~1632,setxattr user.rgw.acl (123),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.59e4fd08) v4 currently waiting for subops from [56,0]
2013-05-16 18:21:20.482542 osd.38 10.64.0.176:6831/22256 1618 : [WRN] slow request 60.370701 seconds old, received at 2013-05-16 18:20:20.111762: osd_op(client.25068.0:11839883 6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png [create 0~0,delete,writefull 0~1632,setxattr user.rgw.acl (123),setxattr user.rgw.content_type (10),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.59e4fd08) v4 currently waiting for subops from [56,0]
2013-05-16 18:21:21.482818 osd.38 10.64.0.176:6831/22256 1619 : [WRN] 10 slow requests, 4 included below; oldest blocked for > 229.845272 secs
2013-05-16 18:21:21.482827 osd.38 10.64.0.176:6831/22256 1620 : [WRN] slow request 60.843659 seconds old, received at 2013-05-16 18:20:20.639063: osd_op(client.27297.0:50134711 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:21.482847 osd.38 10.64.0.176:6831/22256 1621 : [WRN] slow request 60.834902 seconds old, received at 2013-05-16 18:20:20.647820: osd_op(client.25068.0:11839898 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:21.482859 osd.38 10.64.0.176:6831/22256 1622 : [WRN] slow request 60.811281 seconds old, received at 2013-05-16 18:20:20.671441: osd_op(client.27297.0:50134713 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:21.482872 osd.38 10.64.0.176:6831/22256 1623 : [WRN] slow request 60.711364 seconds old, received at 2013-05-16 18:20:20.771358: osd_op(client.25068.0:11839902 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.483745 osd.38 10.64.0.176:6831/22256 1624 : [WRN] 16 slow requests, 6 included below; oldest blocked for > 233.846205 secs
2013-05-16 18:21:25.483751 osd.38 10.64.0.176:6831/22256 1625 : [WRN] slow request 30.993001 seconds old, received at 2013-05-16 18:20:54.490654: osd_op(client.27152.0:11796340 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.483755 osd.38 10.64.0.176:6831/22256 1626 : [WRN] slow request 30.981599 seconds old, received at 2013-05-16 18:20:54.502056: osd_op(client.27152.0:11796341 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.483760 osd.38 10.64.0.176:6831/22256 1627 : [WRN] slow request 30.970989 seconds old, received at 2013-05-16 18:20:54.512666: osd_op(client.27495.0:5300712 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.483764 osd.38 10.64.0.176:6831/22256 1628 : [WRN] slow request 30.968045 seconds old, received at 2013-05-16 18:20:54.515610: osd_op(client.27297.0:50136036 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.483767 osd.38 10.64.0.176:6831/22256 1629 : [WRN] slow request 30.941820 seconds old, received at 2013-05-16 18:20:54.541835: osd_op(client.25068.0:11840907 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:25.512580 mon.0 10.64.0.167:6789/0 456725 : [INF] pgmap v7225091: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 252KB/s wr, 32op/s; 1/806457539 degraded (0.000%)
2013-05-16 18:21:26.483994 osd.38 10.64.0.176:6831/22256 1630 : [WRN] 18 slow requests, 5 included below; oldest blocked for > 234.846461 secs
2013-05-16 18:21:26.484010 osd.38 10.64.0.176:6831/22256 1631 : [WRN] slow request 60.230168 seconds old, received at 2013-05-16 18:20:26.253743: osd_op(client.27495.0:5299738 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:26.484015 osd.38 10.64.0.176:6831/22256 1632 : [WRN] slow request 60.216814 seconds old, received at 2013-05-16 18:20:26.267097: osd_op(client.25068.0:11840100 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:26.484019 osd.38 10.64.0.176:6831/22256 1633 : [WRN] slow request 31.915840 seconds old, received at 2013-05-16 18:20:54.568071: osd_op(client.25068.0:11840908 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:26.484023 osd.38 10.64.0.176:6831/22256 1634 : [WRN] slow request 31.891992 seconds old, received at 2013-05-16 18:20:54.591919: osd_op(client.27495.0:5300717 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:26.484027 osd.38 10.64.0.176:6831/22256 1635 : [WRN] slow request 31.772406 seconds old, received at 2013-05-16 18:20:54.711505: osd_op(client.27297.0:50136041 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:26.895062 mon.0 10.64.0.167:6789/0 456726 : [INF] pgmap v7225092: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 202KB/s wr, 27op/s; 1/806457557 degraded (0.000%)
2013-05-16 18:21:27.112581 osd.114 10.64.32.15:6851/28569 1688 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.125800 secs
2013-05-16 18:21:27.112644 osd.114 10.64.32.15:6851/28569 1689 : [WRN] slow request 30.125800 seconds old, received at 2013-05-16 18:20:56.986700: osd_op(client.27495.0:5300769 10267.582_d/d3/Neukirchen_am_Ostrong_-_Pfarrhof.JPG/300px-Neukirchen_am_Ostrong_-_Pfarrhof.JPG [create 0~0,delete,writefull 0~22514,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (61),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.4ac84c74) v4 currently waiting for subops from [38,0]
2013-05-16 18:21:20.220479 osd.0 10.64.0.173:6822/19826 532 : [WRN] 186 slow requests, 4 included below; oldest blocked for > 199.544839 secs
2013-05-16 18:21:20.220489 osd.0 10.64.0.173:6822/19826 533 : [WRN] slow request 60.120243 seconds old, received at 2013-05-16 18:20:20.100116: osd_sub_op(osd.38.0:9042983 3.3d08 a2457d08/.dir.10267.498/head//3 [push] v 180321'1594145 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:20.220497 osd.0 10.64.0.173:6822/19826 534 : [WRN] slow request 30.798445 seconds old, received at 2013-05-16 18:20:49.421914: osd_op(client.27297.0:50135833 21406.1730_925418464e919cae531010594fe36100.map [getxattrs,stat,read 0~524288] 3.a56bb46b) v4 currently no flag points reached
2013-05-16 18:21:20.220503 osd.0 10.64.0.173:6822/19826 535 : [WRN] slow request 30.475983 seconds old, received at 2013-05-16 18:20:49.744376: osd_op(client.27495.0:5300523 6900.2080_5/e/e/5ee89e20c6045f8563cef772bf1130fe.png [getxattrs,stat,read 0~524288] 3.84c68730) v4 currently no flag points reached
2013-05-16 18:21:20.220509 osd.0 10.64.0.173:6822/19826 536 : [WRN] slow request 30.427580 seconds old, received at 2013-05-16 18:20:49.792779: osd_sub_op(client.25068.0:11840770 3.1c4d 29de9c4d/18499.167_d/de/George_Sandys,_1615.png/774px-George_Sandys,_1615.png/head//3 [] v 180331'20283 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:21.220773 osd.0 10.64.0.173:6822/19826 537 : [WRN] 147 slow requests, 6 included below; oldest blocked for > 200.545128 secs
2013-05-16 18:21:21.220784 osd.0 10.64.0.173:6822/19826 538 : [WRN] slow request 120.774853 seconds old, received at 2013-05-16 18:19:20.445795: osd_sub_op(client.25068.0:11837568 3.201a e5cd201a/10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/head//3 [] v 180331'21662 snapset=0=[]:[] snapc=0=[]) v7 currently waiting for active
2013-05-16 18:21:21.220791 osd.0 10.64.0.173:6822/19826 539 : [WRN] slow request 120.581247 seconds old, received at 2013-05-16 18:19:20.639401: osd_op(client.27297.0:50132610 21406.4154_02fdb62c32d2088ab3ed9875c3b5b2aa.err [getxattrs,stat,read 0~524288] 3.8a231db4) v4 currently no flag points reached
2013-05-16 18:21:21.220798 osd.0 10.64.0.173:6822/19826 540 : [WRN] slow request 60.991683 seconds old, received at 2013-05-16 18:20:20.228965: osd_sub_op(client.27495.0:5299567 3.3d08 59e4fd08/6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png/head//3 [] v 180331'1594146 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:21.220804 osd.0 10.64.0.173:6822/19826 541 : [WRN] slow request 60.991586 seconds old, received at 2013-05-16 18:20:20.229062: osd_sub_op(client.25068.0:11839883 3.3d08 59e4fd08/6900.2063_a/7/7/a77b196740cb1d9d6bbf04a433528d4c.png/head//3 [] v 180331'1594147 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:21.220810 osd.0 10.64.0.173:6822/19826 542 : [WRN] slow request 60.968046 seconds old, received at 2013-05-16 18:20:20.252602: osd_op(client.27152.0:11795306 6900.1954_3/f/f/3ff46dd4c09b416a7d57c7457c0a2cb9.png [getxattrs,stat] 3.e9a57159) v4 currently no flag points reached
2013-05-16 18:21:21.262288 osd.101 10.64.32.14:6818/2810 1856 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 120.817514 secs
2013-05-16 18:21:21.262306 osd.101 10.64.32.14:6818/2810 1857 : [WRN] slow request 120.817514 seconds old, received at 2013-05-16 18:19:20.444711: osd_op(client.25068.0:11837568 10267.557_1/1a/Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg/75px-Birch_House,_Oughtibridge_-_3_-_geograph.org.uk_-_991087.jpg [create 0~0,delete,writefull 0~2097,setxattr user.rgw.acl (123),setxattr user.rgw.content_disposition (87),setxattr user.rgw.content_type (11),setxattr user.rgw.etag (33),setxattr user.rgw.x-amz-meta-sha1base36 (32)] 3.e5cd201a) v4 currently waiting for subops from [0,25]
2013-05-16 18:21:22.221059 osd.0 10.64.0.173:6822/19826 543 : [WRN] 188 slow requests, 6 included below; oldest blocked for > 201.545432 secs
2013-05-16 18:21:22.221067 osd.0 10.64.0.173:6822/19826 544 : [WRN] slow request 120.759285 seconds old, received at 2013-05-16 18:19:21.461667: osd_op(client.27152.0:11793151 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:21:22.221071 osd.0 10.64.0.173:6822/19826 545 : [WRN] slow request 61.780188 seconds old, received at 2013-05-16 18:20:20.440764: osd_op(client.25068.0:11839892 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:21:22.221076 osd.0 10.64.0.173:6822/19826 546 : [WRN] slow request 61.554442 seconds old, received at 2013-05-16 18:20:20.666510: osd_op(client.27495.0:5299594 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:21:22.221080 osd.0 10.64.0.173:6822/19826 547 : [WRN] slow request 60.706724 seconds old, received at 2013-05-16 18:20:21.514228: osd_op(client.27297.0:50134778 18499.51_6/63/INTI_International_University_Logo.png/545px-INTI_International_University_Logo.png [getxattrs,stat,read 0~524288] 3.cc04ae7d) v4 currently no flag points reached
2013-05-16 18:21:22.221084 osd.0 10.64.0.173:6822/19826 548 : [WRN] slow request 31.270139 seconds old, received at 2013-05-16 18:20:50.950813: osd_op(client.25068.0:11840804 21406.4154_45fb871beb4ddddf0ff5da5816e37825.map [getxattrs,stat,read 0~524288] 3.6d152929) v4 currently no flag points reached
2013-05-16 18:21:23.221347 osd.0 10.64.0.173:6822/19826 549 : [WRN] 191 slow requests, 6 included below; oldest blocked for > 202.545702 secs
2013-05-16 18:21:23.221357 osd.0 10.64.0.173:6822/19826 550 : [WRN] slow request 120.367530 seconds old, received at 2013-05-16 18:19:22.853692: osd_op(client.27495.0:5297108 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:23.221364 osd.0 10.64.0.173:6822/19826 551 : [WRN] slow request 60.126250 seconds old, received at 2013-05-16 18:20:23.094972: osd_op(client.25068.0:11840011 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:23.221370 osd.0 10.64.0.173:6822/19826 552 : [WRN] slow request 32.129846 seconds old, received at 2013-05-16 18:20:51.091376: osd_op(client.27297.0:50135925 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:23.221388 osd.0 10.64.0.173:6822/19826 553 : [WRN] slow request 32.122043 seconds old, received at 2013-05-16 18:20:51.099179: osd_op(client.27152.0:11796262 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:23.221392 osd.0 10.64.0.173:6822/19826 554 : [WRN] slow request 31.368769 seconds old, received at 2013-05-16 18:20:51.852453: osd_op(client.27297.0:50135970 21406.1730_9c2cb6a818b135e68bfc75a383310d82.map [getxattrs,stat,read 0~524288] 3.bdc24c6d) v4 currently no flag points reached
2013-05-16 18:21:24.221696 osd.0 10.64.0.173:6822/19826 555 : [WRN] 194 slow requests, 6 included below; oldest blocked for > 203.546078 secs
2013-05-16 18:21:24.221703 osd.0 10.64.0.173:6822/19826 556 : [WRN] slow request 60.954222 seconds old, received at 2013-05-16 18:20:23.267376: osd_op(client.27495.0:5299649 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat,read 0~524288] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:21:24.221708 osd.0 10.64.0.173:6822/19826 557 : [WRN] slow request 60.934791 seconds old, received at 2013-05-16 18:20:23.286807: osd_op(client.27297.0:50134961 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:24.221716 osd.0 10.64.0.173:6822/19826 558 : [WRN] slow request 31.452504 seconds old, received at 2013-05-16 18:20:52.769094: osd_op(client.27495.0:5300612 21406.4154_ce15dad2e8ff52336d0cb98162a8d080.err [getxattrs,stat,read 0~524288] 3.b8b7d65) v4 currently no flag points reached
2013-05-16 18:21:24.221721 osd.0 10.64.0.173:6822/19826 559 : [WRN] slow request 31.093320 seconds old, received at 2013-05-16 18:20:53.128278: osd_op(client.27297.0:50136004 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:24.221732 osd.0 10.64.0.173:6822/19826 560 : [WRN] slow request 30.470487 seconds old, received at 2013-05-16 18:20:53.751111: osd_op(client.27297.0:50136019 6900.1991_b/8/3/b8375e70b66148be8f6c766462fd4b37.png [getxattrs,stat,read 0~524288] 3.a1768795) v4 currently no flag points reached
2013-05-16 18:21:25.222013 osd.0 10.64.0.173:6822/19826 561 : [WRN] 195 slow requests, 6 included below; oldest blocked for > 204.546361 secs
2013-05-16 18:21:25.222023 osd.0 10.64.0.173:6822/19826 562 : [WRN] slow request 120.316341 seconds old, received at 2013-05-16 18:19:24.905540: osd_op(client.25068.0:11837653 6900.1942_3/9/3/image_3336f9d6_3938ed8917ea182e.png [getxattrs,stat,read 0~524288] 3.1507b6c9) v4 currently no flag points reached
2013-05-16 18:21:25.222030 osd.0 10.64.0.173:6822/19826 563 : [WRN] slow request 120.146170 seconds old, received at 2013-05-16 18:19:25.075711: osd_op(client.27152.0:11793231 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat,read 0~524288] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:21:25.222036 osd.0 10.64.0.173:6822/19826 564 : [WRN] slow request 60.856355 seconds old, received at 2013-05-16 18:20:24.365526: osd_op(client.27495.0:5299670 4320.11_c/ce/Rimbaud_in_Museum.JPG [getxattrs,stat,read 0~524288] 3.597aa411) v4 currently no flag points reached
2013-05-16 18:21:25.222043 osd.0 10.64.0.173:6822/19826 565 : [WRN] slow request 60.030386 seconds old, received at 2013-05-16 18:20:25.191495: osd_op(client.27152.0:11795436 21406.14196_0/01/Ctchannel3.png/720px-Ctchannel3.png [getxattrs,stat,read 0~524288] 3.c21c6411) v4 currently no flag points reached
2013-05-16 18:21:25.222053 osd.0 10.64.0.173:6822/19826 566 : [WRN] slow request 31.406526 seconds old, received at 2013-05-16 18:20:53.815355: osd_op(client.27495.0:5300688 6900.1863_0/6/f/06f95dbbaffb2f4c84c32a8d86b6f550.png [getxattrs,stat] 3.e8f7f5d7) v4 currently no flag points reached
2013-05-16 18:21:26.306000 osd.0 10.64.0.173:6822/19826 567 : [WRN] 196 slow requests, 2 included below; oldest blocked for > 205.630377 secs
2013-05-16 18:21:26.306012 osd.0 10.64.0.173:6822/19826 568 : [WRN] slow request 32.419595 seconds old, received at 2013-05-16 18:20:53.886302: osd_op(client.27297.0:50136025 21406.4154_d1f05780edad84c5b99072e0794dbdf4.map [getxattrs,stat,read 0~524288] 3.4758b191) v4 currently no flag points reached
2013-05-16 18:21:26.306018 osd.0 10.64.0.173:6822/19826 569 : [WRN] slow request 31.345075 seconds old, received at 2013-05-16 18:20:54.960822: osd_op(client.25068.0:11840916 4320.11_c/ce/Rimbaud_in_Museum.JPG [getxattrs,stat,read 0~524288] 3.597aa411) v4 currently no flag points reached
2013-05-16 18:21:27.306210 osd.0 10.64.0.173:6822/19826 570 : [WRN] 197 slow requests, 5 included below; oldest blocked for > 206.630601 secs
2013-05-16 18:21:27.306217 osd.0 10.64.0.173:6822/19826 571 : [WRN] slow request 120.883944 seconds old, received at 2013-05-16 18:19:26.422177: osd_op(client.27297.0:50132717 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat,read 0~524288] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:21:27.306222 osd.0 10.64.0.173:6822/19826 572 : [WRN] slow request 60.683710 seconds old, received at 2013-05-16 18:20:26.622411: osd_op(client.25068.0:11840106 4320.16_7/7d/Ioannina_orismos_1430.jpg [getxattrs,stat,read 0~524288] 3.371d374b) v4 currently no flag points reached
2013-05-16 18:21:27.306229 osd.0 10.64.0.173:6822/19826 573 : [WRN] slow request 60.488689 seconds old, received at 2013-05-16 18:20:26.817432: osd_op(client.27297.0:50135129 10267.579_7/77/Valleberga_triptych_altar.jpg/75px-Valleberga_triptych_altar.jpg [getxattrs,stat] 3.5580e157) v4 currently no flag points reached
2013-05-16 18:21:27.306235 osd.0 10.64.0.173:6822/19826 574 : [WRN] slow request 60.056766 seconds old, received at 2013-05-16 18:20:27.249355: osd_op(client.25068.0:11840120 21406.1730_e8c95a401e29cd7e9c06e276e65a45cb.err [getxattrs,stat,read 0~524288] 3.bc51352f) v4 currently no flag points reached
2013-05-16 18:21:27.306242 osd.0 10.64.0.173:6822/19826 575 : [WRN] slow request 30.317850 seconds old, received at 2013-05-16 18:20:56.988271: osd_sub_op(client.27495.0:5300769 3.c74 4ac84c74/10267.582_d/d3/Neukirchen_am_Ostrong_-_Pfarrhof.JPG/300px-Neukirchen_am_Ostrong_-_Pfarrhof.JPG/head//3 [] v 180331'20403 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:28.250948 mon.0 10.64.0.167:6789/0 456727 : [INF] pgmap v7225093: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 74068B/s wr, 30op/s; 1/806457575 degraded (0.000%)
2013-05-16 18:21:29.306689 osd.0 10.64.0.173:6822/19826 576 : [WRN] 199 slow requests, 4 included below; oldest blocked for > 208.631082 secs
2013-05-16 18:21:29.306694 osd.0 10.64.0.173:6822/19826 577 : [WRN] slow request 60.513648 seconds old, received at 2013-05-16 18:20:28.792954: osd_op(client.27152.0:11795607 6900.1991_b/8/3/b8375e70b66148be8f6c766462fd4b37.png [getxattrs,stat] 3.a1768795) v4 currently no flag points reached
2013-05-16 18:21:29.306705 osd.0 10.64.0.173:6822/19826 578 : [WRN] slow request 60.280538 seconds old, received at 2013-05-16 18:20:29.026064: osd_op(client.25068.0:11840169 10267.546_e/e4/Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu/page1-375px-Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu.jpg [getxattrs,stat,read 0~524288] 3.556b58de) v4 currently no flag points reached
2013-05-16 18:21:29.306710 osd.0 10.64.0.173:6822/19826 579 : [WRN] slow request 30.181669 seconds old, received at 2013-05-16 18:20:59.124933: osd_op(client.27495.0:5300820 10267.546_e/e4/Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu/page1-375px-Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu.jpg [getxattrs,stat,read 0~524288] 3.556b58de) v4 currently no flag points reached
2013-05-16 18:21:29.306722 osd.0 10.64.0.173:6822/19826 580 : [WRN] slow request 30.096547 seconds old, received at 2013-05-16 18:20:59.210055: osd_op(client.27152.0:11796500 21406.4154_d0b3ad5fdb77f600760b613aecdd4e6e.err [getxattrs,stat,read 0~524288] 3.49387b02) v4 currently no flag points reached
2013-05-16 18:21:29.680052 mon.0 10.64.0.167:6789/0 456728 : [INF] pgmap v7225094: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 40772B/s wr, 15op/s; 1/806457584 degraded (0.000%)
2013-05-16 18:21:31.107393 mon.0 10.64.0.167:6789/0 456729 : [INF] pgmap v7225095: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 205KB/s wr, 39op/s; 1/806457608 degraded (0.000%)
2013-05-16 18:21:32.580087 mon.0 10.64.0.167:6789/0 456730 : [INF] pgmap v7225096: 16760 pgs: 118 active, 16625 active+clean, 1 active+recovery_wait, 16 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 98955B/s wr, 14op/s; 1/806457617 degraded (0.000%)
2013-05-16 18:21:34.094574 mon.0 10.64.0.167:6789/0 456731 : [INF] pgmap v7225097: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 85563B/s wr, 19op/s; 1/806457629 degraded (0.000%)
2013-05-16 18:21:32.485209 osd.38 10.64.0.176:6831/22256 1636 : [WRN] 18 slow requests, 1 included below; oldest blocked for > 240.847698 secs
2013-05-16 18:21:32.485219 osd.38 10.64.0.176:6831/22256 1637 : [WRN] slow request 240.847698 seconds old, received at 2013-05-16 18:17:31.637450: osd_op(client.27297.0:50128581 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:35.464318 mon.0 10.64.0.167:6789/0 456732 : [INF] pgmap v7225098: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 73652B/s wr, 16op/s; 1/806457638 degraded (0.000%)
2013-05-16 18:21:35.485766 osd.38 10.64.0.176:6831/22256 1638 : [WRN] 18 slow requests, 1 included below; oldest blocked for > 243.848242 secs
2013-05-16 18:21:35.485780 osd.38 10.64.0.176:6831/22256 1639 : [WRN] slow request 60.858526 seconds old, received at 2013-05-16 18:20:34.627166: osd_op(client.25068.0:11840339 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:37.045281 mon.0 10.64.0.167:6789/0 456733 : [INF] pgmap v7225099: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 56099B/s wr, 13op/s; 1/806457644 degraded (0.000%)
2013-05-16 18:21:30.505057 osd.0 10.64.0.173:6822/19826 581 : [WRN] 199 slow requests, 2 included below; oldest blocked for > 209.829424 secs
2013-05-16 18:21:30.505066 osd.0 10.64.0.173:6822/19826 582 : [WRN] slow request 60.963736 seconds old, received at 2013-05-16 18:20:29.541208: osd_op(client.25068.0:11840187 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:30.505075 osd.0 10.64.0.173:6822/19826 583 : [WRN] slow request 60.898592 seconds old, received at 2013-05-16 18:20:29.606352: osd_op(client.27297.0:50135253 21406.4154_a8f01675114bec2866ba345f6bc2ec58.err [getxattrs,stat,read 0~524288] 3.3eddfb97) v4 currently no flag points reached
2013-05-16 18:21:31.505325 osd.0 10.64.0.173:6822/19826 584 : [WRN] 199 slow requests, 3 included below; oldest blocked for > 210.829694 secs
2013-05-16 18:21:31.505335 osd.0 10.64.0.173:6822/19826 585 : [WRN] slow request 120.408380 seconds old, received at 2013-05-16 18:19:31.096834: osd_op(client.27152.0:11793342 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:31.505342 osd.0 10.64.0.173:6822/19826 586 : [WRN] slow request 60.134833 seconds old, received at 2013-05-16 18:20:31.370381: osd_op(client.25068.0:11840220 6900.1942_7/f/c/image_abf01d00_7fcc279d4d760833.png [getxattrs,stat,read 0~524288] 3.c1dccee7) v4 currently no flag points reached
2013-05-16 18:21:31.505349 osd.0 10.64.0.173:6822/19826 587 : [WRN] slow request 30.561267 seconds old, received at 2013-05-16 18:21:00.943947: osd_op(client.27152.0:11796624 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:32.505604 osd.0 10.64.0.173:6822/19826 588 : [WRN] 199 slow requests, 3 included below; oldest blocked for > 211.829975 secs
2013-05-16 18:21:32.505613 osd.0 10.64.0.173:6822/19826 589 : [WRN] slow request 120.607401 seconds old, received at 2013-05-16 18:19:31.898094: osd_op(client.27495.0:5297521 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:32.505619 osd.0 10.64.0.173:6822/19826 590 : [WRN] slow request 120.602170 seconds old, received at 2013-05-16 18:19:31.903325: osd_op(client.27297.0:50133033 6900.22__shadow__NxOUgLw-O1fOlZQ3K8l_-S6qCUqGlfz_28 [read 0~524288] 3.b7a0f159) v4 currently no flag points reached
2013-05-16 18:21:32.505626 osd.0 10.64.0.173:6822/19826 591 : [WRN] slow request 120.388814 seconds old, received at 2013-05-16 18:19:32.116681: osd_op(client.27297.0:50133037 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:21:33.505937 osd.0 10.64.0.173:6822/19826 592 : [WRN] 201 slow requests, 4 included below; oldest blocked for > 212.830248 secs
2013-05-16 18:21:33.505948 osd.0 10.64.0.173:6822/19826 593 : [WRN] slow request 120.877992 seconds old, received at 2013-05-16 18:19:32.627776: osd_op(client.27495.0:5297532 21409.4219_06b1df1f6a961a97b875dc841cc27fbe.err [getxattrs,stat,read 0~524288] 3.6f079cfb) v4 currently no flag points reached
2013-05-16 18:21:33.505955 osd.0 10.64.0.173:6822/19826 594 : [WRN] slow request 60.048896 seconds old, received at 2013-05-16 18:20:33.456872: osd_op(client.27297.0:50135351 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:21:33.505962 osd.0 10.64.0.173:6822/19826 595 : [WRN] slow request 30.439533 seconds old, received at 2013-05-16 18:21:03.066235: osd_op(client.27152.0:11796673 21406.1730_27b4ccfe59186c77e71f2a152da12675.err [getxattrs,stat,read 0~524288] 3.98f72a3) v4 currently no flag points reached
2013-05-16 18:21:33.505969 osd.0 10.64.0.173:6822/19826 596 : [WRN] slow request 30.056306 seconds old, received at 2013-05-16 18:21:03.449462: osd_op(client.25068.0:11841224 6900.2103_9/f/9/9f96d268f1bdd35241f9a205b3c129fd.png [getxattrs,stat,read 0~524288] 3.7a10b191) v4 currently no flag points reached
2013-05-16 18:21:34.602135 osd.0 10.64.0.173:6822/19826 597 : [WRN] 203 slow requests, 2 included below; oldest blocked for > 213.926498 secs
2013-05-16 18:21:34.602147 osd.0 10.64.0.173:6822/19826 598 : [WRN] slow request 30.635245 seconds old, received at 2013-05-16 18:21:03.966773: osd_op(client.25068.0:11841231 10267.546_e/e4/Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu/page1-375px-Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu.jpg [getxattrs,stat,read 0~524288] 3.556b58de) v4 currently no flag points reached
2013-05-16 18:21:34.602155 osd.0 10.64.0.173:6822/19826 599 : [WRN] slow request 30.628192 seconds old, received at 2013-05-16 18:21:03.973826: osd_op(client.27152.0:11796687 10267.546_e/e4/Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu/page1-375px-Il_crowdsourcing_tra_necessità_di_coordinamento_e_perdita_di_controllo.djvu.jpg [getxattrs,stat,read 0~524288] 3.556b58de) v4 currently no flag points reached
2013-05-16 18:21:35.602424 osd.0 10.64.0.173:6822/19826 600 : [WRN] 205 slow requests, 6 included below; oldest blocked for > 214.926770 secs
2013-05-16 18:21:35.602433 osd.0 10.64.0.173:6822/19826 601 : [WRN] slow request 120.617818 seconds old, received at 2013-05-16 18:19:34.984472: osd_op(client.27152.0:11793433 21406.10556_558a3253a20b22b48adfbd41958c3bd6.map [getxattrs,stat,read 0~524288] 3.c885f2e8) v4 currently no flag points reached
2013-05-16 18:21:35.602445 osd.0 10.64.0.173:6822/19826 602 : [WRN] slow request 60.940788 seconds old, received at 2013-05-16 18:20:34.661502: osd_op(client.25068.0:11840340 6900.1878_7/4/f/74f765c63701e4a309f1f6213634edc8.png [getxattrs,stat] 3.4626ce2e) v4 currently no flag points reached
2013-05-16 18:21:35.602451 osd.0 10.64.0.173:6822/19826 603 : [WRN] slow request 60.418226 seconds old, received at 2013-05-16 18:20:35.184064: osd_op(client.27152.0:11795788 21406.14196_0/01/Ctchannel3.png/720px-Ctchannel3.png [getxattrs,stat,read 0~524288] 3.c21c6411) v4 currently no flag points reached
2013-05-16 18:21:35.602457 osd.0 10.64.0.173:6822/19826 604 : [WRN] slow request 60.366878 seconds old, received at 2013-05-16 18:20:35.235412: osd_op(client.27495.0:5300055 21406.4154_d0b3ad5fdb77f600760b613aecdd4e6e.err [getxattrs,stat,read 0~524288] 3.49387b02) v4 currently no flag points reached
2013-05-16 18:21:35.602463 osd.0 10.64.0.173:6822/19826 605 : [WRN] slow request 30.644060 seconds old, received at 2013-05-16 18:21:04.958230: osd_op(client.27495.0:5300955 4320.11_c/ce/Rimbaud_in_Museum.JPG [getxattrs,stat,read 0~524288] 3.597aa411) v4 currently no flag points reached
2013-05-16 18:21:36.602738 osd.0 10.64.0.173:6822/19826 606 : [WRN] 206 slow requests, 4 included below; oldest blocked for > 215.927095 secs
2013-05-16 18:21:36.602750 osd.0 10.64.0.173:6822/19826 607 : [WRN] slow request 120.896681 seconds old, received at 2013-05-16 18:19:35.705934: osd_op(client.27152.0:11793455 21406.1730_d5d29fdc7a4f92c12cd935e043c6f46a.map [getxattrs,stat,read 0~524288] 3.c4a5e27a) v4 currently no flag points reached
2013-05-16 18:21:36.602757 osd.0 10.64.0.173:6822/19826 608 : [WRN] slow request 60.487893 seconds old, received at 2013-05-16 18:20:36.114722: osd_op(client.27297.0:50135424 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:36.602764 osd.0 10.64.0.173:6822/19826 609 : [WRN] slow request 31.334351 seconds old, received at 2013-05-16 18:21:05.268264: osd_op(client.27152.0:11796705 21406.4154_d0b3ad5fdb77f600760b613aecdd4e6e.err [getxattrs,stat,read 0~524288] 3.49387b02) v4 currently no flag points reached
2013-05-16 18:21:36.602770 osd.0 10.64.0.173:6822/19826 610 : [WRN] slow request 31.228629 seconds old, received at 2013-05-16 18:21:05.373986: osd_op(client.25068.0:11841252 21409.4219_35b1ad8fe7acce82f7ef64e06a61d921.map [getxattrs,stat,read 0~524288] 3.c335be06) v4 currently no flag points reached
2013-05-16 18:21:37.603017 osd.0 10.64.0.173:6822/19826 611 : [WRN] 206 slow requests, 1 included below; oldest blocked for > 216.927429 secs
2013-05-16 18:21:37.603026 osd.0 10.64.0.173:6822/19826 612 : [WRN] slow request 60.134488 seconds old, received at 2013-05-16 18:20:37.468461: osd_op(client.27152.0:11795843 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:38.603233 osd.0 10.64.0.173:6822/19826 613 : [WRN] 169 slow requests, 6 included below; oldest blocked for > 217.927618 secs
2013-05-16 18:21:38.603241 osd.0 10.64.0.173:6822/19826 614 : [WRN] slow request 120.937384 seconds old, received at 2013-05-16 18:19:37.665754: osd_op(client.27152.0:11793522 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:21:38.603244 osd.0 10.64.0.173:6822/19826 615 : [WRN] slow request 120.819940 seconds old, received at 2013-05-16 18:19:37.783198: osd_op(client.27495.0:5297687 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:38.603248 osd.0 10.64.0.173:6822/19826 616 : [WRN] slow request 120.396787 seconds old, received at 2013-05-16 18:19:38.206351: osd_op(client.25068.0:11837884 21406.4154_f840c83148edf2d7fdf77682eeaa33db.err [getxattrs,stat,read 0~524288] 3.19ec1263) v4 currently no flag points reached
2013-05-16 18:21:38.603251 osd.0 10.64.0.173:6822/19826 617 : [WRN] slow request 120.370698 seconds old, received at 2013-05-16 18:19:38.232440: osd_op(client.27495.0:5297690 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:38.603255 osd.0 10.64.0.173:6822/19826 618 : [WRN] slow request 120.299670 seconds old, received at 2013-05-16 18:19:38.303468: osd_op(client.27495.0:5297693 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:38.625984 mon.0 10.64.0.167:6789/0 456734 : [INF] pgmap v7225100: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 382KB/s wr, 10op/s; 1/806457650 degraded (0.000%)
2013-05-16 18:21:39.603466 osd.0 10.64.0.173:6822/19826 619 : [WRN] 208 slow requests, 3 included below; oldest blocked for > 218.927865 secs
2013-05-16 18:21:39.603474 osd.0 10.64.0.173:6822/19826 620 : [WRN] slow request 61.783558 seconds old, received at 2013-05-16 18:20:37.819827: osd_op(client.27297.0:50135464 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat,read 0~524288] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:21:39.603479 osd.0 10.64.0.173:6822/19826 621 : [WRN] slow request 31.318094 seconds old, received at 2013-05-16 18:21:08.285291: osd_op(client.27297.0:50136391 4143.177_archive/3/36/20090216231640!DominicanRepublicSubdivisions.png [getxattrs,stat,read 0~524288] 3.faae4795) v4 currently no flag points reached
2013-05-16 18:21:39.603482 osd.0 10.64.0.173:6822/19826 622 : [WRN] slow request 31.230299 seconds old, received at 2013-05-16 18:21:08.373086: osd_op(client.25068.0:11841329 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:21:39.894230 mon.0 10.64.0.167:6789/0 456735 : [INF] pgmap v7225101: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 5105KB/s wr, 29op/s; 1/806457668 degraded (0.000%)
2013-05-16 18:21:41.271612 mon.0 10.64.0.167:6789/0 456736 : [INF] pgmap v7225102: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3393KB/s wr, 48op/s; 1/806457689 degraded (0.000%)
2013-05-16 18:21:42.752584 mon.0 10.64.0.167:6789/0 456737 : [INF] pgmap v7225103: 16760 pgs: 118 active, 16624 active+clean, 1 active+recovery_wait, 17 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3465KB/s wr, 36op/s; 1/806457713 degraded (0.000%)
2013-05-16 18:21:44.125026 mon.0 10.64.0.167:6789/0 456738 : [INF] pgmap v7225104: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 703KB/s wr, 36op/s; 1/806457731 degraded (0.000%)
2013-05-16 18:21:45.514132 mon.0 10.64.0.167:6789/0 456739 : [INF] pgmap v7225105: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 82009B/s wr, 20op/s; 1/806457743 degraded (0.000%)
2013-05-16 18:21:46.767141 mon.0 10.64.0.167:6789/0 456740 : [INF] pgmap v7225106: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 87747B/s wr, 30op/s; 1/806457761 degraded (0.000%)
2013-05-16 18:21:40.694214 osd.0 10.64.0.173:6822/19826 623 : [WRN] 209 slow requests, 5 included below; oldest blocked for > 220.018536 secs
2013-05-16 18:21:40.694229 osd.0 10.64.0.173:6822/19826 624 : [WRN] slow request 120.857468 seconds old, received at 2013-05-16 18:19:39.836588: osd_op(client.27297.0:50133216 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:21:40.694235 osd.0 10.64.0.173:6822/19826 625 : [WRN] slow request 120.709256 seconds old, received at 2013-05-16 18:19:39.984800: osd_op(client.27152.0:11793566 21406.4154_5d71b735166a2a43fe3e63c76e64c20f.png [getxattrs,stat,read 0~524288] 3.9c83319e) v4 currently no flag points reached
2013-05-16 18:21:40.694241 osd.0 10.64.0.173:6822/19826 626 : [WRN] slow request 120.694774 seconds old, received at 2013-05-16 18:19:39.999282: osd_op(client.27495.0:5297745 21406.4154_f1bddcd36ff4b6dfb750562a355c60ea.map [getxattrs,stat,read 0~524288] 3.9b32827a) v4 currently no flag points reached
2013-05-16 18:21:40.694247 osd.0 10.64.0.173:6822/19826 627 : [WRN] slow request 60.148264 seconds old, received at 2013-05-16 18:20:40.545792: osd_op(client.27297.0:50135519 10267.429_5/50/Morshyn_herb2.jpg/110px-Morshyn_herb2.jpg [getxattrs,stat,read 0~524288] 3.9d81346b) v4 currently no flag points reached
2013-05-16 18:21:40.694260 osd.0 10.64.0.173:6822/19826 628 : [WRN] slow request 30.243621 seconds old, received at 2013-05-16 18:21:10.450435: osd_op(client.27152.0:11796807 10267.571_1/11/EstMafra.jpg/75px-EstMafra.jpg [getxattrs,stat,read 0~524288] 3.d74acd34) v4 currently no flag points reached
2013-05-16 18:21:41.970883 osd.0 10.64.0.173:6822/19826 629 : [WRN] 211 slow requests, 5 included below; oldest blocked for > 221.295221 secs
2013-05-16 18:21:41.970898 osd.0 10.64.0.173:6822/19826 630 : [WRN] slow request 120.413825 seconds old, received at 2013-05-16 18:19:41.556916: osd_op(client.27152.0:11793588 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:41.970905 osd.0 10.64.0.173:6822/19826 631 : [WRN] slow request 120.411052 seconds old, received at 2013-05-16 18:19:41.559689: osd_op(client.27495.0:5297789 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:41.970912 osd.0 10.64.0.173:6822/19826 632 : [WRN] slow request 60.916200 seconds old, received at 2013-05-16 18:20:41.054541: osd_op(client.25068.0:11840480 21406.4154_970c97357c6f7d1479ecc2c1ae524eab.err [getxattrs,stat,read 0~524288] 3.3e864a98) v4 currently no flag points reached
2013-05-16 18:21:41.970918 osd.0 10.64.0.173:6822/19826 633 : [WRN] slow request 30.880928 seconds old, received at 2013-05-16 18:21:11.089813: osd_op(client.25068.0:11841418 21406.4154_970c97357c6f7d1479ecc2c1ae524eab.err [getxattrs,stat,read 0~524288] 3.3e864a98) v4 currently no flag points reached
2013-05-16 18:21:41.970924 osd.0 10.64.0.173:6822/19826 634 : [WRN] slow request 30.611807 seconds old, received at 2013-05-16 18:21:11.358934: osd_op(client.25068.0:11841421 4320.11_c/ce/Rimbaud_in_Museum.JPG [getxattrs,stat,read 0~524288] 3.597aa411) v4 currently no flag points reached
2013-05-16 18:21:42.971136 osd.0 10.64.0.173:6822/19826 635 : [WRN] 213 slow requests, 5 included below; oldest blocked for > 222.295518 secs
2013-05-16 18:21:42.971144 osd.0 10.64.0.173:6822/19826 636 : [WRN] slow request 120.098438 seconds old, received at 2013-05-16 18:19:42.872600: osd_op(client.27495.0:5297853 10267.550_f/fa/Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg/180px-Aerial_view_of_west_Hadleigh_-_geograph.org.uk_-_1594271.jpg [getxattrs,stat,read 0~524288] 3.f5ebda99) v4 currently no flag points reached
2013-05-16 18:21:42.971148 osd.0 10.64.0.173:6822/19826 637 : [WRN] slow request 60.638149 seconds old, received at 2013-05-16 18:20:42.332889: osd_op(client.25068.0:11840513 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:42.971152 osd.0 10.64.0.173:6822/19826 638 : [WRN] slow request 60.431051 seconds old, received at 2013-05-16 18:20:42.539987: osd_op(client.27495.0:5300299 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:42.971158 osd.0 10.64.0.173:6822/19826 639 : [WRN] slow request 30.605137 seconds old, received at 2013-05-16 18:21:12.365901: osd_op(client.27297.0:50136524 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:42.971162 osd.0 10.64.0.173:6822/19826 640 : [WRN] slow request 30.005259 seconds old, received at 2013-05-16 18:21:12.965779: osd_op(client.27495.0:5301138 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:43.971397 osd.0 10.64.0.173:6822/19826 641 : [WRN] 213 slow requests, 2 included below; oldest blocked for > 223.295778 secs
2013-05-16 18:21:43.971409 osd.0 10.64.0.173:6822/19826 642 : [WRN] slow request 120.875716 seconds old, received at 2013-05-16 18:19:43.095582: osd_op(client.25068.0:11838000 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:43.971416 osd.0 10.64.0.173:6822/19826 643 : [WRN] slow request 120.430085 seconds old, received at 2013-05-16 18:19:43.541213: osd_op(client.27152.0:11793646 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:44.799705 osd.65 10.64.32.11:6838/9550 1848 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.697394 secs
2013-05-16 18:21:44.799725 osd.65 10.64.32.11:6838/9550 1849 : [WRN] slow request 60.697394 seconds old, received at 2013-05-16 18:20:44.102237: osd_op(client.25068.0:11840558 10267.561_e/e9/Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/180px-Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg [setxattr user.rgw.idtag (33),setxattr user.rgw.shadow_name (209),call refcount.put] 3.9132b23a) v4 currently waiting for subops from [98,0]
2013-05-16 18:21:45.063935 osd.0 10.64.0.173:6822/19826 644 : [WRN] 213 slow requests, 3 included below; oldest blocked for > 224.388288 secs
2013-05-16 18:21:45.063945 osd.0 10.64.0.173:6822/19826 645 : [WRN] slow request 61.033996 seconds old, received at 2013-05-16 18:20:44.029812: osd_op(client.27297.0:50135674 4143.170_3/3c/Dictionary_of_National_Biography_volume_09.djvu [getxattrs,stat,read 0~524288] 3.f5bd829f) v4 currently no flag points reached
2013-05-16 18:21:45.063953 osd.0 10.64.0.173:6822/19826 646 : [WRN] slow request 60.978432 seconds old, received at 2013-05-16 18:20:44.085376: osd_op(client.25068.0:11840557 21406.10171_7a2ed2ba34ec0f212059f271719ae721.png [getxattrs,stat,read 0~524288] 3.c98c8f5a) v4 currently no flag points reached
2013-05-16 18:21:45.063963 osd.0 10.64.0.173:6822/19826 647 : [WRN] slow request 60.959939 seconds old, received at 2013-05-16 18:20:44.103869: osd_sub_op(client.25068.0:11840558 3.323a 9132b23a/10267.561_e/e9/Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/180px-Bundesarchiv_Bild_101I-721-0398-09A,_Frankreich,_Panzertransport_mit_Eisenbahn.jpg/head//3 [] v 180331'20978 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:46.064208 osd.0 10.64.0.173:6822/19826 648 : [WRN] 213 slow requests, 1 included below; oldest blocked for > 225.388603 secs
2013-05-16 18:21:46.064218 osd.0 10.64.0.173:6822/19826 649 : [WRN] slow request 30.713991 seconds old, received at 2013-05-16 18:21:15.350132: osd_sub_op(client.27297.0:50136627 3.2da3 a0666da3/.dir.4320.35/head//3 [] v 180331'182425 snapset=0=[]:[] snapc=0=[]) v7 currently no flag points reached
2013-05-16 18:21:47.064442 osd.0 10.64.0.173:6822/19826 650 : [WRN] 175 slow requests, 6 included below; oldest blocked for > 226.388827 secs
2013-05-16 18:21:47.064450 osd.0 10.64.0.173:6822/19826 651 : [WRN] slow request 120.983072 seconds old, received at 2013-05-16 18:19:46.081275: osd_op(client.27152.0:11793704 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat,read 0~524288] 3.857f39f8) v4 currently no flag points reached
2013-05-16 18:21:47.064456 osd.0 10.64.0.173:6822/19826 652 : [WRN] slow request 120.638119 seconds old, received at 2013-05-16 18:19:46.426228: osd_op(client.27495.0:5297990 10264.13_1/12/Powerpoint_mac_2008_icon.png/75px-Powerpoint_mac_2008_icon.png [getxattrs,stat,read 0~524288] 3.7b921e36) v4 currently no flag points reached
2013-05-16 18:21:47.064460 osd.0 10.64.0.173:6822/19826 653 : [WRN] slow request 120.558299 seconds old, received at 2013-05-16 18:19:46.506048: osd_op(client.25068.0:11838079 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:47.064464 osd.0 10.64.0.173:6822/19826 654 : [WRN] slow request 120.543401 seconds old, received at 2013-05-16 18:19:46.520946: osd_op(client.27152.0:11793718 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat,read 0~524288] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:47.064487 osd.0 10.64.0.173:6822/19826 655 : [WRN] slow request 60.948051 seconds old, received at 2013-05-16 18:20:46.116296: osd_op(client.27297.0:50135714 10267.583_archive/0/06/20070301131335!Dnipro_Basin_River_Town_International.png/117px-Dnipro_Basin_River_Town_International.png [getxattrs,stat,read 0~524288] 3.ba7e338c) v4 currently no flag points reached
2013-05-16 18:21:48.064779 osd.0 10.64.0.173:6822/19826 656 : [WRN] 212 slow requests, 6 included below; oldest blocked for > 227.389127 secs
2013-05-16 18:21:48.064796 osd.0 10.64.0.173:6822/19826 657 : [WRN] slow request 120.009030 seconds old, received at 2013-05-16 18:19:48.055617: osd_op(client.27495.0:5298102 10267.450_archive/b/b9/20080806165933!South_asia.jpg/93px-South_asia.jpg [getxattrs,stat,read 0~524288] 3.63ca4146) v4 currently no flag points reached
2013-05-16 18:21:48.064803 osd.0 10.64.0.173:6822/19826 658 : [WRN] slow request 61.660463 seconds old, received at 2013-05-16 18:20:46.404184: osd_op(client.25068.0:11840630 .dir.10267.376 [call rgw.bucket_prepare_op] 3.9a9b0184) v4 currently no flag points reached
2013-05-16 18:21:48.064809 osd.0 10.64.0.173:6822/19826 659 : [WRN] slow request 60.962693 seconds old, received at 2013-05-16 18:20:47.101954: osd_sub_op(client.27152.0:11796137 3.3e5f 6a3c3e5f/.dir.4465.36/head//3 [] v 180331'186399 snapset=0=[]:[] snapc=0=[]) v7 currently commit sent
2013-05-16 18:21:48.064815 osd.0 10.64.0.173:6822/19826 660 : [WRN] slow request 60.413188 seconds old, received at 2013-05-16 18:20:47.651459: osd_op(client.27495.0:5300436 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:48.064820 osd.0 10.64.0.173:6822/19826 661 : [WRN] slow request 60.388662 seconds old, received at 2013-05-16 18:20:47.675985: osd_op(client.27297.0:50135743 10267.429_5/50/Pocket_radio_open.jpg/683px-Pocket_radio_open.jpg [getxattrs,stat] 3.d7e7ee7d) v4 currently no flag points reached
2013-05-16 18:21:48.231811 mon.0 10.64.0.167:6789/0 456741 : [INF] pgmap v7225107: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 290KB/s wr, 43op/s; 1/806457785 degraded (0.000%)
2013-05-16 18:21:49.587710 mon.0 10.64.0.167:6789/0 456742 : [INF] pgmap v7225108: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3558KB/s wr, 47op/s; 1/806457812 degraded (0.000%)
2013-05-16 18:21:46.052204 osd.28 10.64.0.175:6803/12389 697 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 30.703029 secs
2013-05-16 18:21:46.052216 osd.28 10.64.0.175:6803/12389 698 : [WRN] slow request 30.703029 seconds old, received at 2013-05-16 18:21:15.349117: osd_op(client.27297.0:50136627 .dir.4320.35 [call rgw.bucket_complete_op] 3.a0666da3) v4 currently waiting for subops from [0,125]
2013-05-16 18:21:50.945403 mon.0 10.64.0.167:6789/0 456743 : [INF] pgmap v7225109: 16760 pgs: 118 active, 16623 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2764KB/s wr, 27op/s; 1/806457830 degraded (0.000%)
2013-05-16 18:21:47.726835 osd.78 10.64.32.12:6849/4095 2410 : [WRN] 1 slow requests, 1 included below; oldest blocked for > 60.655215 secs
2013-05-16 18:21:47.726852 osd.78 10.64.32.12:6849/4095 2411 : [WRN] slow request 60.655215 seconds old, received at 2013-05-16 18:20:47.071550: osd_op(client.27152.0:11796137 .dir.4465.36 [call rgw.bucket_prepare_op] 3.6a3c3e5f) v4 currently waiting for subops from [0,142]
2013-05-16 18:21:52.460116 mon.0 10.64.0.167:6789/0 456744 : [INF] pgmap v7225110: 16760 pgs: 16741 active+clean, 1 active+recovery_wait, 18 active+clean+scrubbing+deep; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 1194KB/s wr, 72op/s; 1/806457854 degraded (0.000%)
2013-05-16 18:21:53.874520 mon.0 10.64.0.167:6789/0 456745 : [INF] pgmap v7225111: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 4236KB/s wr, 88op/s; 1/806457896 degraded (0.000%)
2013-05-16 18:21:55.246934 mon.0 10.64.0.167:6789/0 456746 : [INF] pgmap v7225112: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2688KB/s wr, 33op/s; 1/806457917 degraded (0.000%)
2013-05-16 18:21:55.489845 osd.38 10.64.0.176:6831/22256 1640 : [WRN] 14 slow requests, 6 included below; oldest blocked for > 263.852295 secs
2013-05-16 18:21:55.489853 osd.38 10.64.0.176:6831/22256 1641 : [WRN] slow request 60.999091 seconds old, received at 2013-05-16 18:20:54.490654: osd_op(client.27152.0:11796340 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:55.489857 osd.38 10.64.0.176:6831/22256 1642 : [WRN] slow request 60.987689 seconds old, received at 2013-05-16 18:20:54.502056: osd_op(client.27152.0:11796341 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:55.489862 osd.38 10.64.0.176:6831/22256 1643 : [WRN] slow request 60.977079 seconds old, received at 2013-05-16 18:20:54.512666: osd_op(client.27495.0:5300712 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:55.489869 osd.38 10.64.0.176:6831/22256 1644 : [WRN] slow request 60.974135 seconds old, received at 2013-05-16 18:20:54.515610: osd_op(client.27297.0:50136036 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:55.489873 osd.38 10.64.0.176:6831/22256 1645 : [WRN] slow request 60.947910 seconds old, received at 2013-05-16 18:20:54.541835: osd_op(client.25068.0:11840907 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:56.490193 osd.38 10.64.0.176:6831/22256 1646 : [WRN] 16 slow requests, 3 included below; oldest blocked for > 264.852648 secs
2013-05-16 18:21:56.490209 osd.38 10.64.0.176:6831/22256 1647 : [WRN] slow request 61.922027 seconds old, received at 2013-05-16 18:20:54.568071: osd_op(client.25068.0:11840908 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:56.490219 osd.38 10.64.0.176:6831/22256 1648 : [WRN] slow request 61.898179 seconds old, received at 2013-05-16 18:20:54.591919: osd_op(client.27495.0:5300717 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:56.490228 osd.38 10.64.0.176:6831/22256 1649 : [WRN] slow request 61.778593 seconds old, received at 2013-05-16 18:20:54.711505: osd_op(client.27297.0:50136041 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:21:56.536081 mon.0 10.64.0.167:6789/0 456747 : [INF] pgmap v7225113: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 38817B/s wr, 23op/s; 1/806457923 degraded (0.000%)
2013-05-16 18:21:49.740183 osd.0 10.64.0.173:6822/19826 662 : [WRN] 92 slow requests, 5 included below; oldest blocked for > 182.696018 secs
2013-05-16 18:21:49.740205 osd.0 10.64.0.173:6822/19826 663 : [WRN] slow request 60.568180 seconds old, received at 2013-05-16 18:20:49.171884: osd_op(client.27297.0:50135819 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat,read 0~524288] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:21:49.740212 osd.0 10.64.0.173:6822/19826 664 : [WRN] slow request 60.318150 seconds old, received at 2013-05-16 18:20:49.421914: osd_op(client.27297.0:50135833 21406.1730_925418464e919cae531010594fe36100.map [getxattrs,stat,read 0~524288] 3.a56bb46b) v4 currently no flag points reached
2013-05-16 18:21:49.740231 osd.0 10.64.0.173:6822/19826 665 : [WRN] slow request 33.095575 seconds old, received at 2013-05-16 18:21:16.644489: osd_op(client.27495.0:5301267 4465.40_4/43/Federațiunea_1868-01-05,_nr._2.pdf [getxattrs,stat,read 0~524288] 3.7fc693d6) v4 currently no flag points reached
2013-05-16 18:21:49.740237 osd.0 10.64.0.173:6822/19826 666 : [WRN] slow request 32.386533 seconds old, received at 2013-05-16 18:21:17.353531: osd_op(client.27297.0:50136701 10267.603_1/16/DNA_orbit_animated.gif/180px-DNA_orbit_animated.gif [getxattrs,stat,read 0~524288] 3.8498218b) v4 currently no flag points reached
2013-05-16 18:21:49.740242 osd.0 10.64.0.173:6822/19826 667 : [WRN] slow request 30.055056 seconds old, received at 2013-05-16 18:21:19.685008: osd_op(client.27495.0:5301378 6900.2024_1/3/6/136ed7f81226edbb12a37aef4ea1067c.png [getxattrs,stat] 3.b4aed8de) v4 currently no flag points reached
2013-05-16 18:21:50.740815 osd.0 10.64.0.173:6822/19826 668 : [WRN] 28 slow requests, 2 included below; oldest blocked for > 79.370370 secs
2013-05-16 18:21:50.740826 osd.0 10.64.0.173:6822/19826 669 : [WRN] slow request 30.525491 seconds old, received at 2013-05-16 18:21:20.215260: osd_op(client.25068.0:11841646 21406.4154_c191f7d049d8014e54a784ea7d6620a4.map [getxattrs,stat,read 0~524288] 3.323ddf60) v4 currently no flag points reached
2013-05-16 18:21:50.740833 osd.0 10.64.0.173:6822/19826 670 : [WRN] slow request 30.492066 seconds old, received at 2013-05-16 18:21:20.248685: osd_op(client.27297.0:50136792 6900.2080_5/e/e/5ee89e20c6045f8563cef772bf1130fe.png [getxattrs,stat] 3.84c68730) v4 currently no flag points reached
2013-05-16 18:21:57.942026 mon.0 10.64.0.167:6789/0 456748 : [INF] pgmap v7225114: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 892KB/s wr, 121op/s; 1/806457950 degraded (0.000%)
2013-05-16 18:21:59.522386 mon.0 10.64.0.167:6789/0 456749 : [INF] pgmap v7225115: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1243KB/s wr, 17op/s; 1/806457962 degraded (0.000%)
2013-05-16 18:22:01.121358 mon.0 10.64.0.167:6789/0 456750 : [INF] pgmap v7225116: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 3650KB/s wr, 38op/s; 1/806457986 degraded (0.000%)
2013-05-16 18:22:02.478089 mon.0 10.64.0.167:6789/0 456751 : [INF] pgmap v7225117: 16760 pgs: 16741 active+clean, 18 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 4097KB/s wr, 33op/s; 1/806458013 degraded (0.000%)
2013-05-16 18:22:03.949016 mon.0 10.64.0.167:6789/0 456752 : [INF] pgmap v7225118: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 738KB/s wr, 42op/s; 1/806458034 degraded (0.000%)
2013-05-16 18:22:05.405059 mon.0 10.64.0.167:6789/0 456753 : [INF] pgmap v7225119: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2022KB/s wr, 50op/s; 1/806458064 degraded (0.000%)
2013-05-16 18:22:06.794458 mon.0 10.64.0.167:6789/0 456754 : [INF] pgmap v7225120: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2962KB/s wr, 35op/s; 1/806458088 degraded (0.000%)
2013-05-16 18:22:00.759005 osd.109 10.64.32.15:6805/28447 1539 : [INF] 3.11c3 deep-scrub ok
2013-05-16 18:22:01.470047 osd.0 10.64.0.173:6822/19826 671 : [WRN] 2 slow requests, 1 included below; oldest blocked for > 75.065812 secs
2013-05-16 18:22:01.470059 osd.0 10.64.0.173:6822/19826 672 : [WRN] slow request 30.024227 seconds old, received at 2013-05-16 18:21:31.445769: osd_op(client.25068.0:11842184 21406.10555_032a51ba004d7693c7a7c24641c3df2e.png [getxattrs,stat] 3.857f39f8) v4 currently started
2013-05-16 18:22:08.228791 mon.0 10.64.0.167:6789/0 456755 : [INF] pgmap v7225121: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 352KB/s wr, 42op/s; 1/806458109 degraded (0.000%)
2013-05-16 18:22:09.664317 mon.0 10.64.0.167:6789/0 456756 : [INF] pgmap v7225122: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 368KB/s wr, 15op/s; 1/806458118 degraded (0.000%)
2013-05-16 18:22:11.016398 mon.0 10.64.0.167:6789/0 456757 : [INF] pgmap v7225123: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 129KB/s wr, 19op/s; 1/806458130 degraded (0.000%)
2013-05-16 18:22:12.659609 mon.0 10.64.0.167:6789/0 456758 : [INF] pgmap v7225124: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 10602B/s wr, 5op/s; 1/806458133 degraded (0.000%)
2013-05-16 18:22:14.062348 mon.0 10.64.0.167:6789/0 456759 : [INF] pgmap v7225125: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 585KB/s wr, 101op/s; 1/806458157 degraded (0.000%)
2013-05-16 18:22:15.496168 osd.38 10.64.0.176:6831/22256 1650 : [WRN] 17 slow requests, 1 included below; oldest blocked for > 283.858632 secs
2013-05-16 18:22:15.496177 osd.38 10.64.0.176:6831/22256 1651 : [WRN] slow request 30.572792 seconds old, received at 2013-05-16 18:21:44.923290: osd_op(client.27297.0:50137799 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:15.527259 mon.0 10.64.0.167:6789/0 456760 : [INF] pgmap v7225126: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 75112B/s wr, 49op/s; 1/806458187 degraded (0.000%)
2013-05-16 18:22:16.977726 mon.0 10.64.0.167:6789/0 456761 : [INF] pgmap v7225127: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 198KB/s wr, 23op/s; 1/806458202 degraded (0.000%)
2013-05-16 18:22:18.467273 mon.0 10.64.0.167:6789/0 456762 : [INF] pgmap v7225128: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 389KB/s wr, 55op/s; 1/806458229 degraded (0.000%)
2013-05-16 18:22:20.223399 mon.0 10.64.0.167:6789/0 456763 : [INF] pgmap v7225129: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 560KB/s wr, 59op/s; 1/806458259 degraded (0.000%); recovering 0 o/s, 0B/s, 48573 key/s
2013-05-16 18:22:21.595737 mon.0 10.64.0.167:6789/0 456764 : [INF] pgmap v7225130: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 2802KB/s wr, 12op/s; 1/806458262 degraded (0.000%)
2013-05-16 18:22:23.043754 mon.0 10.64.0.167:6789/0 456765 : [INF] pgmap v7225131: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 57452B/s wr, 21op/s; 1/806458274 degraded (0.000%)
2013-05-16 18:22:24.449866 mon.0 10.64.0.167:6789/0 456766 : [INF] pgmap v7225132: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 1719KB/s wr, 37op/s; 1/806458298 degraded (0.000%)
2013-05-16 18:22:25.822463 mon.0 10.64.0.167:6789/0 456767 : [INF] pgmap v7225133: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 2186KB/s wr, 21op/s; 1/806458310 degraded (0.000%)
2013-05-16 18:22:21.497691 osd.38 10.64.0.176:6831/22256 1652 : [WRN] 17 slow requests, 4 included below; oldest blocked for > 289.860145 secs
2013-05-16 18:22:21.497701 osd.38 10.64.0.176:6831/22256 1653 : [WRN] slow request 120.858532 seconds old, received at 2013-05-16 18:20:20.639063: osd_op(client.27297.0:50134711 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:21.497707 osd.38 10.64.0.176:6831/22256 1654 : [WRN] slow request 120.849775 seconds old, received at 2013-05-16 18:20:20.647820: osd_op(client.25068.0:11839898 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:21.497713 osd.38 10.64.0.176:6831/22256 1655 : [WRN] slow request 120.826154 seconds old, received at 2013-05-16 18:20:20.671441: osd_op(client.27297.0:50134713 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:21.497719 osd.38 10.64.0.176:6831/22256 1656 : [WRN] slow request 120.726237 seconds old, received at 2013-05-16 18:20:20.771358: osd_op(client.25068.0:11839902 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:26.498618 osd.38 10.64.0.176:6831/22256 1657 : [WRN] 17 slow requests, 2 included below; oldest blocked for > 294.861101 secs
2013-05-16 18:22:26.498628 osd.38 10.64.0.176:6831/22256 1658 : [WRN] slow request 120.244808 seconds old, received at 2013-05-16 18:20:26.253743: osd_op(client.27495.0:5299738 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:26.498636 osd.38 10.64.0.176:6831/22256 1659 : [WRN] slow request 120.231454 seconds old, received at 2013-05-16 18:20:26.267097: osd_op(client.25068.0:11840100 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:27.128594 mon.0 10.64.0.167:6789/0 456768 : [INF] pgmap v7225134: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 10675KB/s wr, 75op/s; 1/806458343 degraded (0.000%)
2013-05-16 18:22:28.617859 mon.0 10.64.0.167:6789/0 456769 : [INF] pgmap v7225135: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43236 GB data, 133 TB used, 126 TB / 259 TB avail; 19911KB/s wr, 60op/s; 1/806458376 degraded (0.000%)
2013-05-16 18:22:29.908624 mon.0 10.64.0.167:6789/0 456770 : [INF] pgmap v7225136: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 38955KB/s wr, 83op/s; 1/806458418 degraded (0.000%)
2013-05-16 18:22:31.346594 mon.0 10.64.0.167:6789/0 456771 : [INF] pgmap v7225137: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 63687KB/s wr, 152op/s; 1/806458490 degraded (0.000%)
2013-05-16 18:22:32.710986 mon.0 10.64.0.167:6789/0 456772 : [INF] pgmap v7225138: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 26510KB/s wr, 57op/s; 1/806458523 degraded (0.000%)
2013-05-16 18:22:34.166770 mon.0 10.64.0.167:6789/0 456773 : [INF] pgmap v7225139: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 26925KB/s wr, 71op/s; 1/806458559 degraded (0.000%)
2013-05-16 18:22:35.597971 mon.0 10.64.0.167:6789/0 456774 : [INF] pgmap v7225140: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 15034KB/s wr, 55op/s; 1/806458595 degraded (0.000%)
2013-05-16 18:22:29.499342 osd.38 10.64.0.176:6831/22256 1660 : [WRN] 18 slow requests, 1 included below; oldest blocked for > 297.861832 secs
2013-05-16 18:22:29.499352 osd.38 10.64.0.176:6831/22256 1661 : [WRN] slow request 30.220094 seconds old, received at 2013-05-16 18:21:59.279188: osd_op(client.27495.0:5302713 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:35.500690 osd.38 10.64.0.176:6831/22256 1662 : [WRN] 18 slow requests, 1 included below; oldest blocked for > 303.863175 secs
2013-05-16 18:22:35.500701 osd.38 10.64.0.176:6831/22256 1663 : [WRN] slow request 120.873459 seconds old, received at 2013-05-16 18:20:34.627166: osd_op(client.25068.0:11840339 .dir.10267.498 [call rgw.bucket_prepare_op] 3.a2457d08) v4 currently waiting for degraded object
2013-05-16 18:22:37.104165 mon.0 10.64.0.167:6789/0 456775 : [INF] pgmap v7225141: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 892KB/s wr, 40op/s; 1/806458613 degraded (0.000%)
2013-05-16 18:22:38.460279 mon.0 10.64.0.167:6789/0 456776 : [INF] pgmap v7225142: 16760 pgs: 16742 active+clean, 17 active+clean+scrubbing+deep, 1 active+recovering; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 18828B/s wr, 4op/s; 1/806458616 degraded (0.000%)
2013-05-16 18:22:39.857992 mon.0 10.64.0.167:6789/0 456777 : [INF] pgmap v7225143: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 87155B/s wr, 36op/s; recovering 0 o/s, 0B/s, 481K key/s
2013-05-16 18:22:41.368690 mon.0 10.64.0.167:6789/0 456778 : [INF] pgmap v7225144: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 890KB/s wr, 56op/s
2013-05-16 18:22:42.783768 mon.0 10.64.0.167:6789/0 456779 : [INF] pgmap v7225145: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3438KB/s wr, 75op/s
2013-05-16 18:22:44.206707 mon.0 10.64.0.167:6789/0 456780 : [INF] pgmap v7225146: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 3915KB/s wr, 69op/s
2013-05-16 18:22:45.832490 mon.0 10.64.0.167:6789/0 456781 : [INF] pgmap v7225147: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 783KB/s wr, 66op/s
2013-05-16 18:22:47.219396 mon.0 10.64.0.167:6789/0 456782 : [INF] pgmap v7225148: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 5215KB/s wr, 36op/s
2013-05-16 18:22:48.835662 mon.0 10.64.0.167:6789/0 456783 : [INF] pgmap v7225149: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 672KB/s wr, 40op/s
2013-05-16 18:22:50.192037 mon.0 10.64.0.167:6789/0 456784 : [INF] pgmap v7225150: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail
2013-05-16 18:22:51.531401 mon.0 10.64.0.167:6789/0 456785 : [INF] pgmap v7225151: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 178KB/s wr, 20op/s
2013-05-16 18:22:51.395956 osd.11 10.64.0.173:6869/13701 1593 : [INF] 3.1766 deep-scrub ok
2013-05-16 18:22:53.037841 mon.0 10.64.0.167:6789/0 456786 : [INF] pgmap v7225152: 16760 pgs: 16743 active+clean, 17 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 16157B/s wr, 18op/s
2013-05-16 18:22:54.435255 mon.0 10.64.0.167:6789/0 456787 : [INF] pgmap v7225153: 16760 pgs: 16744 active+clean, 16 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 45041B/s wr, 17op/s
2013-05-16 18:22:55.799530 mon.0 10.64.0.167:6789/0 456788 : [INF] pgmap v7225154: 16760 pgs: 16744 active+clean, 16 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 446KB/s wr, 10op/s
2013-05-16 18:22:57.248046 mon.0 10.64.0.167:6789/0 456789 : [INF] pgmap v7225155: 16760 pgs: 16744 active+clean, 16 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 197KB/s wr, 37op/s
2013-05-16 18:22:58.486996 mon.0 10.64.0.167:6789/0 456790 : [INF] pgmap v7225156: 16760 pgs: 16744 active+clean, 16 active+clean+scrubbing+deep; 43237 GB data, 133 TB used, 126 TB / 259 TB avail; 0B/s rd, 528KB/s wr, 44op/s
(3-3/13)