Project

General

Profile

Actions

Bug #15127

closed

fsx tests taking too long and possibly stuck

Added by Vasu Kulkarni about 8 years ago. Updated almost 8 years ago.

Status:
Closed
Priority:
High
Assignee:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I have noticed in the rbd suite, that fsx tests are taking just too long and they might be stuck, I am seeing
the same pattern for few of the fsx tests, this run was on jewel

logs: http://magna002.ceph.redhat.com/vasu-2016-03-11_19:44:11-rbd-jewel---basic-magna/215196/teuthology.log

  description: rbd/librbd/{cache/writeback.yaml cachepool/none.yaml clusters/fixed-3.yaml
    copy-on-read/on.yaml fs/xfs.yaml msgr-failures/few.yaml workloads/fsx.yaml}

2016-03-13T22:01:48.778 INFO:teuthology.orchestra.run.magna051.stdout:14285 trunc    from 0x377badb to 0x5718734
2016-03-13T22:01:48.880 INFO:teuthology.orchestra.run.magna051.stdout:14286 read    0x35ef2e0 thru    0x35f6e37    (0x7b58 bytes)
2016-03-13T22:01:48.881 INFO:teuthology.orchestra.run.magna051.stdout:14287 write    0x4c3d733 thru    0x4c41020    (0x38ee bytes)
2016-03-13T22:01:48.905 INFO:teuthology.orchestra.run.magna051.stdout:14288 punch    from 0x13f83 to 0x1aca0, (0x6d1d bytes)
2016-03-13T22:01:48.927 INFO:teuthology.orchestra.run.magna051.stdout:14289 read    0x5d65d4 thru    0x5dc03c    (0x5a69 bytes)
2016-03-13T22:01:48.928 INFO:teuthology.orchestra.run.magna051.stdout:14291 clone    130 order 23 su 32768 sc 14
2016-03-13T22:01:50.607 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone129 from 0x5718734 (overlap 0xcfeede) to 0xa658b1
2016-03-13T22:01:51.701 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #128, image image_client.0-clone128 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent129
2016-03-13T22:01:51.957 INFO:teuthology.orchestra.run.magna051.stdout:14294 trunc    from 0x5718734 to 0x264fa1b
2016-03-13T22:01:54.432 INFO:teuthology.orchestra.run.magna051.stdout:14296 trunc    from 0x264fa1b to 0x13c8f9
2016-03-13T22:01:55.330 INFO:teuthology.orchestra.run.magna051.stdout:14299 write    0x8156190 thru    0x816352d    (0xd39e bytes)
2016-03-13T22:01:55.910 INFO:teuthology.orchestra.run.magna051.stdout:14300 read    0x804e346 thru    0x805370a    (0x53c5 bytes)
2016-03-13T22:01:55.912 INFO:teuthology.orchestra.run.magna051.stdout:14301 punch    from 0x18b9631 to 0x18c6a2c, (0xd3fb bytes)
2016-03-13T22:01:56.191 INFO:teuthology.orchestra.run.magna051.stdout:14303 read    0x74ee6f5 thru    0x74f4d11    (0x661d bytes)
2016-03-13T22:01:56.192 INFO:teuthology.orchestra.run.magna051.stdout:14306 read    0x548947f thru    0x548cc40    (0x37c2 bytes)
2016-03-13T22:01:56.193 INFO:teuthology.orchestra.run.magna051.stdout:14307 read    0x4d5928a thru    0x4d5f071    (0x5de8 bytes)
2016-03-13T22:01:56.560 INFO:teuthology.orchestra.run.magna051.stdout:14309 write    0xd9bd750 thru    0xd9cb4fa    (0xddab bytes)
2016-03-13T22:01:56.916 INFO:teuthology.orchestra.run.magna051.stdout:14310 read    0x226885c thru    0x2277fe5    (0xf78a bytes)
2016-03-13T22:01:57.029 INFO:teuthology.orchestra.run.magna051.stdout:14314 write    0x67c9ed1 thru    0x67d708e    (0xd1be bytes)
2016-03-13T22:01:57.190 INFO:teuthology.orchestra.run.magna051.stdout:14315 punch    from 0x6536586 to 0x6542ab5, (0xc52f bytes)
2016-03-13T22:01:57.363 INFO:teuthology.orchestra.run.magna051.stdout:14316 write    0x7f0d262 thru    0x7f0f9ba    (0x2759 bytes)
2016-03-13T22:01:57.581 INFO:teuthology.orchestra.run.magna051.stdout:14317 punch    from 0x49b318b to 0x49c16f1, (0xe566 bytes)
2016-03-13T22:01:57.819 INFO:teuthology.orchestra.run.magna051.stdout:14318 write    0x9b51cfb thru    0x9b5dded    (0xc0f3 bytes)
2016-03-13T22:01:58.237 INFO:teuthology.orchestra.run.magna051.stdout:14319 read    0xf94637 thru    0xf98ac4    (0x448e bytes)
2016-03-13T22:01:58.239 INFO:teuthology.orchestra.run.magna051.stdout:14320 punch    from 0x4eb0e40 to 0x4ebb4bf, (0xa67f bytes)
2016-03-13T22:01:58.295 INFO:teuthology.orchestra.run.magna051.stdout:14321 write    0xa2ce9e5 thru    0xa2da644    (0xbc60 bytes)
2016-03-13T22:01:58.567 INFO:teuthology.orchestra.run.magna051.stdout:14322 read    0xcaf67f7 thru    0xcb00813    (0xa01d bytes)
2016-03-13T22:01:58.569 INFO:teuthology.orchestra.run.magna051.stdout:14323 punch    from 0x33cb6d1 to 0x33db2e3, (0xfc12 bytes)
2016-03-13T22:01:58.646 INFO:teuthology.orchestra.run.magna051.stdout:14324 clone    131 order 22 su 262144 sc 14
2016-03-13T22:02:11.344 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone130 from 0xd9cb4fb (overlap 0x13c8f9) to 0x7e9fd
2016-03-13T22:02:13.181 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #129, image image_client.0-clone129 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent130
2016-03-13T22:02:13.794 INFO:teuthology.orchestra.run.magna051.stdout:14327 read    0x309f217 thru    0x30a7b54    (0x893e bytes)
2016-03-13T22:02:14.196 INFO:teuthology.orchestra.run.magna051.stdout:14331 punch    from 0x77cab9b to 0x77cdc30, (0x3095 bytes)
2016-03-13T22:02:14.905 INFO:teuthology.orchestra.run.magna051.stdout:14333 trunc    from 0xd9cb4fb to 0xb344278
2016-03-13T22:03:28.554 INFO:teuthology.orchestra.run.magna051.stdout:14334 write    0xaeaced8 thru    0xaeb5342    (0x846b bytes)
2016-03-13T22:03:28.599 INFO:teuthology.orchestra.run.magna051.stdout:14336 read    0x8576cce thru    0x857a46a    (0x379d bytes)
2016-03-13T22:03:28.607 INFO:teuthology.orchestra.run.magna051.stdout:14337 write    0x9cbe9f3 thru    0x9cc227d    (0x388b bytes)
2016-03-13T22:03:28.690 INFO:teuthology.orchestra.run.magna051.stdout:14339 write    0x42608ff thru    0x4269b7e    (0x9280 bytes)
2016-03-13T22:03:28.757 INFO:teuthology.orchestra.run.magna051.stdout:14341 write    0x526bdca thru    0x5274e87    (0x90be bytes)
2016-03-13T22:03:28.811 INFO:teuthology.orchestra.run.magna051.stdout:14342 punch    from 0x773671a to 0x773b615, (0x4efb bytes)
2016-03-13T22:03:28.853 INFO:teuthology.orchestra.run.magna051.stdout:14343 punch    from 0xa6bf4e8 to 0xa6cb86f, (0xc387 bytes)
2016-03-13T22:03:28.918 INFO:teuthology.orchestra.run.magna051.stdout:14344 read    0xa6a01d9 thru    0xa6a42ae    (0x40d6 bytes)
2016-03-13T22:03:28.920 INFO:teuthology.orchestra.run.magna051.stdout:14346 write    0xa8e9373 thru    0xa8ec775    (0x3403 bytes)
2016-03-13T22:03:28.950 INFO:teuthology.orchestra.run.magna051.stdout:14347 write    0xa6523e7 thru    0xa65f74f    (0xd369 bytes)
2016-03-13T22:03:29.077 INFO:teuthology.orchestra.run.magna051.stdout:14349 read    0xaa44fd5 thru    0xaa4e602    (0x962e bytes)
2016-03-13T22:03:29.080 INFO:teuthology.orchestra.run.magna051.stdout:14350 write    0x5ace203 thru    0x5ad0a35    (0x2833 bytes)
2016-03-13T22:03:29.105 INFO:teuthology.orchestra.run.magna051.stdout:14351 trunc    from 0xb344278 to 0xd9d5257
2016-03-13T22:03:29.140 INFO:teuthology.orchestra.run.magna051.stdout:14352 punch    from 0xc5a0692 to 0xc5a82f0, (0x7c5e bytes)
2016-03-13T22:03:29.235 INFO:teuthology.orchestra.run.magna051.stdout:14353 read    0xbf6a1d1 thru    0xbf6a63a    (0x46a bytes)
2016-03-13T22:03:29.236 INFO:teuthology.orchestra.run.magna051.stdout:14354 punch    from 0x4b27433 to 0x4b2e0a9, (0x6c76 bytes)
2016-03-13T22:03:29.384 INFO:teuthology.orchestra.run.magna051.stdout:14355 write    0x42cb676 thru    0x42cb839    (0x1c4 bytes)
2016-03-13T22:03:29.641 INFO:teuthology.orchestra.run.magna051.stdout:14357 read    0xb109073 thru    0xb10cc66    (0x3bf4 bytes)
2016-03-13T22:03:29.644 INFO:teuthology.orchestra.run.magna051.stdout:14358 read    0x46af1a1 thru    0x46b6c55    (0x7ab5 bytes)
2016-03-13T22:03:29.654 INFO:teuthology.orchestra.run.magna051.stdout:14359 read    0x355c44 thru    0x365326    (0xf6e3 bytes)
2016-03-13T22:03:29.731 INFO:teuthology.orchestra.run.magna051.stdout:14363 read    0x296ade3 thru    0x296f7ab    (0x49c9 bytes)
2016-03-13T22:03:29.897 INFO:teuthology.orchestra.run.magna051.stdout:14364 write    0x7afa8b5 thru    0x7b020bd    (0x7809 bytes)
2016-03-13T22:03:30.152 INFO:teuthology.orchestra.run.magna051.stdout:14365 write    0xc0857a9 thru    0xc08ebf3    (0x944b bytes)
2016-03-13T22:03:30.285 INFO:teuthology.orchestra.run.magna051.stdout:14366 write    0x22f6e78 thru    0x2303da8    (0xcf31 bytes)
2016-03-13T22:03:30.579 INFO:teuthology.orchestra.run.magna051.stdout:14367 punch    from 0x1832878 to 0x183f203, (0xc98b bytes)
2016-03-13T22:03:30.862 INFO:teuthology.orchestra.run.magna051.stdout:14368 write    0xe179a74 thru    0xe17ba9c    (0x2029 bytes)
2016-03-13T22:03:31.204 INFO:teuthology.orchestra.run.magna051.stdout:14369 trunc    from 0xe17ba9d to 0xda27463
2016-03-13T22:03:31.803 INFO:teuthology.orchestra.run.magna051.stdout:14370 read    0xc422a62 thru    0xc42d782    (0xad21 bytes)
2016-03-13T22:03:31.805 INFO:teuthology.orchestra.run.magna051.stdout:14371 punch    from 0x1fd83f2 to 0x1fe73f7, (0xf005 bytes)
2016-03-13T22:03:32.178 INFO:teuthology.orchestra.run.magna051.stdout:14372 write    0x8f8af5e thru    0x8f9a169    (0xf20c bytes)
2016-03-13T22:03:32.607 INFO:teuthology.orchestra.run.magna051.stdout:14375 punch    from 0x2e412e3 to 0x2e46236, (0x4f53 bytes)
2016-03-13T22:03:33.188 INFO:teuthology.orchestra.run.magna051.stdout:14376 trunc    from 0xda27463 to 0x9a832b8
2016-03-13T22:03:34.371 INFO:teuthology.orchestra.run.magna051.stdout:14379 write    0xa26797f thru    0xa27346b    (0xbaed bytes)
2016-03-13T22:03:34.567 INFO:teuthology.orchestra.run.magna051.stdout:14380 trunc    from 0xa27346c to 0x14834f9
2016-03-13T22:03:51.021 INFO:teuthology.orchestra.run.magna051.stdout:14381 write    0xbf44301 thru    0xbf53b7b    (0xf87b bytes)
2016-03-13T22:03:51.076 INFO:teuthology.orchestra.run.magna051.stdout:14382 clone    132 order 18 su 65536 sc 11
2016-03-13T22:04:14.631 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone131 from 0xbf53b7c (overlap 0x14834f9) to 0x20d218
2016-03-13T22:04:15.598 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #130, image image_client.0-clone130 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent131
2016-03-13T22:04:15.695 INFO:teuthology.orchestra.run.magna051.stdout:14383 read    0x2f48ad thru    0x2f9030    (0x4784 bytes)
2016-03-13T22:04:15.761 INFO:teuthology.orchestra.run.magna051.stdout:14385 read    0x54eaa41 thru    0x54eca5d    (0x201d bytes)
2016-03-13T22:04:15.763 INFO:teuthology.orchestra.run.magna051.stdout:14386 punch    from 0xa82475e to 0xa831b29, (0xd3cb bytes)
2016-03-13T22:04:16.165 INFO:teuthology.orchestra.run.magna051.stdout:14388 write    0x8b105ed thru    0x8b1a8a4    (0xa2b8 bytes)
2016-03-13T22:04:16.569 INFO:teuthology.orchestra.run.magna051.stdout:14389 write    0x6e4101d thru    0x6e42ad2    (0x1ab6 bytes)
2016-03-13T22:04:16.737 INFO:teuthology.orchestra.run.magna051.stdout:14390 write    0x2d65f36 thru    0x2d7541f    (0xf4ea bytes)
2016-03-13T22:04:17.252 INFO:teuthology.orchestra.run.magna051.stdout:14391 write    0x3458e5d thru    0x3459d75    (0xf19 bytes)
2016-03-13T22:04:17.585 INFO:teuthology.orchestra.run.magna051.stdout:14392 write    0xbf02135 thru    0xbf02f2c    (0xdf8 bytes)
2016-03-13T22:04:17.839 INFO:teuthology.orchestra.run.magna051.stdout:14393 punch    from 0x63c7016 to 0x63ce8d6, (0x78c0 bytes)
2016-03-13T22:04:18.148 INFO:teuthology.orchestra.run.magna051.stdout:14394 punch    from 0xa1fdbd5 to 0xa2044c3, (0x68ee bytes)
2016-03-13T22:04:18.239 INFO:teuthology.orchestra.run.magna051.stdout:14395 write    0x55476d8 thru    0x5556761    (0xf08a bytes)
2016-03-13T22:04:18.447 INFO:teuthology.orchestra.run.magna051.stdout:14396 write    0x566a1f5 thru    0x5674b42    (0xa94e bytes)
2016-03-13T22:04:18.595 INFO:teuthology.orchestra.run.magna051.stdout:14398 write    0xbc5be7d thru    0xbc5f235    (0x33b9 bytes)
2016-03-13T22:04:19.287 INFO:teuthology.orchestra.run.magna051.stdout:14399 read    0xe2d511 thru    0xe30af2    (0x35e2 bytes)
2016-03-13T22:04:19.336 INFO:teuthology.orchestra.run.magna051.stdout:14400 read    0x9feb5b6 thru    0x9fedae1    (0x252c bytes)
2016-03-13T22:04:19.338 INFO:teuthology.orchestra.run.magna051.stdout:14401 trunc    from 0xbf53b7c to 0xc76687b
2016-03-13T22:04:25.958 INFO:teuthology.orchestra.run.magna051.stdout:14402 punch    from 0x84a03d7 to 0x84af5f0, (0xf219 bytes)
2016-03-13T22:04:26.017 INFO:teuthology.orchestra.run.magna051.stdout:14404 read    0x5b35280 thru    0x5b3d9e3    (0x8764 bytes)
2016-03-13T22:04:26.018 INFO:teuthology.orchestra.run.magna051.stdout:14406 write    0x3c8fa47 thru    0x3c92027    (0x25e1 bytes)
2016-03-13T22:04:26.068 INFO:teuthology.orchestra.run.magna051.stdout:14407 read    0x25e9300 thru    0x25ec53e    (0x323f bytes)
2016-03-13T22:04:26.070 INFO:teuthology.orchestra.run.magna051.stdout:14409 write    0x5d857da thru    0x5d8862b    (0x2e52 bytes)
2016-03-13T22:04:26.104 INFO:teuthology.orchestra.run.magna051.stdout:14410 read    0x6171781 thru    0x6171f5d    (0x7dd bytes)
2016-03-13T22:04:26.105 INFO:teuthology.orchestra.run.magna051.stdout:14412 write    0x1c38cbf thru    0x1c43aad    (0xadef bytes)
2016-03-13T22:04:26.158 INFO:teuthology.orchestra.run.magna051.stdout:14413 write    0x4ca5d44 thru    0x4ca71e8    (0x14a5 bytes)
2016-03-13T22:04:26.194 INFO:teuthology.orchestra.run.magna051.stdout:14415 write    0x2f793e thru    0x301b36    (0xa1f9 bytes)
2016-03-13T22:04:26.260 INFO:teuthology.orchestra.run.magna051.stdout:14416 trunc    from 0xc76687b to 0x2f0bfa7
2016-03-13T22:04:26.552 INFO:teuthology.orchestra.run.magna051.stdout:14417 write    0xcae2050 thru    0xcae314c    (0x10fd bytes)
2016-03-13T22:04:26.596 INFO:teuthology.orchestra.run.magna051.stdout:14418 trunc    from 0xcae314d to 0x729233c
2016-03-13T22:04:26.677 INFO:teuthology.orchestra.run.magna051.stdout:14419 trunc    from 0x729233c to 0x626ea8
2016-03-13T22:04:26.852 INFO:teuthology.orchestra.run.magna051.stdout:14420 read    0x140a73 thru    0x141c5f    (0x11ed bytes)
2016-03-13T22:04:26.854 INFO:teuthology.orchestra.run.magna051.stdout:14421 punch    from 0x2c0783 to 0x2c1588, (0xe05 bytes)
2016-03-13T22:05:04.506 INFO:teuthology.orchestra.run.magna051.stdout:14422 write    0x60bfb0d thru    0x60c22b6    (0x27aa bytes)
2016-03-13T22:05:04.662 INFO:teuthology.orchestra.run.magna051.stdout:14423 read    0x60e782 thru    0x619343    (0xabc2 bytes)
2016-03-13T22:05:04.676 INFO:teuthology.orchestra.run.magna051.stdout:14424 trunc    from 0x60c22b7 to 0x1fb14e5
2016-03-13T22:05:04.750 INFO:teuthology.orchestra.run.magna051.stdout:14425 punch    from 0x1833921 to 0x1839de7, (0x64c6 bytes)
2016-03-13T22:05:04.773 INFO:teuthology.orchestra.run.magna051.stdout:14429 trunc    from 0x1fb14e5 to 0x24d598d
2016-03-13T22:05:04.809 INFO:teuthology.orchestra.run.magna051.stdout:14430 clone    133 order 22 su 16384 sc 3
2016-03-13T22:05:05.514 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone132 from 0x24d598d (overlap 0x626ea8) to 0x0
2016-03-13T22:05:12.238 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #131, image image_client.0-clone131 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent132
2016-03-13T22:05:12.672 INFO:teuthology.orchestra.run.magna051.stdout:14431 read    0x1b55175 thru    0x1b58d61    (0x3bed bytes)
2016-03-13T22:05:29.219 INFO:teuthology.orchestra.run.magna051.stdout:14433 read    0x1ad5df thru    0x1b1244    (0x3c66 bytes)
2016-03-13T22:05:29.239 INFO:teuthology.orchestra.run.magna051.stdout:14434 punch    from 0xc27757 to 0xc2bb0d, (0x43b6 bytes)
2016-03-13T22:05:29.378 INFO:teuthology.orchestra.run.magna051.stdout:14437 punch    from 0x2351769 to 0x235f19c, (0xda33 bytes)
2016-03-13T22:05:29.546 INFO:teuthology.orchestra.run.magna051.stdout:14440 write    0x280cf1d thru    0x281094d    (0x3a31 bytes)
2016-03-13T22:05:29.667 INFO:teuthology.orchestra.run.magna051.stdout:14442 punch    from 0xdda8ae to 0xddfa01, (0x5153 bytes)
2016-03-13T22:05:29.739 INFO:teuthology.orchestra.run.magna051.stdout:14444 write    0x3fffa14 thru    0x400d597    (0xdb84 bytes)
2016-03-13T22:05:29.810 INFO:teuthology.orchestra.run.magna051.stdout:14446 punch    from 0x1037b14 to 0x10421ab, (0xa697 bytes)
2016-03-13T22:05:29.838 INFO:teuthology.orchestra.run.magna051.stdout:14447 read    0x3299d34 thru    0x32a3bd2    (0x9e9f bytes)
2016-03-13T22:05:29.873 INFO:teuthology.orchestra.run.magna051.stdout:14449 write    0x6321b19 thru    0x632dadc    (0xbfc4 bytes)
2016-03-13T22:05:29.958 INFO:teuthology.orchestra.run.magna051.stdout:14451 write    0x6d820e5 thru    0x6d872c8    (0x51e4 bytes)
2016-03-13T22:05:30.011 INFO:teuthology.orchestra.run.magna051.stdout:14452 trunc    from 0x6d872c9 to 0x3cc193d
2016-03-13T22:05:30.114 INFO:teuthology.orchestra.run.magna051.stdout:14453 trunc    from 0x3cc193d to 0x92c7160
2016-03-13T22:05:30.149 INFO:teuthology.orchestra.run.magna051.stdout:14454 write    0x4eaf441 thru    0x4eb9544    (0xa104 bytes)
2016-03-13T22:05:30.218 INFO:teuthology.orchestra.run.magna051.stdout:14456 punch    from 0x69e6fe7 to 0x69e9068, (0x2081 bytes)
2016-03-13T22:05:30.307 INFO:teuthology.orchestra.run.magna051.stdout:14458 write    0xb934614 thru    0xb93da87    (0x9474 bytes)
2016-03-13T22:05:30.377 INFO:teuthology.orchestra.run.magna051.stdout:14461 write    0xa7b0ced thru    0xa7b0e8e    (0x1a2 bytes)
2016-03-13T22:05:30.425 INFO:teuthology.orchestra.run.magna051.stdout:14464 read    0xca0a3 thru    0xcb87a    (0x17d8 bytes)
2016-03-13T22:05:30.427 INFO:teuthology.orchestra.run.magna051.stdout:14466 read    0x5da80f thru    0x5e52b0    (0xaaa2 bytes)
2016-03-13T22:05:30.877 INFO:teuthology.orchestra.run.magna051.stdout:14467 write    0xd6057b3 thru    0xd6116d6    (0xbf24 bytes)
2016-03-13T22:05:31.072 INFO:teuthology.orchestra.run.magna051.stdout:14469 write    0x5244622 thru    0x524bebd    (0x789c bytes)
2016-03-13T22:05:31.424 INFO:teuthology.orchestra.run.magna051.stdout:14471 read    0x9604c1a thru    0x960dd7f    (0x9166 bytes)
2016-03-13T22:05:31.425 INFO:teuthology.orchestra.run.magna051.stdout:14472 punch    from 0x19ed451 to 0x19f92c8, (0xbe77 bytes)
2016-03-13T22:05:31.625 INFO:teuthology.orchestra.run.magna051.stdout:14474 write    0x5657b7b thru    0x5663f46    (0xc3cc bytes)
2016-03-13T22:05:31.812 INFO:teuthology.orchestra.run.magna051.stdout:14476 trunc    from 0xd6116d7 to 0xc2c9549
2016-03-13T22:05:32.104 INFO:teuthology.orchestra.run.magna051.stdout:14477 punch    from 0x4f53019 to 0x4f5b17b, (0x8162 bytes)
2016-03-13T22:05:32.186 INFO:teuthology.orchestra.run.magna051.stdout:14478 write    0xb0c1118 thru    0xb0c6fa0    (0x5e89 bytes)
2016-03-13T22:05:32.326 INFO:teuthology.orchestra.run.magna051.stdout:14480 write    0x1b633c4 thru    0x1b64f29    (0x1b66 bytes)
2016-03-13T22:05:32.471 INFO:teuthology.orchestra.run.magna051.stdout:14481 read    0x8ec0407 thru    0x8ec1703    (0x12fd bytes)
2016-03-13T22:05:32.472 INFO:teuthology.orchestra.run.magna051.stdout:14483 read    0x3a8f8d6 thru    0x3a97d6b    (0x8496 bytes)
2016-03-13T22:05:32.473 INFO:teuthology.orchestra.run.magna051.stdout:14485 write    0x7e6cec2 thru    0x7e73349    (0x6488 bytes)
2016-03-13T22:05:32.583 INFO:teuthology.orchestra.run.magna051.stdout:14486 write    0x4f7db5c thru    0x4f7e690    (0xb35 bytes)
2016-03-13T22:05:32.685 INFO:teuthology.orchestra.run.magna051.stdout:14487 write    0xa276255 thru    0xa282f48    (0xccf4 bytes)
2016-03-13T22:05:33.112 INFO:teuthology.orchestra.run.magna051.stdout:14488 write    0xca4bf6b thru    0xca50045    (0x40db bytes)
2016-03-13T22:05:33.209 INFO:teuthology.orchestra.run.magna051.stdout:14489 write    0x62fe74 thru    0x6326ef    (0x287c bytes)
2016-03-13T22:05:33.235 INFO:teuthology.orchestra.run.magna051.stdout:14492 read    0x8ef1c9 thru    0x8fb51b    (0xc353 bytes)
2016-03-13T22:05:33.236 INFO:teuthology.orchestra.run.magna051.stdout:14493 punch    from 0x8af9fd3 to 0x8afd2b1, (0x32de bytes)
2016-03-13T22:05:33.350 INFO:teuthology.orchestra.run.magna051.stdout:14494 read    0xbba6c2d thru    0xbbb1e9c    (0xb270 bytes)
2016-03-13T22:05:33.501 INFO:teuthology.orchestra.run.magna051.stdout:14495 write    0x5afd3d3 thru    0x5b0a44e    (0xd07c bytes)
2016-03-13T22:05:33.822 INFO:teuthology.orchestra.run.magna051.stdout:14497 write    0xec5b23c thru    0xec6a536    (0xf2fb bytes)
2016-03-13T22:05:34.236 INFO:teuthology.orchestra.run.magna051.stdout:14498 trunc    from 0xec6a537 to 0x91e0f38
2016-03-13T22:05:34.741 INFO:teuthology.orchestra.run.magna051.stdout:14499 write    0x998d75c thru    0x99952f6    (0x7b9b bytes)
2016-03-13T22:05:35.078 INFO:teuthology.orchestra.run.magna051.stdout:14500 write    0x569d2e6 thru    0x56a9cda    (0xc9f5 bytes)
2016-03-13T22:05:35.223 INFO:teuthology.orchestra.run.magna051.stdout:14501 punch    from 0x134e015 to 0x135d266, (0xf251 bytes)
2016-03-13T22:05:35.791 INFO:teuthology.orchestra.run.magna051.stdout:14503 write    0xb680367 thru    0xb684552    (0x41ec bytes)
2016-03-13T22:05:36.164 INFO:teuthology.orchestra.run.magna051.stdout:14508 write    0x5e9ab5a thru    0x5e9c6e1    (0x1b88 bytes)
2016-03-13T22:05:36.513 INFO:teuthology.orchestra.run.magna051.stdout:14509 read    0x999b9fa thru    0x99a144f    (0x5a56 bytes)
2016-03-13T22:05:36.592 INFO:teuthology.orchestra.run.magna051.stdout:14511 read    0x18bafd3 thru    0x18cab67    (0xfb95 bytes)
2016-03-13T22:05:36.687 INFO:teuthology.orchestra.run.magna051.stdout:14512 read    0x2c637db thru    0x2c67073    (0x3899 bytes)
2016-03-13T22:05:36.736 INFO:teuthology.orchestra.run.magna051.stdout:14513 read    0xa7ee83 thru    0xa8887e    (0x99fc bytes)
2016-03-13T22:05:36.738 INFO:teuthology.orchestra.run.magna051.stdout:14514 read    0x436b2d8 thru    0x43748dd    (0x9606 bytes)
2016-03-13T22:05:36.740 INFO:teuthology.orchestra.run.magna051.stdout:14515 read    0xaa56bf8 thru    0xaa63b84    (0xcf8d bytes)
2016-03-13T22:05:36.741 INFO:teuthology.orchestra.run.magna051.stdout:14516 read    0xa33c97a thru    0xa3422ce    (0x5955 bytes)
2016-03-13T22:05:36.742 INFO:teuthology.orchestra.run.magna051.stdout:14519 read    0x99510a0 thru    0x9957dd7    (0x6d38 bytes)
2016-03-13T22:05:36.821 INFO:teuthology.orchestra.run.magna051.stdout:14520 read    0xa65d077 thru    0xa65d9c7    (0x951 bytes)
2016-03-13T22:05:36.822 INFO:teuthology.orchestra.run.magna051.stdout:14521 write    0x7204577 thru    0x720c41b    (0x7ea5 bytes)
2016-03-13T22:05:37.170 INFO:teuthology.orchestra.run.magna051.stdout:14526 trunc    from 0xb684553 to 0xe82ed4e
2016-03-13T22:05:37.202 INFO:teuthology.orchestra.run.magna051.stdout:14527 write    0x64bbfa3 thru    0x64c77d0    (0xb82e bytes)
2016-03-13T22:05:37.237 INFO:teuthology.orchestra.run.magna051.stdout:14528 read    0x4b03b84 thru    0x4b12226    (0xe6a3 bytes)
2016-03-13T22:05:37.367 INFO:teuthology.orchestra.run.magna051.stdout:14529 write    0x527bbc1 thru    0x528b7b8    (0xfbf8 bytes)
2016-03-13T22:05:37.533 INFO:teuthology.orchestra.run.magna051.stdout:14530 write    0xa20c626 thru    0xa213789    (0x7164 bytes)
2016-03-13T22:05:37.792 INFO:teuthology.orchestra.run.magna051.stdout:14532 write    0x430f3e thru    0x437ff9    (0x70bc bytes)
2016-03-13T22:05:37.839 INFO:teuthology.orchestra.run.magna051.stdout:14533 read    0xca06f2 thru    0xca12f2    (0xc01 bytes)
2016-03-13T22:05:37.842 INFO:teuthology.orchestra.run.magna051.stdout:14536 write    0x555e718 thru    0x555f5c4    (0xead bytes)
2016-03-13T22:05:37.918 INFO:teuthology.orchestra.run.magna051.stdout:14539 punch    from 0x97a7121 to 0x97af6b8, (0x8597 bytes)
2016-03-13T22:05:38.024 INFO:teuthology.orchestra.run.magna051.stdout:14540 clone    134 order 19 su 16384 sc 5
2016-03-13T22:05:42.088 INFO:teuthology.orchestra.run.magna051.stdout:leaving image image_client.0-clone133 intact
2016-03-13T22:09:08.948 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #132, image image_client.0-clone132 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent133
2016-03-13T22:09:09.119 INFO:teuthology.orchestra.run.magna051.stdout:14541 write    0xc133c92 thru    0xc13ac11    (0x6f80 bytes)
2016-03-13T22:09:09.249 INFO:teuthology.orchestra.run.magna051.stdout:14545 write    0x976d9d2 thru    0x976da1a    (0x49 bytes)
2016-03-13T22:09:09.282 INFO:teuthology.orchestra.run.magna051.stdout:14546 read    0x76f90e6 thru    0x76fefd4    (0x5eef bytes)
2016-03-13T22:09:09.288 INFO:teuthology.orchestra.run.magna051.stdout:14547 clone    135 order 24 su 8388608 sc 2
2016-03-13T22:09:10.532 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone134 from 0xe82ed4e (overlap 0xe82ed4e) to 0x0
2016-03-13T22:10:04.862 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #133, image image_client.0-clone133 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent134
2016-03-13T22:11:32.766 INFO:teuthology.orchestra.run.magna051.stdout:14550 trunc    from 0xe82ed4e to 0x1fc3021
2016-03-13T22:11:34.428 INFO:teuthology.orchestra.run.magna051.stdout:14553 write    0x67fe741 thru    0x680d9e2    (0xf2a2 bytes)
2016-03-13T22:11:35.147 INFO:teuthology.orchestra.run.magna051.stdout:14555 write    0x21ffb5d thru    0x220b76a    (0xbc0e bytes)
2016-03-13T22:11:35.658 INFO:teuthology.orchestra.run.magna051.stdout:14556 trunc    from 0x680d9e3 to 0x61bde43
2016-03-13T22:11:36.511 INFO:teuthology.orchestra.run.magna051.stdout:14560 punch    from 0x779079 to 0x78118b, (0x8112 bytes)
2016-03-13T22:11:38.119 INFO:teuthology.orchestra.run.magna051.stdout:14561 trunc    from 0x61bde43 to 0xd4e0071
2016-03-13T22:11:38.533 INFO:teuthology.orchestra.run.magna051.stdout:14563 write    0xc56d533 thru    0xc57a23f    (0xcd0d bytes)
2016-03-13T22:11:38.920 INFO:teuthology.orchestra.run.magna051.stdout:14564 read    0xbffe9b9 thru    0xbfff742    (0xd8a bytes)
2016-03-13T22:11:38.922 INFO:teuthology.orchestra.run.magna051.stdout:14566 punch    from 0xeff2cc to 0xf07e55, (0x8b89 bytes)
2016-03-13T22:11:39.183 INFO:teuthology.orchestra.run.magna051.stdout:14567 read    0x23334df thru    0x2336a15    (0x3537 bytes)
2016-03-13T22:11:39.290 INFO:teuthology.orchestra.run.magna051.stdout:14570 punch    from 0x9d64413 to 0x9d662a4, (0x1e91 bytes)
2016-03-13T22:11:39.387 INFO:teuthology.orchestra.run.magna051.stdout:14571 read    0x280f6e3 thru    0x2819d7c    (0xa69a bytes)
2016-03-13T22:11:39.388 INFO:teuthology.orchestra.run.magna051.stdout:14573 write    0x9d0088b thru    0x9d01b87    (0x12fd bytes)
2016-03-13T22:11:39.589 INFO:teuthology.orchestra.run.magna051.stdout:14575 trunc    from 0xd4e0071 to 0xbc48d0c
2016-03-13T22:11:40.073 INFO:teuthology.orchestra.run.magna051.stdout:14576 read    0x81252a5 thru    0x812e59c    (0x92f8 bytes)
2016-03-13T22:11:40.074 INFO:teuthology.orchestra.run.magna051.stdout:14577 punch    from 0x1cb7631 to 0x1cbd563, (0x5f32 bytes)
2016-03-13T22:11:40.289 INFO:teuthology.orchestra.run.magna051.stdout:14578 punch    from 0x55b3578 to 0x55b76c2, (0x414a bytes)
2016-03-13T22:11:40.997 INFO:teuthology.orchestra.run.magna051.stdout:14580 write    0x4d80bbb thru    0x4d8c805    (0xbc4b bytes)
2016-03-13T22:11:41.297 INFO:teuthology.orchestra.run.magna051.stdout:14581 read    0x6e261ed thru    0x6e2c1bd    (0x5fd1 bytes)
2016-03-13T22:11:41.298 INFO:teuthology.orchestra.run.magna051.stdout:14582 read    0x497bd2a thru    0x4988aed    (0xcdc4 bytes)
2016-03-13T22:11:41.333 INFO:teuthology.orchestra.run.magna051.stdout:14583 read    0xffcab thru    0x10b377    (0xb6cd bytes)
2016-03-13T22:11:41.427 INFO:teuthology.orchestra.run.magna051.stdout:14584 trunc    from 0xbc48d0c to 0xbe90da4
2016-03-13T22:11:41.659 INFO:teuthology.orchestra.run.magna051.stdout:14585 write    0x48bf3e3 thru    0x48cc9a0    (0xd5be bytes)
2016-03-13T22:11:41.874 INFO:teuthology.orchestra.run.magna051.stdout:14586 trunc    from 0xbe90da4 to 0x6dcd20b
2016-03-13T22:11:42.075 INFO:teuthology.orchestra.run.magna051.stdout:14587 read    0x640fbb8 thru    0x641f9f7    (0xfe40 bytes)
2016-03-13T22:11:42.455 INFO:teuthology.orchestra.run.magna051.stdout:14589 write    0xd424c3a thru    0xd425fe3    (0x13aa bytes)
2016-03-13T22:11:42.757 INFO:teuthology.orchestra.run.magna051.stdout:14590 write    0x7e3f2df thru    0x7e40eea    (0x1c0c bytes)
2016-03-13T22:11:42.925 INFO:teuthology.orchestra.run.magna051.stdout:14591 write    0x994e422 thru    0x995dfcb    (0xfbaa bytes)
2016-03-13T22:11:43.196 INFO:teuthology.orchestra.run.magna051.stdout:14592 trunc    from 0xd425fe4 to 0x62f9ce6
2016-03-13T22:11:43.940 INFO:teuthology.orchestra.run.magna051.stdout:14593 write    0xa7b2b26 thru    0xa7b2bb6    (0x91 bytes)
2016-03-13T22:11:44.258 INFO:teuthology.orchestra.run.magna051.stdout:14594 clone    136 order 23 su 2097152 sc 5
2016-03-13T22:11:50.783 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone135 from 0xa7b2bb7 (overlap 0x1fc3021) to 0x1968ce7
2016-03-13T22:11:53.765 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #134, image image_client.0-clone134 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent135
2016-03-13T22:11:54.444 INFO:teuthology.orchestra.run.magna051.stdout:14595 read    0x3a953ed thru    0x3aa116c    (0xbd80 bytes)
2016-03-13T22:11:55.426 INFO:teuthology.orchestra.run.magna051.stdout:14596 write    0xd1e4274 thru    0xd1f00d4    (0xbe61 bytes)
2016-03-13T22:11:55.654 INFO:teuthology.orchestra.run.magna051.stdout:14597 trunc    from 0xd1f00d5 to 0x7dcdc1b
2016-03-13T22:11:56.455 INFO:teuthology.orchestra.run.magna051.stdout:14599 read    0x6e006e7 thru    0x6e0ae91    (0xa7ab bytes)
2016-03-13T22:11:56.466 INFO:teuthology.orchestra.run.magna051.stdout:14601 trunc    from 0x7dcdc1b to 0x3105206
2016-03-13T22:11:57.941 INFO:teuthology.orchestra.run.magna051.stdout:14602 write    0x8c70a7 thru    0x8cf044    (0x7f9e bytes)
2016-03-13T22:11:58.180 INFO:teuthology.orchestra.run.magna051.stdout:14603 trunc    from 0x3105206 to 0x62569b8
2016-03-13T22:11:58.376 INFO:teuthology.orchestra.run.magna051.stdout:14604 write    0x3d6b8ca thru    0x3d6f221    (0x3958 bytes)
2016-03-13T22:11:58.575 INFO:teuthology.orchestra.run.magna051.stdout:14607 read    0x356549b thru    0x3569f32    (0x4a98 bytes)
2016-03-13T22:11:58.618 INFO:teuthology.orchestra.run.magna051.stdout:14608 punch    from 0x1b849fc to 0x1b9473f, (0xfd43 bytes)
2016-03-13T22:11:59.453 INFO:teuthology.orchestra.run.magna051.stdout:14611 flatten
2016-03-13T22:12:00.523 INFO:teuthology.orchestra.run.magna051.stdout:14612 trunc    from 0x62569b8 to 0x65d7853
2016-03-13T22:12:01.496 INFO:teuthology.orchestra.run.magna051.stdout:14614 write    0x9837ce6 thru    0x9842059    (0xa374 bytes)
2016-03-13T22:12:01.772 INFO:teuthology.orchestra.run.magna051.stdout:14616 read    0x33bb92b thru    0x33c426b    (0x8941 bytes)
2016-03-13T22:12:02.284 INFO:teuthology.orchestra.run.magna051.stdout:14617 write    0xe908429 thru    0xe916a85    (0xe65d bytes)
2016-03-13T22:12:02.555 INFO:teuthology.orchestra.run.magna051.stdout:14618 write    0x4a3071c thru    0x4a33753    (0x3038 bytes)
2016-03-13T22:12:02.677 INFO:teuthology.orchestra.run.magna051.stdout:14619 trunc    from 0xe916a86 to 0xed1549a
2016-03-13T22:12:02.871 INFO:teuthology.orchestra.run.magna051.stdout:14620 read    0x3550c0c thru    0x35516f8    (0xaed bytes)
2016-03-13T22:12:02.873 INFO:teuthology.orchestra.run.magna051.stdout:14623 write    0x907218b thru    0x90781ff    (0x6075 bytes)
2016-03-13T22:12:03.063 INFO:teuthology.orchestra.run.magna051.stdout:14624 write    0x2355542 thru    0x235754e    (0x200d bytes)
2016-03-13T22:12:03.325 INFO:teuthology.orchestra.run.magna051.stdout:14625 punch    from 0xd193cda to 0xd1a1e27, (0xe14d bytes)
2016-03-13T22:12:03.444 INFO:teuthology.orchestra.run.magna051.stdout:14627 read    0x8d7a609 thru    0x8d7d8ce    (0x32c6 bytes)
2016-03-13T22:12:03.448 INFO:teuthology.orchestra.run.magna051.stdout:14628 trunc    from 0xed1549a to 0xd42f87f
2016-03-13T22:12:03.697 INFO:teuthology.orchestra.run.magna051.stdout:14629 clone    137 order 21 su 1048576 sc 3
2016-03-13T22:12:47.594 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #135, image image_client.0-clone135 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent136
2016-03-13T22:12:48.000 INFO:teuthology.orchestra.run.magna051.stdout:14630 punch    from 0x1d6f414 to 0x1d72dcb, (0x39b7 bytes)
2016-03-13T22:12:49.334 INFO:teuthology.orchestra.run.magna051.stdout:14631 read    0x9973a44 thru    0x99758ae    (0x1e6b bytes)
2016-03-13T22:12:49.407 INFO:teuthology.orchestra.run.magna051.stdout:14632 read    0x41cbf4a thru    0x41cd667    (0x171e bytes)
2016-03-13T22:12:49.612 INFO:teuthology.orchestra.run.magna051.stdout:14633 trunc    from 0xd42f87f to 0xa4e56c8
2016-03-13T22:14:00.218 INFO:teuthology.orchestra.run.magna051.stdout:14634 write    0x73de367 thru    0x73e3d8d    (0x5a27 bytes)
2016-03-13T22:14:00.329 INFO:teuthology.orchestra.run.magna051.stdout:14635 write    0xc1466f2 thru    0xc14f7e5    (0x90f4 bytes)
2016-03-13T22:14:00.372 INFO:teuthology.orchestra.run.magna051.stdout:14637 write    0xbc6b315 thru    0xbc757f2    (0xa4de bytes)
2016-03-13T22:14:00.407 INFO:teuthology.orchestra.run.magna051.stdout:14638 read    0x4ccb2b7 thru    0x4ccbfad    (0xcf7 bytes)
2016-03-13T22:14:00.409 INFO:teuthology.orchestra.run.magna051.stdout:14641 read    0xac5e68f thru    0xac60fa1    (0x2913 bytes)
2016-03-13T22:14:00.410 INFO:teuthology.orchestra.run.magna051.stdout:14642 write    0xbdded04 thru    0xbde37c3    (0x4ac0 bytes)
2016-03-13T22:14:00.455 INFO:teuthology.orchestra.run.magna051.stdout:14646 read    0x96bd8e5 thru    0x96c614b    (0x8867 bytes)
2016-03-13T22:14:00.457 INFO:teuthology.orchestra.run.magna051.stdout:14647 read    0x3e78816 thru    0x3e7ac3a    (0x2425 bytes)
2016-03-13T22:14:00.459 INFO:teuthology.orchestra.run.magna051.stdout:14649 punch    from 0x90d6322 to 0x90e49b7, (0xe695 bytes)
2016-03-13T22:14:00.557 INFO:teuthology.orchestra.run.magna051.stdout:14650 read    0x95bb84 thru    0x95eb73    (0x2ff0 bytes)
2016-03-13T22:14:00.577 INFO:teuthology.orchestra.run.magna051.stdout:14652 trunc    from 0xc14f7e6 to 0x6601ea3
2016-03-13T22:14:00.943 INFO:teuthology.orchestra.run.magna051.stdout:14653 write    0xa119775 thru    0xa11d577    (0x3e03 bytes)
2016-03-13T22:14:01.083 INFO:teuthology.orchestra.run.magna051.stdout:14654 trunc    from 0xa11d578 to 0x78f59a5
2016-03-13T22:14:01.499 INFO:teuthology.orchestra.run.magna051.stdout:14655 read    0x32bfcb1 thru    0x32ceacd    (0xee1d bytes)
2016-03-13T22:14:01.506 INFO:teuthology.orchestra.run.magna051.stdout:14656 trunc    from 0x78f59a5 to 0x6280784
2016-03-13T22:14:01.813 INFO:teuthology.orchestra.run.magna051.stdout:14657 read    0x170a880 thru    0x171521a    (0xa99b bytes)
2016-03-13T22:14:01.990 INFO:teuthology.orchestra.run.magna051.stdout:14658 read    0x1d45b20 thru    0x1d47425    (0x1906 bytes)
2016-03-13T22:14:01.992 INFO:teuthology.orchestra.run.magna051.stdout:14659 read    0xced4e6 thru    0xcf7b06    (0xa621 bytes)
2016-03-13T22:14:02.101 INFO:teuthology.orchestra.run.magna051.stdout:14660 write    0xf5b6d0 thru    0xf6a9ed    (0xf31e bytes)
2016-03-13T22:14:02.205 INFO:teuthology.orchestra.run.magna051.stdout:14661 punch    from 0xe61430 to 0xe6c9b4, (0xb584 bytes)
2016-03-13T22:14:02.273 INFO:teuthology.orchestra.run.magna051.stdout:14663 write    0x5c48c3d thru    0x5c5443b    (0xb7ff bytes)
2016-03-13T22:14:02.510 INFO:teuthology.orchestra.run.magna051.stdout:14665 read    0x5eddfbf thru    0x5edfcb5    (0x1cf7 bytes)
2016-03-13T22:14:02.512 INFO:teuthology.orchestra.run.magna051.stdout:14667 write    0x5a5d677 thru    0x5a6d5aa    (0xff34 bytes)
2016-03-13T22:14:02.620 INFO:teuthology.orchestra.run.magna051.stdout:14668 read    0x4a0846a thru    0x4a0e035    (0x5bcc bytes)
2016-03-13T22:14:02.628 INFO:teuthology.orchestra.run.magna051.stdout:14669 trunc    from 0x6280784 to 0x37ca26f
2016-03-13T22:14:03.608 INFO:teuthology.orchestra.run.magna051.stdout:14670 write    0x3fbe308 thru    0x3fc7204    (0x8efd bytes)
2016-03-13T22:14:03.750 INFO:teuthology.orchestra.run.magna051.stdout:14671 punch    from 0x327a036 to 0x3285602, (0xb5cc bytes)
2016-03-13T22:14:03.799 INFO:teuthology.orchestra.run.magna051.stdout:14674 trunc    from 0x3fc7205 to 0x9f33451
2016-03-13T22:14:03.861 INFO:teuthology.orchestra.run.magna051.stdout:14675 punch    from 0x4db88d5 to 0x4dc78bf, (0xefea bytes)
2016-03-13T22:14:03.975 INFO:teuthology.orchestra.run.magna051.stdout:14676 write    0x3712ba thru    0x379b5f    (0x88a6 bytes)
2016-03-13T22:14:04.478 INFO:teuthology.orchestra.run.magna051.stdout:14678 write    0xed12e86 thru    0xed1f1c8    (0xc343 bytes)
2016-03-13T22:14:04.508 INFO:teuthology.orchestra.run.magna051.stdout:14680 read    0x5325373 thru    0x532a440    (0x50ce bytes)
2016-03-13T22:14:04.509 INFO:teuthology.orchestra.run.magna051.stdout:14681 punch    from 0x1f34b16 to 0x1f44294, (0xf77e bytes)
2016-03-13T22:14:04.639 INFO:teuthology.orchestra.run.magna051.stdout:14682 punch    from 0x37c703 to 0x38444f, (0x7d4c bytes)
2016-03-13T22:14:04.821 INFO:teuthology.orchestra.run.magna051.stdout:14684 trunc    from 0xed1f1c9 to 0x5254625
2016-03-13T22:14:04.900 INFO:teuthology.orchestra.run.magna051.stdout:14685 read    0x1644ad5 thru    0x16533ba    (0xe8e6 bytes)
2016-03-13T22:14:05.047 INFO:teuthology.orchestra.run.magna051.stdout:14686 read    0x1c72b8 thru    0x1d6e5c    (0xfba5 bytes)
2016-03-13T22:14:05.048 INFO:teuthology.orchestra.run.magna051.stdout:14687 punch    from 0x808d44 to 0x80bd52, (0x300e bytes)
2016-03-13T22:14:05.461 INFO:teuthology.orchestra.run.magna051.stdout:14688 read    0x42fe515 thru    0x4305cf5    (0x77e1 bytes)
2016-03-13T22:14:05.462 INFO:teuthology.orchestra.run.magna051.stdout:14689 write    0x4287c1e thru    0x428b861    (0x3c44 bytes)
2016-03-13T22:14:05.555 INFO:teuthology.orchestra.run.magna051.stdout:14690 write    0x4329cd6 thru    0x43377d4    (0xdaff bytes)
2016-03-13T22:14:05.586 INFO:teuthology.orchestra.run.magna051.stdout:14692 clone    138 order 19 su 8192 sc 7
2016-03-13T22:14:08.127 INFO:teuthology.orchestra.run.magna051.stdout:flattening image image_client.0-clone137
2016-03-13T22:14:09.939 INFO:teuthology.orchestra.run.magna051.stdout:checking clone #136, image image_client.0-clone136 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent137
2016-03-13T22:14:11.091 INFO:teuthology.orchestra.run.magna051.stdout:14695 punch    from 0x320e9db to 0x321d2fb, (0xe920 bytes)
2016-03-13T22:14:11.829 INFO:teuthology.orchestra.run.magna051.stdout:14696 read    0x31142ee thru    0x31145e8    (0x2fb bytes)
2016-03-13T22:14:11.870 INFO:teuthology.orchestra.run.magna051.stdout:14697 punch    from 0x5c5fc to 0x68ff9, (0xc9fd bytes)
2016-03-13T22:14:12.615 INFO:teuthology.orchestra.run.magna051.stdout:14698 read    0x524edd4 thru    0x5254624    (0x5851 bytes)
2016-03-13T22:14:13.320 INFO:teuthology.orchestra.run.magna051.stdout:14699 write    0x96e4f67 thru    0x96eeb5c    (0x9bf6 bytes)
2016-03-13T22:14:13.733 INFO:teuthology.orchestra.run.magna051.stdout:14700 write    0xcbacc28 thru    0xcbb1a29    (0x4e02 bytes)
2016-03-13T22:14:13.938 INFO:teuthology.orchestra.run.magna051.stdout:14703 read    0xac6e633 thru    0xac7124f    (0x2c1d bytes)
2016-03-13T22:14:13.939 INFO:teuthology.orchestra.run.magna051.stdout:14704 write    0x5e0ced4 thru    0x5e0f208    (0x2335 bytes)
2016-03-13T22:14:14.051 INFO:teuthology.orchestra.run.magna051.stdout:14706 read    0x6cd5edf thru    0x6cd9df6    (0x3f18 bytes)
2016-03-13T22:14:14.052 INFO:teuthology.orchestra.run.magna051.stdout:14707 write    0xb4764d1 thru    0xb47bc83    (0x57b3 bytes)
2016-03-13T22:14:14.442 INFO:teuthology.orchestra.run.magna051.stdout:14709 write    0xe516070 thru    0xe518bbe    (0x2b4f bytes)
2016-03-13T22:14:14.823 INFO:teuthology.orchestra.run.magna051.stdout:14710 trunc    from 0xe518bbf to 0x909c387
2016-03-13T22:14:15.269 INFO:teuthology.orchestra.run.magna051.stdout:14711 write    0x775600e thru    0x77593ff    (0x33f2 bytes)
2016-03-13T22:14:15.446 INFO:teuthology.orchestra.run.magna051.stdout:14712 trunc    from 0x909c387 to 0x988304f
2016-03-13T22:14:15.884 INFO:teuthology.orchestra.run.magna051.stdout:14713 read    0x4e034de thru    0x4e04161    (0xc84 bytes)
2016-03-13T22:14:15.922 INFO:teuthology.orchestra.run.magna051.stdout:14714 read    0x27ad104 thru    0x27aeec9    (0x1dc6 bytes)
2016-03-13T22:14:15.926 INFO:teuthology.orchestra.run.magna051.stdout:14716 read    0x7487a4f thru    0x7491eaa    (0xa45c bytes)
2016-03-13T22:14:15.945 INFO:teuthology.orchestra.run.magna051.stdout:14717 trunc    from 0x988304f to 0xb52562
2016-03-13T22:14:17.215 INFO:teuthology.orchestra.run.magna051.stdout:14718 punch    from 0xb27350 to 0xb36125, (0xedd5 bytes)
2016-03-13T22:14:17.480 INFO:teuthology.orchestra.run.magna051.stdout:14719 read    0x8a89c6 thru    0x8b371b    (0xad56 bytes)
2016-03-13T22:14:18.104 INFO:teuthology.orchestra.run.magna051.stdout:14720 write    0x840a864 thru    0x841518c    (0xa929 bytes)
2016-03-13T22:14:18.502 INFO:teuthology.orchestra.run.magna051.stdout:14722 trunc    from 0x841518d to 0x3d6e248
2016-03-13T22:14:19.212 INFO:teuthology.orchestra.run.magna051.stdout:14724 write    0xa91a69f thru    0xa923e96    (0x97f8 bytes)
2016-03-13T22:14:19.501 INFO:teuthology.orchestra.run.magna051.stdout:14725 read    0x8f547f6 thru    0x8f5498d    (0x198 bytes)
2016-03-13T22:14:19.503 INFO:teuthology.orchestra.run.magna051.stdout:14726 punch    from 0x48dd1e2 to 0x48e19af, (0x47cd bytes)
2016-03-13T22:14:19.536 INFO:teuthology.orchestra.run.magna051.stdout:14727 read    0x1531236 thru    0x1539e25    (0x8bf0 bytes)
2016-03-13T22:14:19.537 INFO:teuthology.orchestra.run.magna051.stdout:14728 punch    from 0x798fa7d to 0x799c069, (0xc5ec bytes)
2016-03-13T22:14:19.603 INFO:teuthology.orchestra.run.magna051.stdout:14730 read    0x34c0059 thru    0x34c3ce2    (0x3c8a bytes)
2016-03-13T22:14:19.604 INFO:teuthology.orchestra.run.magna051.stdout:14731 write    0x2a7dba4 thru    0x2a83394    (0x57f1 bytes)
2016-03-13T22:14:19.882 INFO:teuthology.orchestra.run.magna051.stdout:14732 trunc    from 0xa923e97 to 0x37216c
2016-03-13T22:14:20.444 INFO:teuthology.orchestra.run.magna051.stdout:14733 write    0xa2c9cd8 thru    0xa2ccb62    (0x2e8b bytes)
2016-03-13T22:14:20.502 INFO:teuthology.orchestra.run.magna051.stdout:14734 write    0x9f92b6f thru    0x9f96459    (0x38eb bytes)
2016-03-13T22:14:20.570 INFO:teuthology.orchestra.run.magna051.stdout:14735 trunc    from 0xa2ccb63 to 0xbc2bb88
2016-03-13T22:14:20.612 INFO:teuthology.orchestra.run.magna051.stdout:14736 read    0x270e07 thru    0x27f06e    (0xe268 bytes)
2016-03-13T22:14:20.676 INFO:teuthology.orchestra.run.magna051.stdout:14737 read    0xa334bd1 thru    0xa337a18    (0x2e48 bytes)
2016-03-13T22:14:20.677 INFO:teuthology.orchestra.run.magna051.stdout:14739 punch    from 0x3dc0d54 to 0x3dc8fd4, (0x8280 bytes)
2016-03-13T22:14:20.796 INFO:teuthology.orchestra.run.magna051.stdout:14740 write    0x54158d1 thru    0x54181b6    (0x28e6 bytes)
2016-03-13T22:14:20.857 INFO:teuthology.orchestra.run.magna051.stdout:14741 write    0x66d0806 thru    0x66d4794    (0x3f8f bytes)
2016-03-13T22:14:20.920 INFO:teuthology.orchestra.run.magna051.stdout:14742 punch    from 0x1b4b327 to 0x1b52833, (0x750c bytes)
2016-03-13T22:14:20.971 INFO:teuthology.orchestra.run.magna051.stdout:14743 read    0x73384c2 thru    0x73435bc    (0xb0fb bytes)
2016-03-13T22:14:20.973 INFO:teuthology.orchestra.run.magna051.stdout:14744 write    0x3205e42 thru    0x3213501    (0xd6c0 bytes)
2016-03-13T22:14:21.115 INFO:teuthology.orchestra.run.magna051.stdout:14745 trunc    from 0xbc2bb88 to 0x1341a33
2016-03-13T22:14:21.218 INFO:teuthology.orchestra.run.magna051.stdout:14748 read    0x13410f8 thru    0x1341a32    (0x93b bytes)
2016-03-13T22:14:21.279 INFO:teuthology.orchestra.run.magna051.stdout:14749 write    0x89a44cf thru    0x89a94d7    (0x5009 bytes)
2016-03-13T22:14:21.371 INFO:teuthology.orchestra.run.magna051.stdout:14750 write    0x90001c2 thru    0x900413f    (0x3f7e bytes)
2016-03-13T22:14:21.425 INFO:teuthology.orchestra.run.magna051.stdout:14751 punch    from 0xfef009 to 0xff373a, (0x4731 bytes)
2016-03-13T22:14:21.461 INFO:teuthology.orchestra.run.magna051.stdout:14752 trunc    from 0x9004140 to 0x7ba37c2
2016-03-13T22:14:21.527 INFO:teuthology.orchestra.run.magna051.stdout:14753 punch    from 0x564dd26 to 0x5658f0b, (0xb1e5 bytes)
2016-03-13T22:14:21.608 INFO:teuthology.orchestra.run.magna051.stdout:14756 punch    from 0x47c6434 to 0x47c68ab, (0x477 bytes)
2016-03-13T22:14:21.625 INFO:teuthology.orchestra.run.magna051.stdout:14758 write    0x6ae5ff6 thru    0x6af0b1d    (0xab28 bytes)
2016-03-13T22:14:21.724 INFO:teuthology.orchestra.run.magna051.stdout:14761 write    0x75b7e30 thru    0x75c773e    (0xf90f bytes)
2016-03-13T22:14:21.949 INFO:teuthology.orchestra.run.magna051.stdout:14762 read    0x3d3daa5 thru    0x3d47ab9    (0xa015 bytes)
2016-03-13T22:14:22.021 INFO:teuthology.orchestra.run.magna051.stdout:14763 write    0xe723a3d thru    0xe7261fe    (0x27c2 bytes)
2016-03-13T22:14:22.067 INFO:teuthology.orchestra.run.magna051.stdout:14764 write    0x11bbe1f thru    0x11bbf79    (0x15b bytes)
2016-03-13T22:14:22.092 INFO:teuthology.orchestra.run.magna051.stdout:14765 punch    from 0x9db5478 to 0x9dbd046, (0x7bce bytes)
2016-03-13T22:14:22.143 INFO:teuthology.orchestra.run.magna051.stdout:14766 read    0xd26743c thru    0xd26ac46    (0x380b bytes)
2016-03-13T22:14:22.144 INFO:teuthology.orchestra.run.magna051.stdout:14768 punch    from 0xe1be02c to 0xe1ccc18, (0xebec bytes)
2016-03-13T22:14:22.210 INFO:teuthology.orchestra.run.magna051.stdout:14769 punch    from 0xe0c2ffc to 0xe0ccce7, (0x9ceb bytes)
2016-03-13T22:14:22.224 INFO:teuthology.orchestra.run.magna051.stdout:14770 write    0x6b8b3b0 thru    0x6b96f5d    (0xbbae bytes)
2016-03-13T22:14:22.364 INFO:teuthology.orchestra.run.magna051.stdout:14771 punch    from 0xde1a134 to 0xde20641, (0x650d bytes)
2016-03-13T22:14:22.411 INFO:teuthology.orchestra.run.magna051.stdout:14772 write    0x5128f44 thru    0x5134be0    (0xbc9d bytes)
2016-03-13T22:14:22.532 INFO:teuthology.orchestra.run.magna051.stdout:14777 read    0x5e13db8 thru    0x5e1d43b    (0x9684 bytes)
2016-03-13T22:14:22.548 INFO:teuthology.orchestra.run.magna051.stdout:14778 trunc    from 0xe7261ff to 0x9b307df
2016-03-13T22:14:22.632 INFO:teuthology.orchestra.run.magna051.stdout:14779 punch    from 0x676eced to 0x67733f7, (0x470a bytes)
2016-03-13T22:14:22.678 INFO:teuthology.orchestra.run.magna051.stdout:14780 read    0x454c88c thru    0x4554213    (0x7988 bytes)
2016-03-13T22:14:22.679 INFO:teuthology.orchestra.run.magna051.stdout:14782 punch    from 0x1fea262 to 0x1fefc90, (0x5a2e bytes)
2016-03-13T22:14:22.711 INFO:teuthology.orchestra.run.magna051.stdout:14784 punch    from 0x8bef39 to 0x8cd7b7, (0xe87e bytes)
2016-03-13T22:14:22.795 INFO:teuthology.orchestra.run.magna051.stdout:14785 punch    from 0x264e354 to 0x2652938, (0x45e4 bytes)
2016-03-13T22:14:22.828 INFO:teuthology.orchestra.run.magna051.stdout:14788 read    0x51046d5 thru    0x510c944    (0x8270 bytes)
2016-03-13T22:14:22.830 INFO:teuthology.orchestra.run.magna051.stdout:14789 write    0x4fe6be9 thru    0x4ff10a6    (0xa4be bytes)
2016-03-13T22:14:23.001 INFO:teuthology.orchestra.run.magna051.stdout:14791 trunc    from 0x9b307df to 0x3a1ab17
2016-03-13T22:14:23.364 INFO:teuthology.orchestra.run.magna051.stdout:14792 trunc    from 0x3a1ab17 to 0x311c26
2016-03-13T22:14:23.515 INFO:teuthology.orchestra.run.magna051.stdout:14793 read    0x25e3f8 thru    0x26b40e    (0xd017 bytes)
2016-03-13T22:14:23.636 INFO:teuthology.orchestra.run.magna051.stdout:14794 write    0x6b6384e thru    0x6b658fa    (0x20ad bytes)
2016-03-13T22:14:23.683 INFO:teuthology.orchestra.run.magna051.stdout:14795 write    0x41b49f3 thru    0x41c0825    (0xbe33 bytes)
2016-03-13T22:14:23.848 INFO:teuthology.orchestra.run.magna051.stdout:14796 write    0xb42b3f8 thru    0xb434ec6    (0x9acf bytes)
2016-03-13T22:14:23.941 INFO:teuthology.orchestra.run.magna051.stdout:14797 read    0x1eed386 thru    0x1ef64ae    (0x9129 bytes)
2016-03-13T22:14:23.943 INFO:teuthology.orchestra.run.magna051.stdout:14799 write    0x7a6dc95 thru    0x7a71b4c    (0x3eb8 bytes)
2016-03-13T22:14:23.989 INFO:teuthology.orchestra.run.magna051.stdout:14800 punch    from 0xa822408 to 0xa831345, (0xef3d bytes)
2016-03-13T22:14:24.058 INFO:teuthology.orchestra.run.magna051.stdout:14803 punch    from 0x8282766 to 0x8291051, (0xe8eb bytes)
2016-03-13T22:14:24.274 INFO:teuthology.orchestra.run.magna051.stdout:14805 read    0x4caabd3 thru    0x4cb698d    (0xbdbb bytes)
2016-03-13T22:14:24.279 INFO:teuthology.orchestra.run.magna051.stdout:14808 trunc    from 0xb434ec7 to 0xc7b39f9
2016-03-13T22:14:24.323 INFO:teuthology.orchestra.run.magna051.stdout:14809 read    0xa2f3f06 thru    0xa2f780e    (0x3909 bytes)
2016-03-13T22:14:24.324 INFO:teuthology.orchestra.run.magna051.stdout:14811 write    0xbab92bb thru    0xbabe2ed    (0x5033 bytes)
2016-03-13T22:14:24.400 INFO:teuthology.orchestra.run.magna051.stdout:14812 write    0xa98f472 thru    0xa99213d    (0x2ccc bytes)
2016-03-13T22:14:24.429 INFO:teuthology.orchestra.run.magna051.stdout:14814 read    0x6e4903e thru    0x6e50e74    (0x7e37 bytes)
2016-03-13T22:14:24.430 INFO:teuthology.orchestra.run.magna051.stdout:14817 punch    from 0x5e46b85 to 0x5e49754, (0x2bcf bytes)
2016-03-13T22:14:24.474 INFO:teuthology.orchestra.run.magna051.stdout:14818 trunc    from 0xc7b39f9 to 0xa03fec2
2016-03-13T22:14:24.576 INFO:teuthology.orchestra.run.magna051.stdout:14819 punch    from 0x8b8da03 to 0x8b994e2, (0xbadf bytes)
2016-03-13T22:14:24.658 INFO:teuthology.orchestra.run.magna051.stdout:14820 read    0x44f08a6 thru    0x4500042    (0xf79d bytes)
2016-03-13T22:14:24.660 INFO:teuthology.orchestra.run.magna051.stdout:14821 write    0x54c5fba thru    0x54cb0a5    (0x50ec bytes)
2016-03-13T22:14:24.730 INFO:teuthology.orchestra.run.magna051.stdout:14823 read    0x5b0f94b thru    0x5b10d0d    (0x13c3 bytes)
2016-03-13T22:14:24.732 INFO:teuthology.orchestra.run.magna051.stdout:14824 write    0x1fe0aa1 thru    0x1fe618e    (0x56ee bytes)
2016-03-13T22:14:24.821 INFO:teuthology.orchestra.run.magna051.stdout:14825 write    0x1821df8 thru    0x182bd06    (0x9f0f bytes)
2016-03-13T22:14:25.007 INFO:teuthology.orchestra.run.magna051.stdout:14826 punch    from 0xdcf3dd to 0xdd9a70, (0xa693 bytes)
2016-03-13T22:14:25.499 INFO:teuthology.orchestra.run.magna051.stdout:14828 write    0x3143666 thru    0x31466f9    (0x3094 bytes)
2016-03-13T22:14:25.563 INFO:teuthology.orchestra.run.magna051.stdout:14830 trunc    from 0xa03fec2 to 0x77fd32f
2016-03-13T22:14:25.637 INFO:teuthology.orchestra.run.magna051.stdout:14831 read    0x731d40c thru    0x732633f    (0x8f34 bytes)
2016-03-13T22:14:25.638 INFO:teuthology.orchestra.run.magna051.stdout:14835 read    0x5eb26ed thru    0x5eb6ef5    (0x4809 bytes)
2016-03-13T22:14:25.639 INFO:teuthology.orchestra.run.magna051.stdout:14836 read    0x1e7b4e8 thru    0x1e7fb8e    (0x46a7 bytes)
2016-03-13T22:14:25.640 INFO:teuthology.orchestra.run.magna051.stdout:14837 read    0x1581f7c thru    0x15859a4    (0x3a29 bytes)
2016-03-13T22:14:25.642 INFO:teuthology.orchestra.run.magna051.stdout:14840 read    0x5bc8db7 thru    0x5bcb15b    (0x23a5 bytes)
2016-03-13T22:14:25.643 INFO:teuthology.orchestra.run.magna051.stdout:14841 read    0x3401f32 thru    0x340887a    (0x6949 bytes)
2016-03-13T22:14:25.644 INFO:teuthology.orchestra.run.magna051.stdout:14842 read    0x9a3faa thru    0x9ae284    (0xa2db bytes)
2016-03-13T22:14:25.658 INFO:teuthology.orchestra.run.magna051.stdout:14844 trunc    from 0x77fd32f to 0xe7facc7
2016-03-13T22:14:25.688 INFO:teuthology.orchestra.run.magna051.stdout:14847 read    0x8fa05a9 thru    0x8fae41b    (0xde73 bytes)
2016-03-13T22:14:25.689 INFO:teuthology.orchestra.run.magna051.stdout:14848 read    0x69c182e thru    0x69c6d07    (0x54da bytes)
2016-03-13T22:14:25.691 INFO:teuthology.orchestra.run.magna051.stdout:14849 write    0xd309d9a thru    0xd311ec1    (0x8128 bytes)
2016-03-13T22:14:25.776 INFO:teuthology.orchestra.run.magna051.stdout:14850 punch    from 0x57e6313 to 0x57f13dc, (0xb0c9 bytes)
2016-03-13T22:14:26.111 INFO:teuthology.orchestra.run.magna051.stdout:14852 read    0xe03a66b thru    0xe03dacc    (0x3462 bytes)
2016-03-13T22:14:26.146 INFO:teuthology.orchestra.run.magna051.stdout:14854 trunc    from 0xe7facc7 to 0x2d8752c
2016-03-13T22:14:26.298 INFO:teuthology.orchestra.run.magna051.stdout:14855 read    0x35449e thru    0x35eac2    (0xa625 bytes)
2016-03-13T22:14:26.300 INFO:teuthology.orchestra.run.magna051.stdout:14856 read    0xbed508 thru    0xbf152a    (0x4023 bytes)
2016-03-13T22:14:26.302 INFO:teuthology.orchestra.run.magna051.stdout:14857 read    0x145eff7 thru    0x145f981    (0x98b bytes)
2016-03-13T22:14:26.302 INFO:teuthology.orchestra.run.magna051.stdout:14860 read    0xd4d1f8 thru    0xd564f0    (0x92f9 bytes)
2016-03-13T22:14:26.304 INFO:teuthology.orchestra.run.magna051.stdout:14861 read    0x12557d3 thru    0x12650e8    (0xf916 bytes)
2016-03-13T22:14:26.348 INFO:teuthology.orchestra.run.magna051.stdout:14863 write    0x58aefe7 thru    0x58ba5c5    (0xb5df bytes)
2016-03-13T22:14:26.452 INFO:teuthology.orchestra.run.magna051.stdout:14864 write    0x3fb1a1a thru    0x3fb4528    (0x2b0f bytes)
2016-03-13T22:14:26.502 INFO:teuthology.orchestra.run.magna051.stdout:14865 write    0x1755609 thru    0x17583c6    (0x2dbe bytes)
2016-03-13T22:14:26.522 INFO:teuthology.orchestra.run.magna051.stdout:14866 write    0x4364072 thru    0x43661c7    (0x2156 bytes)
2016-03-13T22:14:26.560 INFO:teuthology.orchestra.run.magna051.stdout:14869 read    0x2b5ecf6 thru    0x2b6338a    (0x4695 bytes)
2016-03-13T22:14:26.561 INFO:teuthology.orchestra.run.magna051.stdout:14871 punch    from 0x294c7d9 to 0x295760c, (0xae33 bytes)
2016-03-13T22:14:26.674 INFO:teuthology.orchestra.run.magna051.stdout:14873 write    0xadae440 thru    0xadbe1a2    (0xfd63 bytes)
2016-03-13T22:14:26.778 INFO:teuthology.orchestra.run.magna051.stdout:14874 read    0xa8cd9cb thru    0xa8d85a8    (0xabde bytes)
2016-03-13T22:14:26.815 INFO:teuthology.orchestra.run.magna051.stdout:14878 write    0xce27da6 thru    0xce36d19    (0xef74 bytes)
2016-03-13T22:14:26.919 INFO:teuthology.orchestra.run.magna051.stdout:14886 punch    from 0xa866831 to 0xa86c344, (0x5b13 bytes)
2016-03-13T22:14:26.968 INFO:teuthology.orchestra.run.magna051.stdout:14887 trunc    from 0xce36d1a to 0xb695839
2016-03-13T22:14:27.194 INFO:teuthology.orchestra.run.magna051.stdout:14890 read    0x7954c1f thru    0x795f7a6    (0xab88 bytes)
2016-03-13T22:14:27.195 INFO:teuthology.orchestra.run.magna051.stdout:14892 punch    from 0x859de0e to 0x859f232, (0x1424 bytes)
2016-03-13T22:14:27.239 INFO:teuthology.orchestra.run.magna051.stdout:14893 punch    from 0x2d4686e to 0x2d4bc77, (0x5409 bytes)
2016-03-13T22:14:27.273 INFO:teuthology.orchestra.run.magna051.stdout:14894 read    0x8fcf8fa thru    0x8fdbc5b    (0xc362 bytes)
2016-03-13T22:14:27.274 INFO:teuthology.orchestra.run.magna051.stdout:14895 punch    from 0x873b715 to 0x87479a3, (0xc28e bytes)
2016-03-13T22:14:27.331 INFO:teuthology.orchestra.run.magna051.stdout:14896 write    0x3628142 thru    0x363256b    (0xa42a bytes)
2016-03-13T22:14:27.422 INFO:teuthology.orchestra.run.magna051.stdout:14897 write    0x2639d78 thru    0x263cedd    (0x3166 bytes)
2016-03-13T22:14:27.470 INFO:teuthology.orchestra.run.magna051.stdout:14899 read    0x9779893 thru    0x978937e    (0xfaec bytes)
2016-03-13T22:14:27.471 INFO:teuthology.orchestra.run.magna051.stdout:14900 punch    from 0xa42e828 to 0xa4339a5, (0x517d bytes)
2016-03-13T22:14:27.530 INFO:teuthology.orchestra.run.magna051.stdout:14902 trunc    from 0xb695839 to 0x4a6c528
2016-03-13T22:14:27.629 INFO:teuthology.orchestra.run.magna051.stdout:14903 read    0x3a849bb thru    0x3a88457    (0x3a9d bytes)
2016-03-13T22:14:27.630 INFO:teuthology.orchestra.run.magna051.stdout:14904 read    0x325dc20 thru    0x3265c73    (0x8054 bytes)
2016-03-13T22:14:27.683 INFO:teuthology.orchestra.run.magna051.stdout:14905 write    0x9351c72 thru    0x9355e25    (0x41b4 bytes)
2016-03-13T22:14:27.745 INFO:teuthology.orchestra.run.magna051.stdout:14906 read    0x6ffbe0c thru    0x7003a19    (0x7c0e bytes)
2016-03-13T22:14:27.746 INFO:teuthology.orchestra.run.magna051.stdout:14907 read    0x1025683 thru    0x1025e98    (0x816 bytes)
2016-03-13T22:14:27.754 INFO:teuthology.orchestra.run.magna051.stdout:14908 trunc    from 0x9355e26 to 0x6879e5c
2016-03-13T22:14:27.838 INFO:teuthology.orchestra.run.magna051.stdout:14909 punch    from 0x540c49c to 0x541698c, (0xa4f0 bytes)
2016-03-13T22:14:27.909 INFO:teuthology.orchestra.run.magna051.stdout:14910 punch    from 0x4dcb877 to 0x4dd5793, (0x9f1c bytes)
2016-03-13T22:14:27.992 INFO:teuthology.orchestra.run.magna051.stdout:14911 trunc    from 0x6879e5c to 0x16ec819
2016-03-13T22:14:28.344 INFO:teuthology.orchestra.run.magna051.stdout:14912 trunc    from 0x16ec819 to 0x5e91d9e
2016-03-13T22:14:28.410 INFO:teuthology.orchestra.run.magna051.stdout:14913 read    0x420750b thru    0x421615f    (0xec55 bytes)
2016-03-13T22:14:28.411 INFO:teuthology.orchestra.run.magna051.stdout:14915 read    0x29eeeb thru    0x2ac5b7    (0xd6cd bytes)
2016-03-13T22:14:28.593 INFO:teuthology.orchestra.run.magna051.stdout:14916 write    0x101cc thru    0x14d57    (0x4b8c bytes)
2016-03-13T22:14:28.654 INFO:teuthology.orchestra.run.magna051.stdout:14917 punch    from 0xeb201a to 0xeb48b0, (0x2896 bytes)
2016-03-13T22:14:28.685 INFO:teuthology.orchestra.run.magna051.stdout:14918 read    0x1894de0 thru    0x18964b2    (0x16d3 bytes)
2016-03-13T22:14:28.708 INFO:teuthology.orchestra.run.magna051.stdout:14920 trunc    from 0x5e91d9e to 0xd55848e
2016-03-13T22:14:28.752 INFO:teuthology.orchestra.run.magna051.stdout:14921 write    0xbfef5e5 thru    0xbff8b79    (0x9595 bytes)
2016-03-13T22:14:28.871 INFO:teuthology.orchestra.run.magna051.stdout:14923 trunc    from 0xd55848e to 0x7346449
2016-03-13T22:14:28.960 INFO:teuthology.orchestra.run.magna051.stdout:14926 write    0x4984c08 thru    0x4986919    (0x1d12 bytes)
2016-03-13T22:14:29.013 INFO:teuthology.orchestra.run.magna051.stdout:14928 punch    from 0x587ce5a to 0x5886d88, (0x9f2e bytes)
2016-03-13T22:14:29.083 INFO:teuthology.orchestra.run.magna051.stdout:14929 read    0x329c4fd thru    0x32a23c6    (0x5eca bytes)
2016-03-13T22:14:29.084 INFO:teuthology.orchestra.run.magna051.stdout:14933 read    0x2e44c20 thru    0x2e47f4a    (0x332b bytes)
2016-03-13T22:14:29.085 INFO:teuthology.orchestra.run.magna051.stdout:14934 read    0x5b108f2 thru    0x5b1d4a0    (0xcbaf bytes)
2016-03-13T22:14:29.086 INFO:teuthology.orchestra.run.magna051.stdout:14935 read    0x48d1149 thru    0x48daa33    (0x98eb bytes)
2016-03-13T22:14:29.087 INFO:teuthology.orchestra.run.magna051.stdout:14936 read    0x633c20a thru    0x6346ac5    (0xa8bc bytes)
2016-03-13T22:14:29.127 INFO:teuthology.orchestra.run.magna051.stdout:14937 write    0x9aea686 thru    0x9af3955    (0x92d0 bytes)
2016-03-13T22:14:29.216 INFO:teuthology.orchestra.run.magna051.stdout:14938 read    0x8995631 thru    0x89a0a25    (0xb3f5 bytes)
2016-03-13T22:14:29.217 INFO:teuthology.orchestra.run.magna051.stdout:14939 read    0x82ab03c thru    0x82abbb7    (0xb7c bytes)
2016-03-13T22:14:29.218 INFO:teuthology.orchestra.run.magna051.stdout:14942 read    0x87e5701 thru    0x87e6796    (0x1096 bytes)
2016-03-13T22:14:29.219 INFO:teuthology.orchestra.run.magna051.stdout:14943 punch    from 0x73fb0e5 to 0x7401083, (0x5f9e bytes)
2016-03-13T22:14:29.270 INFO:teuthology.orchestra.run.magna051.stdout:14944 write    0x7f0d27a thru    0x7f171d1    (0x9f58 bytes)
2016-03-13T22:14:29.370 INFO:teuthology.orchestra.run.magna051.stdout:14945 trunc    from 0x9af3956 to 0xb571bc1
2016-03-13T22:14:29.404 INFO:teuthology.orchestra.run.magna051.stdout:14946 punch    from 0x5889618 to 0x5890e72, (0x785a bytes)
2016-03-13T22:14:29.441 INFO:teuthology.orchestra.run.magna051.stdout:14947 trunc    from 0xb571bc1 to 0xe01afc2
2016-03-13T22:14:29.480 INFO:teuthology.orchestra.run.magna051.stdout:14948 read    0x5fe7b59 thru    0x5ff2d8f    (0xb237 bytes)
2016-03-13T22:14:29.481 INFO:teuthology.orchestra.run.magna051.stdout:14949 read    0xc08176a thru    0xc08299a    (0x1231 bytes)
2016-03-13T22:14:29.482 INFO:teuthology.orchestra.run.magna051.stdout:14950 read    0xbdbca89 thru    0xbdc8e33    (0xc3ab bytes)
2016-03-13T22:14:29.520 INFO:teuthology.orchestra.run.magna051.stdout:14951 trunc    from 0xe01afc2 to 0xcc2181
2016-03-13T22:14:29.679 INFO:teuthology.orchestra.run.magna051.stdout:14953 write    0x6998ecd thru    0x69a3696    (0xa7ca bytes)
2016-03-13T22:14:29.766 INFO:teuthology.orchestra.run.magna051.stdout:14955 write    0x1eb2438 thru    0x1ebbfc4    (0x9b8d bytes)
2016-03-13T22:14:29.912 INFO:teuthology.orchestra.run.magna051.stdout:14956 write    0xe241926 thru    0xe244206    (0x28e1 bytes)
2016-03-13T22:14:29.971 INFO:teuthology.orchestra.run.magna051.stdout:14957 read    0x9b3e9f thru    0x9c379a    (0xf8fc bytes)
2016-03-13T22:14:29.973 INFO:teuthology.orchestra.run.magna051.stdout:14958 read    0xbed013d thru    0xbed1301    (0x11c5 bytes)
2016-03-13T22:14:29.974 INFO:teuthology.orchestra.run.magna051.stdout:14959 write    0x596023f thru    0x596b6d4    (0xb496 bytes)
2016-03-13T22:14:30.068 INFO:teuthology.orchestra.run.magna051.stdout:14960 read    0x193d448 thru    0x1943dea    (0x69a3 bytes)
2016-03-13T22:14:30.069 INFO:teuthology.orchestra.run.magna051.stdout:14962 trunc    from 0xe244207 to 0xe55940d
2016-03-13T22:14:30.106 INFO:teuthology.orchestra.run.magna051.stdout:14964 write    0xd9399 thru    0xe6571    (0xd1d9 bytes)
2016-03-13T22:14:30.139 INFO:teuthology.orchestra.run.magna051.stdout:14966 clone    139 order 25 su 16777216 sc 6
2016-03-13T22:14:32.971 INFO:teuthology.orchestra.run.magna051.stdout:truncating image image_client.0-clone138 from 0xe55940d (overlap 0x311c26) to 0x1d774a
Actions #1

Updated by Jason Dillaman about 8 years ago

  • Status changed from New to Need More Info

@Vasu Kulkarni, do you have any of the node logs? It appears the test was hard-killed.

Actions #2

Updated by Jason Dillaman about 8 years ago

@Vasu Kulkarni, I re-ran the test with the same seed value and was unable to reproduce a hang. Can you please re-test and if it hangs, ensure the logs are retained.

Actions #3

Updated by Vasu Kulkarni about 8 years ago

Jason,

quite a few tests failed in this run, I suspect bad jewel sha as well, As you mentioned this run just ended abruptly without any logs, I will be rerunning again with jewel and update here.

Thanks

Actions #4

Updated by Jason Dillaman almost 8 years ago

@Vasu Kulkarni: can we close this ticket?

Actions #5

Updated by Vasu Kulkarni almost 8 years ago

  • Status changed from Need More Info to Closed

Issue was seen only on that specific run, Closing.

Actions

Also available in: Atom PDF