Project

General

Profile

Bug #50905

Updated by Ilya Dryomov 5 months ago

http://qa-proxy.ceph.com/teuthology/dis-2021-05-19_19:22:53-rbd-wip-dis-testing-distro-basic-smithi/6123072/teuthology.log

<pre>
2021-05-19T21:15:49.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0
2021-05-19T21:15:49.089 INFO:tasks.ceph.osd.3.smithi152.stderr:2021-05-19T21:15:49.091+0000 7fdea17a1700 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 16716) UID: 0
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:mapped writes DISABLED
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:mapped reads DISABLED
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:Seed set to 2386
2021-05-19T21:15:49.103 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd mode enabled
2021-05-19T21:15:55.295 INFO:teuthology.orchestra.run.smithi174.stdout:truncating to largest ever: 0xbf1ee00
2021-05-19T21:15:55.317 INFO:teuthology.orchestra.run.smithi174.stdout:2 trunc from 0x0 to 0xbf1ee00
2021-05-19T21:15:55.346 INFO:teuthology.orchestra.run.smithi174.stdout:5 write 0x9e63800 thru 0x9e69060 (0x5861 bytes)
2021-05-19T21:15:55.346 INFO:teuthology.orchestra.run.smithi174.stdout:7 write 0x5edd200 thru 0x5ee3a26 (0x6827 bytes)
2021-05-19T21:15:55.363 INFO:teuthology.orchestra.run.smithi174.stdout:9 trunc from 0xbf1ee00 to 0x2860c00
...
2021-05-19T21:17:34.832 INFO:teuthology.orchestra.run.smithi174.stdout:1528 read 0x30dce00 thru 0x30ddc33 (0xe34 bytes)
2021-05-19T21:17:34.882 INFO:teuthology.orchestra.run.smithi174.stdout:1530 write 0x801a400 thru 0x801d43f (0x3040 bytes)
2021-05-19T21:17:34.950 INFO:teuthology.orchestra.run.smithi174.stdout:1531 write 0xed15800 thru 0xed23be1 (0xe3e2 bytes)
2021-05-19T21:17:34.951 INFO:teuthology.orchestra.run.smithi174.stdout:1532 write 0x1714c00 thru 0x172159c (0xc99d bytes)
2021-05-19T21:17:34.951 INFO:teuthology.orchestra.run.smithi174.stdout:1533 punch from 0x4553c00 to 0x4555600, (0x1a00 bytes)
2021-05-19T21:17:34.960 INFO:teuthology.orchestra.run.smithi174.stdout:1535 trunc from 0xed23c00 to 0xb741200
2021-05-19T21:17:35.018 INFO:teuthology.orchestra.run.smithi174.stdout:1537 punch from 0x4299e00 to 0x429b600, (0x1800 bytes)
2021-05-19T21:17:35.018 INFO:teuthology.orchestra.run.smithi174.stdout:1538 clone 12 order 20 su 8192 sc 4
2021-05-19T21:17:36.094 INFO:teuthology.orchestra.run.smithi174.stdout:truncating image image_client.0-clone11 from 0xb741200 (overlap 0x54b0a00) to 0x32d0600
2021-05-19T21:17:37.258 INFO:teuthology.orchestra.run.smithi174.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11
2021-05-19T21:17:37.740 INFO:teuthology.orchestra.run.smithi174.stdout:1540 punch from 0xa4c8400 to 0xa4ccc00, (0x4800 bytes)
2021-05-19T21:17:37.760 INFO:teuthology.orchestra.run.smithi174.stdout:1541 punch from 0x3649800 to 0x3653800, (0xa000 bytes)
2021-05-19T21:17:37.809 INFO:teuthology.orchestra.run.smithi174.stdout:1543 trunc from 0xb741200 to 0xbf06800
2021-05-19T21:17:37.922 INFO:teuthology.orchestra.run.smithi174.stdout:1544 write 0xe307c00 thru 0xe3138c3 (0xbcc4 bytes)
2021-05-19T21:17:37.923 INFO:teuthology.orchestra.run.smithi174.stdout:1546 read 0x4e22e00 thru 0x4e22fa2 (0x1a3 bytes)
2021-05-19T21:17:37.923 INFO:teuthology.orchestra.run.smithi174.stdout:1547 punch from 0x61ed800 to 0x61ef000, (0x1800 bytes)
2021-05-19T21:17:37.928 INFO:teuthology.orchestra.run.smithi174.stdout:1548 write 0x5b23a00 thru 0x5b2fd6a (0xc36b bytes)
2021-05-19T21:17:37.932 INFO:teuthology.orchestra.run.smithi174.stdout:1549 read 0xa815000 thru 0xa81eaf2 (0x9af3 bytes)
2021-05-19T21:17:37.933 INFO:teuthology.orchestra.run.smithi174.stdout:1551 punch from 0xb34e00 to 0xb43800, (0xea00 bytes)
2021-05-19T21:17:37.949 INFO:teuthology.orchestra.run.smithi174.stdout:1553 clone 13 order 23 su 524288 sc 11
2021-05-19T21:17:39.342 INFO:teuthology.orchestra.run.smithi174.stdout:truncating image image_client.0-clone12 from 0xe313a00 (overlap 0xb741200) to 0x494d400
</pre>

<pre>
[ 435.513803] block nbd0: Device being setup by another task
[ 437.661777] block nbd0: Device being setup by another task
[ 442.997777] block nbd0: Device being setup by another task
[ 446.737495] block nbd0: Device being setup by another task
[ 511.593046] block nbd0: Device being setup by another task
[ 514.729046] block nbd0: Device being setup by another task
[ 516.057542] blk_update_request: I/O error, dev nbd1, sector 2048 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[ 516.068196] blk_update_request: I/O error, dev nbd1, sector 2048 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 516.078494] Buffer I/O error on dev nbd1, logical block 2048, async page read
[ 516.085677] blk_update_request: I/O error, dev nbd1, sector 2049 op 0x0:(READ) flags 0x0 phys_seg 7 prio class 0
[ 516.095879] Buffer I/O error on dev nbd1, logical block 2049, async page read
[ 516.103020] Buffer I/O error on dev nbd1, logical block 2050, async page read
[ 516.110165] Buffer I/O error on dev nbd1, logical block 2051, async page read
[ 516.117316] Buffer I/O error on dev nbd1, logical block 2052, async page read
[ 516.124465] Buffer I/O error on dev nbd1, logical block 2053, async page read
[ 516.131616] Buffer I/O error on dev nbd1, logical block 2054, async page read
[ 516.138763] Buffer I/O error on dev nbd1, logical block 2055, async page read
[ 519.049045] block nbd0: Device being setup by another task
[ 523.729050] block nbd0: Device being setup by another task
[ 526.057046] block nbd0: Device being setup by another task
[ 527.997049] block nbd0: Device being setup by another task
[ 531.345053] block nbd0: Device being setup by another task
[ 533.924127] blk_update_request: I/O error, dev nbd0, sector 776 op 0x0:(READ) flags 0x80700 phys_seg 12 prio class 0
</pre>

Nothing else -- smithi174 remained locked up and the job hit the timeout.

Back