Project

General

Profile

Actions

Bug #50905

open

[rbd-nbd] kernel lockup during rbd_fsx_nbd

Added by Ilya Dryomov almost 3 years ago. Updated over 2 years ago.

Status:
New
Priority:
High
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

http://qa-proxy.ceph.com/teuthology/dis-2021-05-19_19:22:53-rbd-wip-dis-testing-distro-basic-smithi/6123072/teuthology.log

2021-05-19T21:15:49.069 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0
2021-05-19T21:15:49.089 INFO:tasks.ceph.osd.3.smithi152.stderr:2021-05-19T21:15:49.091+0000 7fdea17a1700 -1 received  signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3  (PID: 16716) UID: 0
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:mapped writes DISABLED
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:mapped reads DISABLED
2021-05-19T21:15:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:Seed set to 2386
2021-05-19T21:15:49.103 INFO:teuthology.orchestra.run.smithi174.stdout:rbd-nbd mode enabled
2021-05-19T21:15:55.295 INFO:teuthology.orchestra.run.smithi174.stdout:truncating to largest ever: 0xbf1ee00
2021-05-19T21:15:55.317 INFO:teuthology.orchestra.run.smithi174.stdout:2 trunc  from 0x0 to 0xbf1ee00
2021-05-19T21:15:55.346 INFO:teuthology.orchestra.run.smithi174.stdout:5 write  0x9e63800 thru  0x9e69060       (0x5861 bytes)
2021-05-19T21:15:55.346 INFO:teuthology.orchestra.run.smithi174.stdout:7 write  0x5edd200 thru  0x5ee3a26       (0x6827 bytes)
2021-05-19T21:15:55.363 INFO:teuthology.orchestra.run.smithi174.stdout:9 trunc  from 0xbf1ee00 to 0x2860c00
...
2021-05-19T21:17:34.832 INFO:teuthology.orchestra.run.smithi174.stdout:1528 read        0x30dce00 thru  0x30ddc33       (0xe34 bytes)
2021-05-19T21:17:34.882 INFO:teuthology.orchestra.run.smithi174.stdout:1530 write       0x801a400 thru  0x801d43f       (0x3040 bytes)
2021-05-19T21:17:34.950 INFO:teuthology.orchestra.run.smithi174.stdout:1531 write       0xed15800 thru  0xed23be1       (0xe3e2 bytes)
2021-05-19T21:17:34.951 INFO:teuthology.orchestra.run.smithi174.stdout:1532 write       0x1714c00 thru  0x172159c       (0xc99d bytes)
2021-05-19T21:17:34.951 INFO:teuthology.orchestra.run.smithi174.stdout:1533 punch       from 0x4553c00 to 0x4555600, (0x1a00 bytes)
2021-05-19T21:17:34.960 INFO:teuthology.orchestra.run.smithi174.stdout:1535 trunc       from 0xed23c00 to 0xb741200
2021-05-19T21:17:35.018 INFO:teuthology.orchestra.run.smithi174.stdout:1537 punch       from 0x4299e00 to 0x429b600, (0x1800 bytes)
2021-05-19T21:17:35.018 INFO:teuthology.orchestra.run.smithi174.stdout:1538 clone       12 order 20 su 8192 sc 4
2021-05-19T21:17:36.094 INFO:teuthology.orchestra.run.smithi174.stdout:truncating image image_client.0-clone11 from 0xb741200 (overlap 0x54b0a00) to 0x32d0600
2021-05-19T21:17:37.258 INFO:teuthology.orchestra.run.smithi174.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11
2021-05-19T21:17:37.740 INFO:teuthology.orchestra.run.smithi174.stdout:1540 punch       from 0xa4c8400 to 0xa4ccc00, (0x4800 bytes)
2021-05-19T21:17:37.760 INFO:teuthology.orchestra.run.smithi174.stdout:1541 punch       from 0x3649800 to 0x3653800, (0xa000 bytes)
2021-05-19T21:17:37.809 INFO:teuthology.orchestra.run.smithi174.stdout:1543 trunc       from 0xb741200 to 0xbf06800
2021-05-19T21:17:37.922 INFO:teuthology.orchestra.run.smithi174.stdout:1544 write       0xe307c00 thru  0xe3138c3       (0xbcc4 bytes)
2021-05-19T21:17:37.923 INFO:teuthology.orchestra.run.smithi174.stdout:1546 read        0x4e22e00 thru  0x4e22fa2       (0x1a3 bytes)
2021-05-19T21:17:37.923 INFO:teuthology.orchestra.run.smithi174.stdout:1547 punch       from 0x61ed800 to 0x61ef000, (0x1800 bytes)
2021-05-19T21:17:37.928 INFO:teuthology.orchestra.run.smithi174.stdout:1548 write       0x5b23a00 thru  0x5b2fd6a       (0xc36b bytes)
2021-05-19T21:17:37.932 INFO:teuthology.orchestra.run.smithi174.stdout:1549 read        0xa815000 thru  0xa81eaf2       (0x9af3 bytes)
2021-05-19T21:17:37.933 INFO:teuthology.orchestra.run.smithi174.stdout:1551 punch       from 0xb34e00 to 0xb43800, (0xea00 bytes)
2021-05-19T21:17:37.949 INFO:teuthology.orchestra.run.smithi174.stdout:1553 clone       13 order 23 su 524288 sc 11
2021-05-19T21:17:39.342 INFO:teuthology.orchestra.run.smithi174.stdout:truncating image image_client.0-clone12 from 0xe313a00 (overlap 0xb741200) to 0x494d400
[  435.513803] block nbd0: Device being setup by another task
[  437.661777] block nbd0: Device being setup by another task
[  442.997777] block nbd0: Device being setup by another task
[  446.737495] block nbd0: Device being setup by another task
[  511.593046] block nbd0: Device being setup by another task
[  514.729046] block nbd0: Device being setup by another task
[  516.057542] blk_update_request: I/O error, dev nbd1, sector 2048 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[  516.068196] blk_update_request: I/O error, dev nbd1, sector 2048 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[  516.078494] Buffer I/O error on dev nbd1, logical block 2048, async page read
[  516.085677] blk_update_request: I/O error, dev nbd1, sector 2049 op 0x0:(READ) flags 0x0 phys_seg 7 prio class 0
[  516.095879] Buffer I/O error on dev nbd1, logical block 2049, async page read
[  516.103020] Buffer I/O error on dev nbd1, logical block 2050, async page read
[  516.110165] Buffer I/O error on dev nbd1, logical block 2051, async page read
[  516.117316] Buffer I/O error on dev nbd1, logical block 2052, async page read
[  516.124465] Buffer I/O error on dev nbd1, logical block 2053, async page read
[  516.131616] Buffer I/O error on dev nbd1, logical block 2054, async page read
[  516.138763] Buffer I/O error on dev nbd1, logical block 2055, async page read
[  519.049045] block nbd0: Device being setup by another task
[  523.729050] block nbd0: Device being setup by another task
[  526.057046] block nbd0: Device being setup by another task
[  527.997049] block nbd0: Device being setup by another task
[  531.345053] block nbd0: Device being setup by another task
[  533.924127] blk_update_request: I/O error, dev nbd0, sector 776 op 0x0:(READ) flags 0x80700 phys_seg 12 prio class 0

Nothing else -- smithi174 remained locked up and the job hit the timeout.


Related issues 1 (1 open0 closed)

Related to rbd - Bug #51529: [rbd-nbd] kernel BUG on "rbd-nbd unmap" during rbd-nbd.shNew

Actions
Actions

Also available in: Atom PDF