Project

General

Profile

Bug #42367

mira117 I/O errors

Added by Kefu Chai about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Category:
Test Node
Target version:
-
Start date:
10/18/2019
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature:

History

#1 Updated by Kefu Chai about 1 month ago

  • Subject changed from mira117 to mira117 I/O errors
  • Category set to Test Node
  • Assignee set to David Galloway
  • Source set to Development
2019-10-17T15:45:15.922711+00:00 mira117 kernel: device-mapper: ioctl: remove_all left 1 open device(s)
2019-10-17T15:45:16.617412+00:00 mira117 kernel: blk_update_request: I/O error, dev dm-0, sector 1953525152
2019-10-17T15:45:16.617462+00:00 mira117 kernel: blk_update_request: I/O error, dev dm-0, sector 1953525152
2019-10-17T15:45:16.617494+00:00 mira117 kernel: Buffer I/O error on dev dm-0, logical block 244190644, async page read
...
2019-10-17T15:50:22.052922+00:00 mira117 kernel: sd 0:0:0:6: [sdf] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2019-10-17T15:50:22.053092+00:00 mira117 kernel: sd 0:0:0:6: [sdf] Sense Key : Illegal Request [current]
2019-10-17T15:50:22.053259+00:00 mira117 kernel: sd 0:0:0:6: [sdf] Add. Sense: Invalid command operation code
2019-10-17T15:50:22.068043+00:00 mira117 kernel: sd 0:0:0:6: [sdf] CDB: Write same(16) 93 08 00 00 00 00 00 00 00 00 00 7f ff ff 00 00
2019-10-17T15:50:22.068100+00:00 mira117 kernel: blk_update_request: 1 callbacks suppressed
2019-10-17T15:50:22.087657+00:00 mira117 kernel: blk_update_request: critical target error, dev sdf, sector 0
...
2019-10-17T15:50:27.311458+00:00 mira117 kernel: blk_update_request: critical target error, dev sdg, sector 0
2019-10-17T15:50:27.311591+00:00 mira117 kernel: sd 0:0:0:7: [sdg] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2019-10-17T15:50:27.311719+00:00 mira117 kernel: sd 0:0:0:7: [sdg] Sense Key : Illegal Request [current]
2019-10-17T15:50:27.311845+00:00 mira117 kernel: sd 0:0:0:7: [sdg] Add. Sense: Invalid command operation code
2019-10-17T15:50:27.311975+00:00 mira117 kernel: sd 0:0:0:7: [sdg] CDB: Write same(16) 93 08 00 00 00 00 00 7f ff ff 00 7f ff ff 00 00
...
2019-10-17T15:50:33.189688+00:00 mira117 kernel: blk_update_request: critical target error, dev sdd, sector 16777214
2019-10-17T15:50:33.197406+00:00 mira117 kernel: sd 0:0:0:4: [sdd] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
2019-10-17T15:50:33.205519+00:00 mira117 kernel: sd 0:0:0:4: [sdd] Sense Key : Illegal Request [current]
2019-10-17T15:50:33.212140+00:00 mira117 kernel: sd 0:0:0:4: [sdd] Add. Sense: Invalid command operation code
2019-10-17T15:50:33.219143+00:00 mira117 kernel: sd 0:0:0:4: [sdd] CDB: Write same(16) 93 08 00 00 00 00 01 7f ff fd 00 7f ff ff 00 00
...
[kchai@mira117 ~]$ sudo /usr/libexec/smart.sh
CRITICAL - 2 of 7 drives need replacing
Drive 1 (sda) has 13 reallocated sectors
Drive 3 failed

#2 Updated by David Galloway about 1 month ago

Yeah, so, I think this is probably related to the issue I see only on the mira where some of the disks were, at some point, configured as multipath devices.

No matter what I do (multipath -f, dd if=/dev/zero of=/dev/sda, etc.) I can't destroy those multipath links. So sometimes when I go to install these machines using Cobbler, the kickstart fails because the OS only sees "mpatha" and not "sda". I basically just have to find a different mira or throw away the disk.

https://tracker.ceph.com/issues/19967

I need to have another drive party soon so I'll replace this one at that time.

Also available in: Atom PDF