Project

General

Profile

Actions
Copy link

Bug #24670

open

mira050: 2 of 7 drives need replacing

Added by Kefu Chai almost 6 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
David Galloway
Category:
Test Node
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

root@mira050:/home/kchai# /usr/libexec/smart.sh
CRITICAL - 2 of 7 drives need replacing
Drive 4 (sdd) has 2 reallocated sectors
Drive 7 failed

dmesg

[  322.034727] sd 0:0:0:5: [sdf] tag#3 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[  322.034736] sd 0:0:0:5: [sdf] tag#3 Sense Key : Illegal Request [current]
[  322.034744] sd 0:0:0:5: [sdf] tag#3 Add. Sense: Invalid command operation code
[  322.034749] sd 0:0:0:5: [sdf] tag#3 CDB: Write same(16) 93 08 00 00 00 00 01 7f ff fd 00 7f ff ff 00 00
[  322.034752] blk_update_request: critical target error, dev sdf, sector 25165821

in osd log

2018-06-26T11:31:52.969 INFO:tasks.ceph.osd.2.mira050.stderr:2018-06-26 11:31:52.964 7f0156ba9700 -1 log_channel(cluster) log [ERR] : Error -5 reading object 2:c077b387:::mira05012520-911 oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo:head

Actions
Copy link
 #1

Updated by Kefu Chai over 5 years ago 

2018-07-31 00:18:04.305 7fc229303700 -1 filestore(/var/lib/ceph/osd/ceph-2) _do_copy_range(3941): read error at 1908736~2465792, (5) Input/output error
2018-07-31 00:18:04.309 7fc229303700 -1 /build/ceph-14.0.0-1729-ge9d8990/src/os/filestore/FileStore.cc: In function 'int FileStore::_do_copy_range(int, int, uint64_t, uint64_t, uin
t64_t, bool)' thread 7fc229303700 time 2018-07-31 00:18:04.307478
/build/ceph-14.0.0-1729-ge9d8990/src/os/filestore/FileStore.cc: 3978: FAILED assert(replaying || pos == end)

 ceph version 14.0.0-1729-ge9d8990 (e9d899031e672701d4d99ae7281cee06d7a5f0f3) nautilus (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x55ea7125e499]
 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x55ea7125e61b]
 3: (FileStore::_do_copy_range(int, int, unsigned long, unsigned long, unsigned long, bool)+0x146a) [0x55ea716adf0a]
 4: (GenericFileStoreBackend::clone_range(int, int, unsigned long, unsigned long, unsigned long)+0x6b) [0x55ea71702acb]
 5: (FileStore::_clone(coll_t const&, ghobject_t const&, ghobject_t const&, SequencerPosition const&)+0x22c) [0x55ea716d3ccc]
 6: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int, ThreadPool::TPHandle*, char const*)+0x301e) [0x55ea716de68e]
 7: (FileStore::_do_transactions(std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, unsigned long, ThreadPool::TPHandle*, char const*)+0x48) [0x55ea
716e28f8]
 8: (FileStore::_do_op(FileStore::OpSequencer*, ThreadPool::TPHandle&)+0x185) [0x55ea716e2ab5]
 9: (ThreadPool::worker(ThreadPool::WorkThread*)+0x8eb) [0x55ea7192429b]
 10: (ThreadPool::WorkThread::entry()+0x10) [0x55ea71924cd0]
 11: (()+0x76db) [0x7fc23ae156db]
 12: (clone()+0x3f) [0x7fc23978f88f]


[ 9142.899442] print_req_error: I/O error, dev sdg, sector 373912
[ 9173.440347] sd 0:0:0:6: [sdg] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9173.440365] sd 0:0:0:6: [sdg] tag#0 Sense Key : Medium Error [current]
[ 9173.440371] sd 0:0:0:6: [sdg] tag#0 Add. Sense: Unrecovered read error
[ 9173.440378] sd 0:0:0:6: [sdg] tag#0 CDB: Read(10) 28 00 00 05 78 98 00 03 08 00
[ 9173.440383] print_req_error: I/O error, dev sdg, sector 358552
[ 9455.251391] sd 0:0:0:6: [sdg] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9455.251413] sd 0:0:0:6: [sdg] tag#1 Sense Key : Medium Error [current]
[ 9455.251419] sd 0:0:0:6: [sdg] tag#1 Add. Sense: Unrecovered read error
[ 9455.251426] sd 0:0:0:6: [sdg] tag#1 CDB: Read(10) 28 00 00 05 b4 98 00 02 30 00
[ 9455.251431] print_req_error: I/O error, dev sdg, sector 373912
[ 9491.745067] sd 0:0:0:6: [sdg] tag#1 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 9491.745072] sd 0:0:0:6: [sdg] tag#1 Sense Key : Medium Error [current]
[ 9491.745074] sd 0:0:0:6: [sdg] tag#1 Add. Sense: Unrecovered read error
[ 9491.745077] sd 0:0:0:6: [sdg] tag#1 CDB: Read(10) 28 00 00 05 78 98 00 03 08 00
[ 9491.745079] print_req_error: I/O error, dev sdg, sector 358552

/usr/libexec/smart.sh
CRITICAL - 2 of 8 drives need replacing
Drive 4 (sdd) has 2 reallocated sectors
Drive 7 (sdg) has 4015 reallocated sectors
Actions
Copy link

Also available in: Atom PDF