Bug #52182: crash: void KernelDevice::_aio_thread(): abort - bluestore - Ceph

Actions

Copy link

Bug #52182

closed

crash: void KernelDevice::_aio_thread(): abort

Added by Telemetry Bot over 2 years ago. Updated about 2 years ago.

Status:

Won't Fix

Priority:

Normal

Assignee:

Target version:

% Done:

Source:

Telemetry

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v16.2.0, Ceph - v16.2.1, Ceph - v16.2.3, Ceph - v16.2.4, Ceph - v16.2.5

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

63830fc5299741097a4a5f8896f1f688dab4bb3e48cacf7e8004a46c2fbe91f2
6d3e8978933aa44023e04069bc5d81077af4f725be37bfd8c6b78f00c1f9d1e8
7086d21b14aa049489ea916e996f3a289a85ca3f793b1f45c77d4b30da6b53a2
8188bd963a77114f4b891cdb9aa6b914a62d8291dae3771172de9393447466cf
9c828ff03cc36c4694a00c715ae7026e8ffeb4fc8c55b93048d90b5719a3431c
a41eb237ceef63ce95cda548310908e728b64a685c4a06bc30905b9552cf5bef
b4bde5ad240c50c2afe0c1fbc2bcc4aa2295f0f47e996a9a78b0fed3bb2b4257
dfbb7b3529653802a503790ee3d41af7a5003a6b72783da352325064394f0150

Crash signature (v2):

03aca490bc247c406663edd41e637b3d48070c6958f331bd1b74c8267637a539

Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=8188bd963a77114f4b891cdb9aa6b914a62d8291dae3771172de9393447466cf

Assert condition: abort
Assert function: void KernelDevice::_aio_thread()

Sanitized backtrace:

    /lib64/libpthread.so.0(
    KernelDevice::_aio_thread()
    KernelDevice::AioCompletionThread::entry()
    /lib64/libpthread.so.0(
    clone()

Crash dump sample:

{
    "archived": "2021-08-08 00:01:00.526683",
    "assert_condition": "abort",
    "assert_file": "blk/kernel/KernelDevice.cc",
    "assert_func": "void KernelDevice::_aio_thread()",
    "assert_line": 600,
    "assert_msg": "blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f1d64c1e700 time 2021-08-07T21:41:42.833910+0000\nblk/kernel/KernelDevice.cc: 600: ceph_abort_msg(\"Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!\")",
    "assert_thread_name": "bstore_aio",
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12b20) [0x7f1d70c1db20]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b6) [0x5616529bf509]",
        "(KernelDevice::_aio_thread()+0x1254) [0x5616534fd954]",
        "(KernelDevice::AioCompletionThread::entry()+0x11) [0x561653508c51]",
        "/lib64/libpthread.so.0(+0x814a) [0x7f1d70c1314a]",
        "clone()" 
    ],
    "ceph_version": "16.2.0",
    "crash_id": "2021-08-07T21:41:42.898675Z_9617b33b-52a2-4dd3-a962-d33589f7e92d",
    "entity_name": "osd.791ba478b89cb18908713faa2cbb995a3f9f511c",
    "io_error": true,
    "io_error_code": -5,
    "io_error_devname": "dm-37",
    "io_error_length": 4096,
    "io_error_offset": 65536,
    "io_error_optype": 8,
    "io_error_path": "/var/lib/ceph/osd/ceph-56/block",
    "os_id": "centos",
    "os_name": "CentOS Linux",
    "os_version": "8",
    "os_version_id": "8",
    "process_name": "ceph-osd",
    "stack_sig": "63830fc5299741097a4a5f8896f1f688dab4bb3e48cacf7e8004a46c2fbe91f2",
    "timestamp": "2021-08-07T21:41:42.898675Z",
    "utsname_machine": "x86_64",
    "utsname_release": "5.4.0-70-generic",
    "utsname_sysname": "Linux",
    "utsname_version": "#78-Ubuntu SMP Fri Mar 19 13:29:52 UTC 2021" 
}