Bug #52184: crash: void KernelDevice::_aio_thread(): abort - bluestore - Ceph

Actions

Copy link

Bug #52184

closed

crash: void KernelDevice::_aio_thread(): abort

Added by Telemetry Bot over 2 years ago. Updated about 2 years ago.

Status:

Won't Fix

Priority:

Normal

Assignee:

Target version:

% Done:

Source:

Telemetry

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v16.2.0, Ceph - v16.2.1, Ceph - v16.2.3, Ceph - v16.2.4, Ceph - v16.2.5

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

4939975fa6e0c6307f99824a995118cdb06b9b8f88ffe0d4f0df0d13901562bd
68e8888499afb583f4ba32eed0cd7357fddb7829af2ce36f8fa3215aa45a814e
6ab9b1b5d48120872492abbffcb1991eb019284b001c8cc891383ff792c81004
7c2c3f345d367ab9252bf679471d3aa61a8a83be13c1b5ccedfdac461b698e7b
a97ce634eb3dd654614734fcae4afc09d1539d7fc8afc31636c3108fdf210483

Crash signature (v2):

03aca490bc247c406663edd41e637b3d48070c6958f331bd1b74c8267637a539

Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=7c2c3f345d367ab9252bf679471d3aa61a8a83be13c1b5ccedfdac461b698e7b

Assert condition: abort
Assert function: void KernelDevice::_aio_thread()

Sanitized backtrace:

    /lib/x86_64-linux-gnu/libpthread.so.0(
    KernelDevice::_aio_thread()
    KernelDevice::AioCompletionThread::entry()
    /lib/x86_64-linux-gnu/libpthread.so.0(
    clone()

Crash dump sample:

{
    "assert_condition": "abort",
    "assert_file": "blk/kernel/KernelDevice.cc",
    "assert_func": "void KernelDevice::_aio_thread()",
    "assert_line": 597,
    "assert_msg": "blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fda26d08700 time 2021-08-10T12:30:33.208858-0700\nblk/kernel/KernelDevice.cc: 597: ceph_abort_msg(\"Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!\")",
    "assert_thread_name": "bstore_aio",
    "backtrace": [
        "/lib/x86_64-linux-gnu/libpthread.so.0(+0x14140) [0x7fda2f65a140]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x18a) [0x560e71f2acd2]",
        "(KernelDevice::_aio_thread()+0x10d5) [0x560e72a81415]",
        "(KernelDevice::AioCompletionThread::entry()+0xd) [0x560e72a8552d]",
        "/lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7) [0x7fda2f64eea7]",
        "clone()" 
    ],
    "ceph_version": "16.2.5",
    "crash_id": "2021-08-10T19:30:33.213650Z_09ab83e7-debd-465e-a07a-a10ffec55cf0",
    "entity_name": "osd.70cede6aa190e2a71c7345b77cbb24b46f02950c",
    "io_error": true,
    "io_error_code": -121,
    "io_error_devname": "dm-18",
    "io_error_length": 65536,
    "io_error_offset": 659095552,
    "io_error_optype": 8,
    "io_error_path": "/var/lib/ceph/osd/ceph-22/block",
    "os_id": "11",
    "os_name": "Debian GNU/Linux 11 (bullseye)",
    "os_version": "11 (bullseye)",
    "os_version_id": "11",
    "process_name": "ceph-osd",
    "stack_sig": "4939975fa6e0c6307f99824a995118cdb06b9b8f88ffe0d4f0df0d13901562bd",
    "timestamp": "2021-08-10T19:30:33.213650Z",
    "utsname_machine": "x86_64",
    "utsname_release": "5.11.22-3-pve",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP PVE 5.11.22-6 (Wed, 28 Jul 2021 10:51:12 +0200)" 
}