Bug #54759: crash: bool HealthMonitor::check_member_health(): assert(store_size > 0) - RADOS - Ceph

Actions

Copy link

Bug #54759

open

crash: bool HealthMonitor::check_member_health(): assert(store_size > 0)

Added by Telemetry Bot about 2 years ago. Updated 12 months ago.

Status:

New

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Telemetry

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

Ceph - v16.2.10, Ceph - v16.2.11, Ceph - v16.2.7, Ceph - v16.2.8, Ceph - v16.2.9, Ceph - v17.2.1, Ceph - v17.2.3, Ceph - v17.2.5

ceph-qa-suite:

Component(RADOS):

Pull request ID:

Crash signature (v1):

402e5da632bf531ae9955c4892b71ed8b2b72ba8071c87a2b0465a857c4e30a8
8f09af414f7e147b0dee761128a407590e6fdfb944e2da89241a0310029ac2f3
0d38a63ce78643145038aab756eeda4606c0851f18d329080cf1f3d1406c3732
10542b0a4f466a753aa1b2165cebadb16d35be8f24dde235e3bac6342a2297d4
6ad3a67676656f5cb61749df1c4974f864d0439aece7baf2903ca912849bc0c0
71a54071bdfa6699e389642e457f7ea0aeb27510fe008dbd5c40b0afee2328b4
78fa7f54cf05803f3979dc145547344bcd8c6b8214a64614d095b5d8ae55c74b
aad9401258a4ae6db902e5af9133898ba4130a263b91e9c47d017dba64bfa9bc
ba4977595bedc0064e0166a537e1882c33b70dc83922b91544a22c3b5764541a
cfab8d8cd5b62bb05d9a917b35d4e0d024a0466a9da55482db5b184b72b94e58
d34481950924c442091c008f62e6b8948e80d0daad6f55769b458b62a23b23f1
ea30814f90b5312a483ed1fce4f2218dfb12ad3974364e3dfbaa7e41823b9c42
f1b9752f3de492a9e4f93de71745aa3aedf4005b8167ae83acaca9a8ae465a60
f9e6257b3aa38235dd39a1f52621a03cbde10a39bae6d9ae74b7f092ced621e2

Crash signature (v2):

8daed7dbe2a4db7e174d5c13e6ec018a763989de4f3ba179d9a79d1b1fa3b86f

Description

http://telemetry.front.sepia.ceph.com:4000/d/jByk5HaMz/crash-spec-x-ray?orgId=1&var-sig_v2=8daed7dbe2a4db7e174d5c13e6ec018a763989de4f3ba179d9a79d1b1fa3b86f

Assert condition: store_size > 0
Assert function: bool HealthMonitor::check_member_health()

Sanitized backtrace:

    HealthMonitor::check_member_health()
    HealthMonitor::tick()
    Monitor::tick()
    Context::complete(int)
    CommonSafeTimer<std::mutex>::timer_thread()
    CommonSafeTimerThread<std::mutex>::entry()

Crash dump sample:

{
    "assert_condition": "store_size > 0",
    "assert_file": "mon/HealthMonitor.cc",
    "assert_func": "bool HealthMonitor::check_member_health()",
    "assert_line": 581,
    "assert_msg": "mon/HealthMonitor.cc: In function 'bool HealthMonitor::check_member_health()' thread 7f33111bd700 time 2022-02-24T15:58:39.495804+0100\nmon/HealthMonitor.cc: 581: FAILED ceph_assert(store_size > 0)",
    "assert_thread_name": "safe_timer",
    "backtrace": [
        "/lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0) [0x7f331768e3c0]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1ad) [0x7f3317ba7db0]",
        "/usr/lib/ceph/libceph-common.so.2(+0x265f5d) [0x7f3317ba7f5d]",
        "(HealthMonitor::check_member_health()+0x1eb8) [0x559c1d7318f8]",
        "(HealthMonitor::tick()+0x58) [0x559c1d732c08]",
        "(Monitor::tick()+0xd0) [0x559c1d650310]",
        "(Context::complete(int)+0xd) [0x559c1d67ca0d]",
        "(CommonSafeTimer<std::mutex>::timer_thread()+0x108) [0x7f3317c9f838]",
        "(CommonSafeTimerThread<std::mutex>::entry()+0x11) [0x7f3317ca09b1]",
        "/lib/x86_64-linux-gnu/libpthread.so.0(+0x9609) [0x7f3317682609]",
        "clone()" 
    ],
    "ceph_version": "16.2.7",
    "crash_id": "2022-02-24T14:58:39.512458Z_00d6231a-0acf-4d2c-9310-3d52d364bc3d",
    "entity_name": "mon.685cf17395935264701519ba92d1d49c0223f37f",
    "os_id": "ubuntu",
    "os_name": "Ubuntu",
    "os_version": "20.04.3 LTS (Focal Fossa)",
    "os_version_id": "20.04",
    "process_name": "ceph-mon",
    "stack_sig": "402e5da632bf531ae9955c4892b71ed8b2b72ba8071c87a2b0465a857c4e30a8",
    "timestamp": "2022-02-24T14:58:39.512458Z",
    "utsname_machine": "x86_64",
    "utsname_release": "5.4.0-97-generic",
    "utsname_sysname": "Linux",
    "utsname_version": "#110-Ubuntu SMP Thu Jan 13 18:22:13 UTC 2022" 
}