Project

General

Profile

Actions
Copy link

Bug #56102

closed

Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in - RocksDBStore::estimate_prefix_size

Added by Vikhyat Umrao almost 2 years ago. Updated almost 2 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

ceph crash info 2022-06-17T18:57:45.194841Z_92068a7d-571d-4bb8-996b-1ec00d7332b1
{
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12ce0) [0x7f63d4b7cce0]",
        "(RocksDBStore::estimate_prefix_size(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x3b6) [0x55c0dd990b46]",
        "(BlueStore::pool_statfs(unsigned long, store_statfs_t*, bool*)+0x1e8) [0x55c0dd371648]",
        "(OSD::collect_pg_stats()+0x741) [0x55c0dce48ac1]",
        "(MgrClient::_send_pgstats()+0x40) [0x55c0dd782b10]",
        "(MgrClient::_send_stats()+0x15) [0x55c0dd785945]",
        "(Context::complete(int)+0xd) [0x55c0dce74d1d]",
        "(CommonSafeTimer<std::mutex>::timer_thread()+0x11a) [0x55c0dd54dafa]",
        "(CommonSafeTimerThread<std::mutex>::entry()+0x11) [0x55c0dd54f121]",
        "/lib64/libpthread.so.0(+0x81ca) [0x7f63d4b721ca]",
        "clone()" 
    ],
    "ceph_version": "17.2.0",
    "crash_id": "2022-06-17T18:57:45.194841Z_92068a7d-571d-4bb8-996b-1ec00d7332b1",
    "entity_name": "osd.31",
    "os_id": "centos",
    "os_name": "CentOS Stream",
    "os_version": "8",
    "os_version_id": "8",
    "process_name": "ceph-osd",
    "stack_sig": "99b56157ea4b210db422174f528bf297535ac749ee952a700e03651444bec3e7",
    "timestamp": "2022-06-17T18:57:45.194841Z",
    "utsname_hostname": "gibba019",
    "utsname_machine": "x86_64",
    "utsname_release": "4.18.0-301.1.el8.x86_64",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP Tue Apr 13 16:24:22 UTC 2021" 
}

[root@gibba001 f9d4cf6a-edcf-11ec-a96a-3cecef3d8fb8]# ceph tell osd.31 version
{
    "version": "17.2.0-436-gda36d2c9",
    "release": "quincy",
    "release_type": "stable" 
}

Files

Download all files
ceph-osd.31.log.gz (72.4 KB) ceph-osd.31.log.gz Laura Flores, 06/21/2022 04:34 PM
ceph-osd.31.log-20220618.gz (380 KB) ceph-osd.31.log-20220618.gz Laura Flores, 06/21/2022 04:34 PM
ceph-osd.31.log-20220619.gz (131 KB) ceph-osd.31.log-20220619.gz Laura Flores, 06/21/2022 04:34 PM
ceph-osd.31.log-20220620.gz (135 KB) ceph-osd.31.log-20220620.gz Laura Flores, 06/21/2022 04:34 PM
ceph-osd.31.log-20220621.gz (138 KB) ceph-osd.31.log-20220621.gz Laura Flores, 06/21/2022 04:34 PM

Related issues 2 (0 open — 2 closed)

Related to bluestore - Bug #59811: crash: RocksDBStore::estimate_prefix_size(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)Duplicate

Actions
Is duplicate of RADOS - Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timerResolvedLaura Flores

Actions
Actions
Copy link
 #1

Updated by Laura Flores almost 2 years ago

  • File ceph-osd.31.log.gz added
  • File ceph-osd.31.log-20220618.gz added
  • File ceph-osd.31.log-20220619.gz added
  • File ceph-osd.31.log-20220620.gz added
  • File ceph-osd.31.log-20220621.gz added

@Adam DC949 I have uploaded all of the relevant logs from gibba019, osd.31 here. I found the crash in ceph-osd.31.log-20220618.gz, so that one will likely be most helpful, but you may be able to find some important context in the other logs as well.

From my understanding, the logs on the gibba cluster refresh after a certain period of time (old logs are lost), so it's important to save a copy of the logs.

Actions
Copy link
 #2

Updated by Laura Flores almost 2 years ago

  • File deleted (ceph-osd.31.log.gz)
Actions
Copy link
 #3

Updated by Laura Flores almost 2 years ago

  • File deleted (ceph-osd.31.log-20220618.gz)
Actions
Copy link
 #4

Updated by Laura Flores almost 2 years ago

  • File deleted (ceph-osd.31.log-20220619.gz)
Actions
Copy link
 #5

Updated by Laura Flores almost 2 years ago

  • File deleted (ceph-osd.31.log-20220620.gz)
Actions
Copy link
 #6

Updated by Laura Flores almost 2 years ago

  • File deleted (ceph-osd.31.log-20220621.gz)
Actions
Copy link
 #8

Updated by Vikhyat Umrao almost 2 years ago

  • Related to Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer added
Actions
Copy link
 #9

Updated by Vikhyat Umrao almost 2 years ago

  • Project changed from bluestore to RADOS
Actions
Copy link
 #10

Updated by Neha Ojha almost 2 years ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #11

Updated by Vikhyat Umrao almost 2 years ago

  • Related to deleted (Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer)
Actions
Copy link
 #12

Updated by Vikhyat Umrao almost 2 years ago

  • Has duplicate Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer added
Actions
Copy link
 #13

Updated by Vikhyat Umrao almost 2 years ago

  • Status changed from New to Duplicate
Actions
Copy link
 #14

Updated by Yaarit Hatuka over 1 year ago

  • Is duplicate of Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer added
Actions
Copy link
 #15

Updated by Yaarit Hatuka over 1 year ago

  • Has duplicate deleted (Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer)
Actions
Copy link
 #16

Updated by Telemetry Bot 12 months ago

  • Related to Bug #59811: crash: RocksDBStore::estimate_prefix_size(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) added
Actions
Copy link

Also available in: Atom PDF