Project

General

Profile

Actions
Copy link

Bug #53359

open

bluestore: missing block.db symlinks leads to confusing crash

Added by Sage Weil over 2 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
Adam Kupczyk
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

A regression in ceph-volume (master branch) led to the block.db symlink not getting created. This leads to OSDs that crash like so:

    "backtrace": [
        "/lib64/libpthread.so.0(+0x12c20) [0x7f3573347c20]",
        "gsignal()",
        "abort()",
        "/lib64/libstdc++.so.6(+0x9009b) [0x7f357295e09b]",
        "/lib64/libstdc++.so.6(+0x9653c) [0x7f357296453c]",
        "/lib64/libstdc++.so.6(+0x96597) [0x7f3572964597]",
        "/lib64/libstdc++.so.6(+0x967f8) [0x7f35729647f8]",
        "/usr/bin/ceph-osd(+0x5c7203) [0x55cc53713203]",
        "(BlueFS::_open_super()+0x18f) [0x55cc53e66cff]",
        "(BlueFS::mount()+0xeb) [0x55cc53e88ddb]",
        "(BlueStore::_open_bluefs(bool, bool)+0x94) [0x55cc53d4bad4]",
        "(BlueStore::_prepare_db_environment(bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)+0x6d9) [0x55cc53d4cc29]",
        "(BlueStore::_open_db(bool, bool, bool)+0x15c) [0x55cc53d4df4c]",
        "(BlueStore::_open_db_and_around(bool, bool)+0x2b4) [0x55cc53dc68d4]",
        "(BlueStore::_mount()+0x1ae) [0x55cc53dc971e]",
        "(OSD::init()+0x3ba) [0x55cc5385711a]",
        "main()",
        "__libc_start_main()",
        "_start()" 
    ],
    "ceph_version": "17.0.0-9073-g6e528ed7",

The on-disk block that we are trying to decode is all zeros.

I thought we had a flag somewhere indicating whether a db and/or wal was expected so that we could provide a meaningful/informative error message, but maybe not?

(ceph-volume fix is here: https://github.com/ceph/ceph/pull/44030)

Actions
Copy link

Also available in: Atom PDF