Project

General

Profile

Bug #55145

Bogus assert in SimpleBitmap

Added by Gabriel BenHanokh 6 months ago. Updated 6 months ago.

Status:
Resolved
Priority:
Immediate
Target version:
-
% Done:

100%

Source:
Tags:
Backport:
quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The boundary check in SimpleBitmap is off by one causing an assert to trigger

2022-03-31T02:30:03.282076Z_1752ad21-6733-4266-8389-b70fb8d94408
{
    "assert_condition": "offset + length < m_num_bits",
    "assert_file": 
"/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-138-g723fda64/rpm/el8/BUILD/ceph-17.1.0-138-g723fda64/src/os/bluestore/simple_bitmap.cc",
    "assert_func": "bool SimpleBitmap::set(uint64_t, uint64_t)",
    "assert_line": 54,
    "assert_msg": 
"/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-138-g723fda64/rpm/el8/BUILD/ceph-17.1.0-138-g723fda64/src/os/bluestore/simple_bitmap.cc:
 In function 'bool SimpleBitmap::set(uint64_t, uint64_t)' thread 
7f077a8913c0 time 
2022-03-31T02:30:03.274438+0000\n/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-138-g723fda64/rpm/el8/BUILD/ceph-17.1.0-138-g723fda64/src/os/bluestore/simple_bitmap.cc:
 54: FAILED ceph_assert(offset + length < m_num_bits)\n",
    "assert_thread_name": "ceph-osd",
    "backtrace": [
        "/lib64/libpthread.so.0(+0x12ce0) [0x7f0778a96ce0]",
        "gsignal()",
        "abort()",
        "(ceph::__ceph_assert_fail(char const*, char const*, int, char 
const*)+0x1b0) [0x56200752f4c2]",
        "/usr/bin/ceph-osd(+0x5d7685) [0x56200752f685]",
        "(SimpleBitmap::set(unsigned long, unsigned long)+0x1221) 
[0x562007c26b41]",
        "(BlueStore::read_allocation_from_single_onode(SimpleBitmap*, 
boost::intrusive_ptr<BlueStore::Onode>&, 
BlueStore::read_alloc_stats_t&)+0x294) [0x562007b4d334]",
        "(BlueStore::read_allocation_from_onodes(SimpleBitmap*, 
BlueStore::read_alloc_stats_t&)+0x8f7) [0x562007b9adb7]",
        "(BlueStore::reconstruct_allocations(SimpleBitmap*, 
BlueStore::read_alloc_stats_t&)+0x5d) [0x562007b9be8d]",
        "(BlueStore::read_allocation_from_drive_on_startup()+0x99) 
[0x562007baeec9]",
        "(BlueStore::_init_alloc(std::map<unsigned long, unsigned 
long, std::less<unsigned long>, 
std::allocator<std::pair<unsigned long const, unsigned long> 
> >*)+0xaeb) [0x562007bafc2b]",
        "(BlueStore::_open_db_and_around(bool, bool)+0x321) 
[0x562007be82a1]",
        "(BlueStore::_mount()+0x1ae) [0x562007beb41e]",
        "(OSD::init()+0x403) [0x56200766d523]",
        "main()",
        "__libc_start_main()",
        "_start()" 
    ],

Related issues

Related to bluestore - Bug #56791: crash: bool SimpleBitmap::set(uint64_t, uint64_t): assert(offset + length < m_num_bits) New
Duplicated by RADOS - Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n" Duplicate
Copied to bluestore - Backport #55180: quincy: Bogus assert in SimpleBitmap Resolved

History

#1 Updated by Gabriel BenHanokh 6 months ago

  • Status changed from In Progress to Fix Under Review
  • % Done changed from 0 to 100
  • Pull request ID set to 45733

#2 Updated by Adam Kupczyk 6 months ago

  • Priority changed from Normal to Immediate

#3 Updated by Neha Ojha 6 months ago

Seen while upgrading the gibba cluster to 723fda64a662bb79871e590698268007049bcf7f

#4 Updated by Vikhyat Umrao 6 months ago

  • Description updated (diff)

#5 Updated by Vikhyat Umrao 6 months ago

  • Related to Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n" added

#6 Updated by Vikhyat Umrao 6 months ago

  • Related to deleted (Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n")

#7 Updated by Vikhyat Umrao 6 months ago

  • Duplicates Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n" added

#8 Updated by Neha Ojha 6 months ago

Quincy backport: https://github.com/ceph/ceph/pull/45738 to expedite merge

#9 Updated by Yuri Weinstein 6 months ago

Neha Ojha wrote:

Quincy backport: https://github.com/ceph/ceph/pull/45738 to expedite merge

merged

#10 Updated by Neha Ojha 6 months ago

  • Status changed from Fix Under Review to Pending Backport

#11 Updated by Backport Bot 6 months ago

#12 Updated by Neha Ojha 6 months ago

  • Status changed from Pending Backport to Resolved

#13 Updated by Yaarit Hatuka 2 months ago

  • Duplicated by Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n" added

#14 Updated by Yaarit Hatuka 2 months ago

  • Duplicates deleted (Bug #55154: Multiple OSD's during upgrade crashed with bluestore/simple_bitmap.cc: 54: FAILED ceph_assert(offset + length < m_num_bits)\n")

#15 Updated by Telemetry Bot 2 months ago

  • Related to Bug #56791: crash: bool SimpleBitmap::set(uint64_t, uint64_t): assert(offset + length < m_num_bits) added

Also available in: Atom PDF