Project

General

Profile

Bug #51843

osd/scrub: OSD crashes at PG removal

Added by Ronen Friedman 2 months ago. Updated 2 months ago.

Status:
Fix Under Review
Priority:
Normal
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

A dout() message in the scrubber code might cause an OSD crash, if issued when the PG is already removed.

BT might look like this:

(gdb) bt
#0 0x00007f1376c09acf in uw_frame_state_for () from /lib64/libgcc_s.so.1
#1 0x00007f1376c0b758 in _Unwind_Backtrace () from /lib64/libgcc_s.so.1
#2 0x00007f137693ebe6 in backtrace () from /lib64/libc.so.6
#3 0x00005638b5ac2c97 in handle_oneshot_fatal_signal(int) ()
#4 <signal handler called>
#5 0x0000000000000003 in ?? ()
#6 0x00005638b540d5a7 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() ()
#7 0x00005638b54f9737 in PG::gen_prefix(std::ostream&) const ()
#8 0x00005638b564e1b4 in std::ostream& _prefix<PG>(std::ostream*, PG*) ()
#9 0x00005638b5652ae9 in PgScrubber::rm_from_osd_scrubbing() ()
#10 0x00005638b566200f in PgScrubber::~PgScrubber() ()
#11 0x00005638b567c3f5 in PrimaryLogScrub::~PrimaryLogScrub() ()
#12 0x00005638b562470d in PrimaryLogPG::~PrimaryLogPG() ()
#13 0x00005638b54fde89 in PG::put(char const*) ()
#14 0x00005638b5521b3b in ContainerContext<boost::intrusive_ptr<PG> >::~ContainerContext() ()
#15 0x00005638b547aad4 in OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*) ()

History

#1 Updated by Ronen Friedman 2 months ago

  • Status changed from New to Triaged

Caused by the incorrect reliance of some Scrubber sub-objects on PG::gen_prefix() for their logging.

#2 Updated by Ronen Friedman 2 months ago

  • Status changed from Triaged to Fix Under Review
  • Pull request ID set to 42479

#3 Updated by Neha Ojha 2 months ago

  • Project changed from Ceph to RADOS
  • Category deleted (OSD)

Also available in: Atom PDF