Project

General

Profile

Actions

Bug #38000

closed

The osd shutdown procedure accesses the memory that has been released

Added by tao ning over 5 years ago. Updated over 5 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

[Switching to thread 2 (Thread 0x7f7314cc8700 (LWP 32025))]
#0 0x00007f73395a842d in _lll_lock_wait () from /lib64/libpthread.so.0
(gdb) bt
#0 0x00007f73395a842d in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f73395a3dcb in _L_lock_812 () from /lib64/libpthread.so.0
#2 0x00007f73395a3c98 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00005609689bf432 in __gthread_mutex_lock (
_mutex=0x5609739e3c18)
at /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/gthr-default.h:748
#4 _gthread_recursive_mutex_lock (_mutex=0x5609739e3c18)
at /usr/include/c++/4.8.2/x86_64-redhat-linux/bits/gthr-default.h:810
#5 lock (this=0x5609739e3c18) at /usr/include/c++/4.8.2/mutex:175
#6 lock_guard (_m=..., this=<synthetic pointer>) at /usr/include/c++/4.8.2/mutex:414
#7 BlueStore::OnodeSpace::clear (this=this@entry=0x560973f37d48)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/os/bluestore/BlueStore.cc:1565
#8 0x0000560968a2a8df in ~OnodeSpace (this=0x560973f37d48, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/os/bluestore/BlueStore.h:1319
#9 ~Collection (this=<optimized out>, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/os/bluestore/BlueStore.h:1339
#10 BlueStore::Collection::~Collection (this=0x560973f37c00, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/os/bluestore/BlueStore.h:1339
#11 0x00005609685d8fbc in RefCountedObject::put (this=0x560973f37c00)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/common/RefCountedObj.h:62
#12 0x00005609686615b2 in ~intrusive_ptr (this=0x5609829b4d60, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:98
#13 PG::~PG (this=0x5609829b4000, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/PG.cc:346
#14 0x000056096879a779 in PrimaryLogPG::~PrimaryLogPG (this=0x5609829b4000, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/PrimaryLogPG.h:1377
#15 0x0000560968687e1f in ~intrusive_ptr (this=0x560998aac518, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/build/boost/include/boost/smart_ptr/intrusive_ptr.hpp:98
#16 ~QueuePeeringEvt (this=0x560998aac510, __in_chrg=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/PG.h:1455
#17 PG::QueuePeeringEvt<PG::DoRecovery>::~QueuePeeringEvt (this=0x560998aac510,
__in_chrg=<optimized out>) at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/PG.h:1455
#18 0x0000560968b43270 in SafeTimer::cancel_all_events (this=this@entry=0x560973e285c0)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/common/Timer.cc:177
#19 0x0000560968b43778 in SafeTimer::shutdown (this=this@entry=0x560973e285c0)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/common/Timer.cc:65
#20 0x0000560968571f2e in OSDService::shutdown (this=0x560973e27968)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/OSD.cc:535
#21 0x00005609685ad7d2 in OSD::shutdown (this=this@entry=0x560973e26000)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/OSD.cc:3534
#22 0x00005609685ae5c6 in OSD::handle_signal (this=0x560973e26000, signum=<optimized out>)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/osd/OSD.cc:2034
#23 0x0000560968b09600 in SignalHandler::entry (this=0x56098bf0c9c0)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/global/signal_handler.cc:304
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) f 7
#7 BlueStore::OnodeSpace::clear (this=this@entry=0x560973f37d48)
at /usr/src/debug/ceph-12.2.7-562-gd19fb7f/src/os/bluestore/BlueStore.cc:1565
1565 std::lock_guard<std::recursive_mutex> l(cache->lock);
(gdb) p cache->lock
$1 = {<std::
_recursive_mutex_base> = {_M_mutex = {__data = {__lock = 2, __count = 22025,
__owner = 1766933424, __nusers = 22025, __kind = 0, __spins = 0, __elision = 0, __list = {
__prev = 0x0, __next = 0x0}},
__size = "\002\000\000\000\tV\000\000

Actions #1

Updated by tao ning over 5 years ago

int OSD::shutdown() {
store->umount();
delete store; // The cache is destroyed
store = 0;
...
service.shutdown(); // Access to the cache
}

Actions #2

Updated by Igor Fedotov over 5 years ago

  • Project changed from Ceph to RADOS
  • Category deleted (OSD)
  • Status changed from New to Duplicate
Actions

Also available in: Atom PDF