Project

General

Profile

Actions

Bug #22120

closed

possible deadlock in various maintenance operations

Added by Jason Dillaman over 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jason Dillaman
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
luminous,jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

If an image needs to be refreshed after the start of an API-initiated maintenance operation (i.e. two or more clients manipulating the same image concurrently), there is a possibility of deadlock.

https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1731819

Thread 14 (Thread 0x7f510784c700 (LWP 759193)):
#0 0x00007f513272603e in pthread_rwlock_wrlock () from /lib64/libpthread.so.0
#1 0x00007f5112a4a83c in RWLock::get_write (this=0x5db1258, lockdep=<optimized out>) at ./common/RWLock.h:123
#2 0x00007f5112ad77c5 in WLocker (lock=..., this=<synthetic pointer>) at ./common/RWLock.h:183
#3 librbd::image::RefreshRequest<librbd::ImageCtx>::apply (this=this@entry=0x7f507c02bf10) at librbd/image/RefreshRequest.cc:855
#4 0x00007f5112ad87f8 in librbd::image::RefreshRequest<librbd::ImageCtx>::handle_v2_apply (this=0x7f507c02bf10, result=result@entry=0x7f510784bb2c) at librbd/image/RefreshRequest.cc:655
#5 0x00007f5112ad89ab in librbd::util::detail::C_StateCallbackAdapter<librbd::image::RefreshRequest<librbd::ImageCtx>, &librbd::image::RefreshRequest<librbd::ImageCtx>::handle_v2_apply, true>::complete (this=0x7f507c31e7b0, r=0) at ./librbd/Utils.h:66
#6 0x00007f5112a3eb54 in ContextWQ::process (this=0x6e96b20, ctx=0x7f507c31e7b0) at ./common/WorkQueue.h:611
#7 0x00007f5112c37a7e in ThreadPool::worker (this=0x7c222b0, wt=0x60fe290) at common/WorkQueue.cc:128
#8 0x00007f5112c38950 in ThreadPool::WorkThread::entry (this=<optimized out>) at common/WorkQueue.h:448
#9 0x00007f5132722dc5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007f5131d4873d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f5132f10740 (LWP 2617826)):
#0 0x00007f51327266d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007f5112a14b60 in Wait (mutex=..., this=0x7ffd0e2ba8f0) at ./common/Cond.h:56
#2 C_SaferCond::wait (this=this@entry=0x7ffd0e2ba890) at ./common/Cond.h:202
#3 0x00007f5112ab5b8e in librbd::Operations<librbd::ImageCtx>::snap_unprotect (this=0x562bdb0, snap_name=snap_name@entry=0x58fa894 "snapshot-90259d85-3edc-40e7-b306-cfff1b855cd6")
    at librbd/Operations.cc:1079
#4 0x00007f51129fb0d4 in rbd_snap_unprotect (image=0x5db10c0, snap_name=snap_name@entry=0x58fa894 "snapshot-90259d85-3edc-40e7-b306-cfff1b855cd6") at librbd/librbd.cc:2385
#5 0x00007f511c32f427 in __pyx_pf_3rbd_5Image_50unprotect_snap (__pyx_v_self=0x7017260, __pyx_v_self=0x7017260, __pyx_v_name=0x58fa870) at rbd.c:12928
#6 __pyx_pw_3rbd_5Image_51unprotect_snap (__pyx_v_self=0x7017260, __pyx_v_name=<optimized out>) at rbd.c:12843
#7 0x00007f5132a1ba62 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0

Related issues 2 (0 open2 closed)

Copied to rbd - Backport #22174: luminous: possible deadlock in various maintenance operationsResolvedPrashant DActions
Copied to rbd - Backport #22175: jewel: possible deadlock in various maintenance operationsResolvedNathan CutlerActions
Actions #1

Updated by Jason Dillaman over 6 years ago

  • Status changed from In Progress to Fix Under Review
Actions #2

Updated by Mykola Golub over 6 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #3

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22174: luminous: possible deadlock in various maintenance operations added
Actions #4

Updated by Nathan Cutler over 6 years ago

  • Copied to Backport #22175: jewel: possible deadlock in various maintenance operations added
Actions #5

Updated by Nathan Cutler about 6 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF