Project

General

Profile

Actions

Bug #46083

closed

[object-map] possible race condition when disabling object map with active IO

Added by Jason Dillaman almost 4 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jason Dillaman
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

If the object-map feature is disabled while there is in-flight IO, there is a possibility that the ObjectMap contains one or more detained writes that might complete after the instance has been destroyed. Delay the close until after all detained IO has been released.

http://qa-proxy.ceph.com/teuthology/trociny-2020-06-18_09:19:50-rbd-wip-mgolub-testing-distro-basic-smithi/5160316/teuthology.log

#0  0x00007f06e47c7fa0 in __GI___pthread_mutex_lock (mutex=0x8) at ../nptl/pthread_mutex_lock.c:65
#1  0x00007f06da2e869d in __gthread_mutex_lock (__mutex=0x8) at /usr/include/x86_64-linux-gnu/c++/7/bits/gthr-default.h:748
#2  0x00007f06da2e869d in std::mutex::lock() (this=0x8) at /usr/include/c++/7/bits/std_mutex.h:103
#3  0x00007f06da2e869d in std::lock_guard<std::mutex>::lock_guard(std::mutex&) (__m=..., this=<synthetic pointer>) at /usr/include/c++/7/bits/std_mutex.h:162
#4  0x00007f06da2e869d in librbd::BlockGuard<librbd::ObjectMap<librbd::ImageCtx>::UpdateOperation>::release(librbd::BlockGuardCell*, std::__cxx11::list<librbd::ObjectMap<librbd::ImageCtx>::UpdateOperation, std::allocator<librbd::ObjectMap<librbd::ImageCtx>::UpdateOperation> >*) (this=0x0, cell=cell@entry=0x7f06787b23f0, block_operations=block_operations@entry=0x7f069a7fb6b0) at ./src/librbd/BlockGuard.h:113
#5  0x00007f06da2ece39 in librbd::ObjectMap<librbd::ImageCtx>::handle_detained_aio_update(librbd::BlockGuardCell*, int, Context*) (this=0x5557524ada10, cell=0x7f06787b23f0, r=0, on_finish=0x7f067800a070)
    at ./src/librbd/ObjectMap.cc:307
#6  0x00007f06da4e17fc in Context::complete(int) (r=<optimized out>, this=0x7f0684015cf0) at ./src/include/Context.h:99
#7  0x00007f06da4e17fc in librbd::AsyncRequest<librbd::ImageCtx>::finish(int) (r=0, this=0x7f069407a720) at ./src/librbd/AsyncRequest.h:60
#8  0x00007f06da4e17fc in librbd::AsyncRequest<librbd::ImageCtx>::finish_and_destroy(int) (this=0x7f069407a720, r=0) at ./src/librbd/AsyncRequest.h:54
#9  0x00007f06da4e18c6 in Context::complete(int) (r=<optimized out>, this=0x555752eb5a40) at ./src/include/Context.h:99
#10 0x00007f06da4e18c6 in librbd::AsyncRequest<librbd::ImageCtx>::finish(int) (r=0, this=0x5557537738f0) at ./src/librbd/AsyncRequest.h:60
#11 0x00007f06da4e18c6 in librbd::AsyncRequest<librbd::ImageCtx>::finish_and_destroy(int) (r=0, this=0x5557537738f0) at ./src/librbd/AsyncRequest.h:54
#12 0x00007f06da4e18c6 in librbd::AsyncRequest<librbd::ImageCtx>::complete(int) (r=<optimized out>, this=0x5557537738f0) at ./src/librbd/AsyncRequest.h:25
#13 0x00007f06da4e18c6 in librbd::util::detail::rados_callback<librbd::AsyncRequest<librbd::ImageCtx> >(void*, void*) (c=<optimized out>, arg=0x5557537738f0) at ./src/librbd/Utils.h:27
#14 0x00007f06d9eaff47 in librados::CB_AioComplete::operator()() (this=<synthetic pointer>) at ./src/librados/AioCompletionImpl.h:140

Related issues 2 (0 open2 closed)

Copied to rbd - Backport #46149: nautilus: [object-map] possible race condition when disabling object map with active IOResolvedMykola GolubActions
Copied to rbd - Backport #46150: octopus: [object-map] possible race condition when disabling object map with active IOResolvedNathan CutlerActions
Actions #1

Updated by Jason Dillaman almost 4 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 35658
Actions #2

Updated by Mykola Golub almost 4 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #3

Updated by Nathan Cutler almost 4 years ago

  • Copied to Backport #46149: nautilus: [object-map] possible race condition when disabling object map with active IO added
Actions #4

Updated by Nathan Cutler almost 4 years ago

  • Copied to Backport #46150: octopus: [object-map] possible race condition when disabling object map with active IO added
Actions #5

Updated by Loïc Dachary almost 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF