Project

General

Profile

Bug #10968

librbd: copy-on-read request outstanding while closing image

Added by Josh Durgin over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Urgent
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

This caused a crash in ceph_test_librbd_fsx:

Thread 35 (Thread 0x7fa7677fe700 (LWP 650)):
#0  0x00007fa77d875dd7 in PerfCounters::inc (this=0x29593e0, idx=26011, amt=1) at common/perf_counters.cc:142
#1  0x00007fa77d709c48 in librbd::aio_read (ictx=0x2956d40, image_extents=..., buf=0x0, pbl=0x7fa6f40e7e98, c=0x7fa7484a35b0, op_flags=0) at librbd/internal.cc:3615
#2  0x00007fa77d6d7ceb in librbd::AioRequest::read_from_parent (this=0x7fa6f40e7e50, image_extents=...) at librbd/AioRequest.cc:60
#3  0x00007fa77d6d9a02 in librbd::AioRead::should_complete (this=0x7fa6f40e7e50, r=<optimized out>) at librbd/AioRequest.cc:147
#4  0x00007fa77d702b03 in complete (r=-2, this=0x7fa6f40e7e50) at ./librbd/AioRequest.h:39
#5  librbd::rados_req_cb (c=<optimized out>, arg=<optimized out>) at librbd/internal.cc:3136
#6  0x00007fa77b1afa5d in librados::C_AioComplete::finish (this=0x7fa700000f30, r=<optimized out>) at ./librados/AioCompletionImpl.h:180
#7  0x00007fa77b18d0d9 in Context::complete (this=0x7fa700000f30, r=<optimized out>) at ./include/Context.h:65
#8  0x00007fa77b2528c0 in Finisher::finisher_thread_entry (this=0x28ce208) at common/Finisher.cc:59
#9  0x00007fa77a9d0e9a in start_thread (arg=0x7fa7677fe700) at pthread_create.c:308
#10 0x00007fa779ce63fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112
#11 0x0000000000000000 in ?? ()

#0  0x00007fa779c28425 in __GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#1  0x00007fa779c2bb8b in __GI_abort () at abort.c:91
#2  0x00007fa77a57b69d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007fa77a579846 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007fa77a579873 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007fa77a57996e in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fa77d7f1209 in ceph::__ceph_assert_fail (assertion=<optimized out>, file=<optimized out>, line=51, func=0x7fa77da85400 "virtual RWLock::~RWLock()") at common/assert.cc:77
#7  0x00007fa77d6cbe72 in RWLock::~RWLock (this=<optimized out>, __in_chrg=<optimized out>) at ./common/RWLock.h:51
#8  0x00007fa77d6ec250 in ~RWLock (this=0x2956270, __in_chrg=<optimized out>) at ./common/RWLock.h:51
#9  librbd::ImageCtx::~ImageCtx (this=0x2955f60, __in_chrg=<optimized out>) at librbd/ImageCtx.cc:115
#10 0x00007fa77d70b4e8 in librbd::close_image (ictx=0x2955f60) at librbd/internal.cc:2448
#11 0x00007fa77d70b458 in librbd::close_image (ictx=0x2940fe0) at librbd/internal.cc:2430
#12 0x00007fa77d6d26e9 in rbd_close (image=<optimized out>) at librbd/librbd.cc:1054
#13 0x00000000004b5345 in __librbd_close (ctx=0x7fffb95dc230) at test/librbd/fsx.c:326
#14 0x00000000004b712c in check_clone (clonenum=<optimized out>) at test/librbd/fsx.c:1529
#15 0x00000000004b7404 in do_clone () at test/librbd/fsx.c:1470
#16 0x00000000004b7cd4 in test () at test/librbd/fsx.c:1725
#17 0x00000000004b3ceb in main (argc=<optimized out>, argv=<optimized out>) at test/librbd/fsx.c:2255

The core and binary are still on mira090:~/cephtest if any further info is needed.

Associated revisions

Revision 137800a3 (diff)
Added by Jason Dillaman over 5 years ago

librbd: delay completion of AioRequest::read_from_parent

If the object map is enabled, it's possible for a read request to
instantly complete due to the skipped librados operations. Now
AioRequest will block the completion of read_from_parent requests
to prevent the possibility of the parent image being closed while
the read_from_parent method invocation is in-progress.

Fixes: #10968
Signed-off-by: Jason Dillaman <>

History

#1 Updated by Jason Dillaman over 5 years ago

  • Status changed from New to In Progress
  • Assignee set to Jason Dillaman

#2 Updated by Jason Dillaman over 5 years ago

  • Status changed from In Progress to Fix Under Review

#3 Updated by Jason Dillaman over 5 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF