Ceph » rbd

When rbd-nbd qos limit is enabled, there is small possibility that m_io_throttled assert fail.

    -1> 2020-08-07 07:19:22.171 7f99357fb700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.9/rpm/el7/BUILD/ceph-14.2.9/src/librbd/io/ImageRequestWQ.cc: In function 'void librbd::io::ImageRequestWQ<ImageCtxT>::handle_throttle_ready(int, librbd::io::ImageDispatchSpec<ImageCtxT>*, uint64_t) [with ImageCtxT = librbd::ImageCtx; uint64_t = long unsigned int]' thread 7f99357fb700 time 2020-08-07 07:19:22.170831
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.9/rpm/el7/BUILD/ceph-14.2.9/src/librbd/io/ImageRequestWQ.cc: 662: FAILED ceph_assert(m_io_throttled.load() > 0)

 ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x7f9955cf0875]
 2: (()+0x253a3d) [0x7f9955cf0a3d]
 3: (()+0x2127e1) [0x7f995f1fe7e1]
 4: (()+0xc654c) [0x7f995f0b254c]
 5: (()+0xab539) [0x7f995f097539]
 6: (TokenBucketThrottle::add_tokens()+0xdc) [0x7f9955dbad8c]
 7: (TokenBucketThrottle::schedule_timer()+0xb3) [0x7f9955dbaea3]
 8: (FunctionContext::finish(int)+0x2c) [0x7f9955dbf11c]
 9: (Context::complete(int)+0x9) [0x7f9955d6fd89]
 10: (SafeTimer::timer_thread()+0x180) [0x7f9955dc1bf0]
 11: (SafeTimerThread::entry()+0xd) [0x7f9955dc345d]
 12: (()+0x7e25) [0x7f99541d7e25]
 13: (clone()+0x6d) [0x7f9952a7d34d]

     0> 2020-08-07 07:19:22.172 7f99357fb700 -1 *** Caught signal (Aborted) **
 in thread 7f99357fb700 thread_name:safe_timer

In some corner case, handle_throttle_ready(safe_timer thread) may run faster than needs_throttle(rbd_op_thread), and before m_io_throttled(old value as 0) plus 1, ceph_assert(m_io_throttled.load() > 0) executed, which cause assert failed.