Project

General

Profile

Bug #14206

BlueStore Crashed

Added by James Liu over 5 years ago. Updated over 5 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

-41> 2015-12-30 14:48:50.952056 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319581184~65536 aio 0x7f0b0a6d1a60
-40> 2015-12-30 14:48:50.955019 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319646720~65536 aio 0x7f0b0a6d1c80
-39> 2015-12-30 14:48:50.957912 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319712256~65536 aio 0x7f0b0a6e4060
-38> 2015-12-30 14:48:50.960838 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319777792~65536 aio 0x7f0b0a6e4260
-37> 2015-12-30 14:48:50.963995 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319843328~65536 aio 0x7f0b0a6e4440
-36> 2015-12-30 14:48:50.967182 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319908864~65536 aio 0x7f0b0a6e46f0
-35> 2015-12-30 14:48:50.970089 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2319974400~65536 aio 0x7f0b0a6e48d0
-34> 2015-12-30 14:48:50.972977 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320039936~65536 aio 0x7f0b0a6e4af0
-33> 2015-12-30 14:48:50.975957 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320105472~65536 aio 0x7f0b0a6e4de0
-32> 2015-12-30 14:48:50.979289 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320171008~65536 aio 0x7f0b0a6f7060
-31> 2015-12-30 14:48:50.982391 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320236544~65536 aio 0x7f0b0a6f7240
-30> 2015-12-30 14:48:50.985282 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320302080~65536 aio 0x7f0b0a6f7530
-29> 2015-12-30 14:48:50.988213 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320367616~65536 aio 0x7f0b0a6f7710
-28> 2015-12-30 14:48:50.991254 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320433152~65536 aio 0x7f0b0a6f7930
-27> 2015-12-30 14:48:50.994567 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320498688~65536 aio 0x7f0b0a6f7c20
-26> 2015-12-30 14:48:50.997491 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320564224~65536 aio 0x7f0b0a6f7e00
-25> 2015-12-30 14:48:51.000370 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320629760~65536 aio 0x7f0b0a70a0a0
-24> 2015-12-30 14:48:51.003303 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320695296~65536 aio 0x7f0b0a70a2c0
-23> 2015-12-30 14:48:51.006507 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320760832~65536 aio 0x7f0b0a70a5b0
-22> 2015-12-30 14:48:51.009658 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320826368~65536 aio 0x7f0b0a70a790
-21> 2015-12-30 14:48:51.012558 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320891904~65536 aio 0x7f0b0a70a9b0
-20> 2015-12-30 14:48:51.015447 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2320957440~65536 aio 0x7f0b0a70aca0
-19> 2015-12-30 14:48:51.018460 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321022976~65536 aio 0x7f0b0a70ae80
-18> 2015-12-30 14:48:51.021757 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321088512~65536 aio 0x7f0b0a71d150
-17> 2015-12-30 14:48:51.024799 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321154048~65536 aio 0x7f0b0a71d370
-16> 2015-12-30 14:48:51.027694 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321219584~65536 aio 0x7f0b0a71d590
-15> 2015-12-30 14:48:51.030639 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321285120~65536 aio 0x7f0b0a71d880
-14> 2015-12-30 14:48:51.033706 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321350656~65536 aio 0x7f0b0a71da60
-13> 2015-12-30 14:48:51.037007 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321416192~65536 aio 0x7f0b0a71dc80
-12> 2015-12-30 14:48:51.039916 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321481728~65536 aio 0x7f0a87ffa060
-11> 2015-12-30 14:48:51.042803 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321547264~65536 aio 0x7f0a87ffa240
-10> 2015-12-30 14:48:51.045745 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321612800~65536 aio 0x7f0a87ffa420
-9> 2015-12-30 14:48:51.048980 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321678336~65536 aio 0x7f0a87ffa6d0
-8> 2015-12-30 14:48:51.052088 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321743872~65536 aio 0x7f0a87ffa8b0
-7> 2015-12-30 14:48:51.054978 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321809408~65536 aio 0x7f0a87ffaa90
-6> 2015-12-30 14:48:51.057872 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321874944~65536 aio 0x7f0a87ffad80
-5> 2015-12-30 14:48:51.060944 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2321940480~65536 aio 0x7f0b0a730060
-4> 2015-12-30 14:48:51.064248 7f0b3d7fa700 5 bdev(/mnt/nvmedevice/data/block) aio_write 2322006016~65536 aio 0x7f0b0a730240
-3> 2015-12-30 14:48:51.125977 7f0b3d7fa700 -1 bdev(/mnt/nvmedevice/data/block) aio_submit retries 1
-2> 2015-12-30 14:48:51.132117 7f0b3d7fa700 -1 bdev(/mnt/nvmedevice/data/block) aio_submit retries 10
-1> 2015-12-30 14:48:51.132135 7f0b3d7fa700 -1 bdev(/mnt/nvmedevice/data/block) aio submit got (11) Resource temporarily unavailable
0> 2015-12-30 14:48:51.133272 7f0b3d7fa700 -1 os/bluestore/BlockDevice.cc: In function 'void BlockDevice::aio_submit(IOContext*)' thread 7f0b3d7fa700 time 2015-12-30 14:48:51.132175
os/bluestore/BlockDevice.cc: 327: FAILED assert(r == 0)

ceph version 10.0.0-1198-g0ad3f26 (0ad3f266ec8e6bf1a57f1773047562800b8ee28b)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f0b4a5c0f1b]
2: (BlockDevice::aio_submit(IOContext*)+0x7f5) [0x7f0b4a3f4e55]
3: (BlueStore::_txc_aio_submit(BlueStore::TransContext*)+0x65) [0x7f0b4a2b31e5]
4: (BlueStore::_txc_state_proc(BlueStore::TransContext*)+0x491) [0x7f0b4a2cd101]
5: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x5c2) [0x7f0b4a2f3b72]
6: (osbench_worker(ObjectStore*, Config const&, coll_t, ghobject_t, unsigned long)+0x6c8) [0x7f0b4a2a96c8]
7: (std::thread::_Impl<std::_Bind_simple<void ((ObjectStore, std::reference_wrapper<Config>, coll_t, ghobject_t, unsigned long))(ObjectStore*, Config const&, coll_t, ghobject_t, unsigned long)> >::_M_run()+0xff) [0x7f0b4a2aa22f]
8: (()+0xb1a40) [0x7f0b47c80a40]
9: (()+0x8182) [0x7f0b489b2182]
10: (clone()+0x6d) [0x7f0b473e830d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

History

#1 Updated by Loïc Dachary over 5 years ago

  • Status changed from New to Rejected

It's moving a lot, hopefully this is fixed now :-) Thanks for your report !

Also available in: Atom PDF