Ceph

Date: Sat, 19 Oct 2013 23:53:57 +0400
From: Andrey Korolyov <andrey@xdel.ru>
To: "ceph-users@lists.ceph.com" <ceph-users@lists.ceph.com>
Subject: [ceph-users] Cuttlefish: pool recreation results in cluster crash
Parts/Attachments:
1 Shown 11 lines Text (charset: UTF-8)
2 5.5 KB Application
3 Shown 5 lines Text
----------------------------------------

[ The following text is in the "UTF-8" character set. ]
    [ Your display is set for the "ANSI_X3.4-1968" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hello,

I was able to reproduce following on the top of current cuttlefish:

- create pool,
- delete it after all pgs initialized,
- create new pool with same name after, say, ten seconds.

All osds dies immediately with attached trace. The problem exists in
bobtail as well.

Thread 1 (Thread 0x7f4fdf42b700 (LWP 28165)):
#0  0x00007f4fe78ee405 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f4fe78f1b5b in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007f4fe81ec875 in __gnu_cxx::__verbose_terminate_handler () at ../../../../src/libstdc++-v3/libsupc++/vterminate.cc:50
#3  0x00007f4fe81ea996 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:40
#4  0x00007f4fe81ea9c3 in std::terminate () at ../../../../src/libstdc++-v3/libsupc++/eh_terminate.cc:50
#5  0x00007f4fe81eabee in __cxxabiv1::__cxa_throw (obj=0x2ec91f0, tinfo=<optimized out>, dest=<optimized out>)
    at ../../../../src/libstdc++-v3/libsupc++/eh_throw.cc:83
#6  0x000000000090d3fa in ceph::__ceph_assert_fail (assertion=0xa2c697 "0 == \"unexpected error\"", file=<optimized out>, line=2787, 
    func=0xa2f7a0 "unsigned int FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int)") at common/assert.cc:77
#7  0x00000000008211ab in FileStore::_do_transaction (this=this@entry=0x24a4000, t=..., op_seq=op_seq@entry=226873, 
    trans_num=trans_num@entry=0) at os/FileStore.cc:2787
#8  0x0000000000824829 in FileStore::_do_transactions (this=this@entry=0x24a4000, tls=std::list, op_seq=226873, 
    handle=handle@entry=0x7f4fdf42ab80) at os/FileStore.cc:2163
#9  0x00000000008249be in FileStore::_do_op (this=0x24a4000, osr=<optimized out>, handle=...) at os/FileStore.cc:1997
#10 0x0000000000903eca in ThreadPool::worker (this=0x24a4a10, wt=0x24617a0) at common/WorkQueue.cc:119
#11 0x0000000000905170 in ThreadPool::WorkThread::entry (this=<optimized out>) at common/WorkQueue.h:316
#12 0x00007f4fe9663e9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#13 0x00007f4fe79aa3dd in clone () from /lib/x86_64-linux-gnu/libc.so.6