Actions
Bug #11302
closednewstore: FAILED assert(nlock == 0)
% Done:
0%
Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Test run on 36 OSD configuration using 0.93-982-g6678e12. Several OSDs failed the same way:
2015-04-01 14:35:45.091984 7effb060f700 -1 common/Mutex.cc: In function 'Mutex::~Mutex()' thread 7effb060f700 time 2015-04-01 14:35:45.087190 common/Mutex.cc: 71: FAILED assert(nlock == 0) ceph version 0.93-982-g6678e12 (6678e120aaf031115343be9cd0b233f21af31af3) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0xcabea5] 2: /usr/bin/ceph-osd() [0xc5aba6] 3: (NewStore::OpSequencer::~OpSequencer()+0xa4) [0xa37c34] 4: (std::tr1::_Sp_counted_base_impl<ObjectStore::Sequencer*, SharedPtrRegistry<spg_t, ObjectStore::Sequencer>::OnRemoval, (__gnu_cxx::_Lock_policy)2>::_M_disp ose()+0x7b) [0x867e5b] 5: (std::tr1::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x39) [0x7184e9] 6: (PG::~PG()+0xa6) [0x811f86] 7: (ReplicatedPG::~ReplicatedPG()+0x9) [0x9256d9] 8: (intrusive_ptr_release(PG*)+0x3d) [0x804e0d] 9: (ObjectStore::C_DeleteTransactionHolder<boost::intrusive_ptr<PG> >::~C_DeleteTransactionHolder()+0x20) [0x708d20] 10: (std::tr1::_Sp_counted_base_impl<RunOnDelete*, std::tr1::_Sp_deleter<RunOnDelete>, (__gnu_cxx::_Lock_policy)2>::_M_dispose()+0x1a) [0x7aca6a] 11: (std::tr1::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x39) [0x7184e9] 12: (Wrapper<std::tr1::shared_ptr<RunOnDelete> >::~Wrapper()+0x20) [0x7acdf0] 13: (Finisher::finisher_thread_entry()+0x168) [0xbd10a8] 14: (()+0x7ee5) [0x7effbafb4ee5] 15: (clone()+0x6d) [0x7effb9a92b8d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -1907> 2015-04-01 14:35:30.994525 7effbc860980 5 asok(0x4758000) register_command perfcounters_dump hook 0x46b8050 -1906> 2015-04-01 14:35:30.994600 7effbc860980 5 asok(0x4758000) register_command 1 hook 0x46b8050 -1905> 2015-04-01 14:35:30.994611 7effbc860980 5 asok(0x4758000) register_command perf dump hook 0x46b8050 -1904> 2015-04-01 14:35:30.994621 7effbc860980 5 asok(0x4758000) register_command perfcounters_schema hook 0x46b8050 -1903> 2015-04-01 14:35:30.994629 7effbc860980 5 asok(0x4758000) register_command 2 hook 0x46b8050 -1902> 2015-04-01 14:35:30.994633 7effbc860980 5 asok(0x4758000) register_command perf schema hook 0x46b8050 -1901> 2015-04-01 14:35:30.994637 7effbc860980 5 asok(0x4758000) register_command perf reset hook 0x46b8050 -1900> 2015-04-01 14:35:30.994641 7effbc860980 5 asok(0x4758000) register_command config show hook 0x46b8050 -1899> 2015-04-01 14:35:30.994645 7effbc860980 5 asok(0x4758000) register_command config set hook 0x46b8050 -1898> 2015-04-01 14:35:30.994649 7effbc860980 5 asok(0x4758000) register_command config get hook 0x46b8050 -1897> 2015-04-01 14:35:30.994657 7effbc860980 5 asok(0x4758000) register_command config diff hook 0x46b8050 -1896> 2015-04-01 14:35:30.994663 7effbc860980 5 asok(0x4758000) register_command log flush hook 0x46b8050 -1895> 2015-04-01 14:35:30.994669 7effbc860980 5 asok(0x4758000) register_command log dump hook 0x46b8050 -1894> 2015-04-01 14:35:30.994673 7effbc860980 5 asok(0x4758000) register_command log reopen hook 0x46b8050 -1893> 2015-04-01 14:35:30.998897 7effbc860980 -1 WARNING: the following dangerous and experimental features are enabled: newstore,rocksdb -1892> 2015-04-01 14:35:30.999033 7effbc860980 -1 WARNING: the following dangerous and experimental features are enabled: newstore,rocksdb -1891> 2015-04-01 14:35:30.999312 7effbc860980 0 ceph version 0.93-982-g6678e12 (6678e120aaf031115343be9cd0b233f21af31af3), process ceph-osd, pid 1363 -1890> 2015-04-01 14:35:30.999340 7effbc860980 -1 WARNING: experimental feature 'newstore' is enabled Please be aware that this feature is experimental, untested, unsupported, and may result in data corruption, data loss, and/or irreparable damage to your cluster. Do not use feature with important data. -1889> 2015-04-01 14:35:31.002798 7effbc860980 1 finished global_init_daemonize -1888> 2015-04-01 14:35:31.005760 7effbc860980 -1 WARNING: the following dangerous and experimental features are enabled: newstore,rocksdb -1887> 2015-04-01 14:35:31.020462 7effbc860980 1 newstore(/tmp/cbt/mnt/osd-device-0-data) mount path /tmp/cbt/mnt/osd-device-0-data -1886> 2015-04-01 14:35:31.020620 7effbc860980 -1 WARNING: experimental feature 'rocksdb' is enabled Please be aware that this feature is experimental, untested, unsupported, and may result in data corruption, data loss, and/or irreparable damage to your cluster. Do not use feature with important data.
Full OSD log attached.
Files
Actions