Actions
Bug #11204
closedceph-objectstore-tool: "error (24) Too many open files not handled on operation" during import of a large number of objects
Status:
Resolved
Priority:
Urgent
Assignee:
David Zafman
Category:
teuthology
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Run: http://pulpito.front.sepia.ceph.com/teuthology-2015-03-22_09:17:25-powercycle-hammer-distro-basic-multi/
Jobs: ['815837', '815865']
Logs for one: http://qa-proxy.ceph.com/teuthology/teuthology-2015-03-22_09:17:25-powercycle-hammer-distro-basic-multi/815865/
015-03-22T09:29:11.936 INFO:teuthology.orchestra.run.burnupi10.stdout:Write 4df9daee/100000001a6.00000000/head//2 2015-03-22T09:29:11.937 INFO:teuthology.orchestra.run.burnupi10.stdout:Write e96b4bee/10000002010.00000000/head//2 2015-03-22T09:29:11.938 INFO:tasks.workunit.client.0.plana80.stdout:linux-3.2.9/arch/powerpc/include/asm/atomic.h 2015-03-22T09:29:11.939 INFO:teuthology.orchestra.run.burnupi10.stderr:os/FileStore.cc: In function 'unsigned int FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t, int, ThreadPool::TPHandle*)' thread 7f3a863f7700 time 2015-03-22 09:29:11.938804 2015-03-22T09:29:11.939 INFO:teuthology.orchestra.run.burnupi10.stderr:os/FileStore.cc: 2757: FAILED assert(0 == "unexpected error") 2015-03-22T09:29:11.939 INFO:teuthology.orchestra.run.burnupi10.stderr: ceph version 0.93-164-g120300f (120300febebdc34c23fc24633edfdc8faf65e93c) 2015-03-22T09:29:11.939 INFO:teuthology.orchestra.run.burnupi10.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xbd239b] 2015-03-22T09:29:11.939 INFO:teuthology.orchestra.run.burnupi10.stderr: 2: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int, ThreadPool::TPHandle*)+0xa52) [0x99aeb2] 2015-03-22T09:29:11.940 INFO:teuthology.orchestra.run.burnupi10.stderr: 3: (FileStore::_do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long, ThreadPool::TPHandle*)+0x64) [0x9a1544] 2015-03-22T09:29:11.940 INFO:teuthology.orchestra.run.burnupi10.stderr: 4: (FileStore::_do_op(FileStore::OpSequencer*, ThreadPool::TPHandle&)+0x16a) [0x9a16ca] 2015-03-22T09:29:11.940 INFO:teuthology.orchestra.run.burnupi10.stderr: 5: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa5e) [0xbc36ce] 2015-03-22T09:29:11.940 INFO:teuthology.orchestra.run.burnupi10.stderr: 6: (ThreadPool::WorkThread::entry()+0x10) [0xbc4770] 2015-03-22T09:29:11.941 INFO:teuthology.orchestra.run.burnupi10.stderr: 7: (()+0x8182) [0x7f3a90848182] 2015-03-22T09:29:11.941 INFO:teuthology.orchestra.run.burnupi10.stderr: 8: (clone()+0x6d) [0x7f3a8c73a38d] 2015-03-22T09:29:11.941 INFO:teuthology.orchestra.run.burnupi10.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2015-03-22T09:29:11.942 INFO:tasks.workunit.client.0.plana80.stdout:linux-3.2.9/arch/powerpc/include/asm/auxvec.h 2015-03-22T09:29:11.942 INFO:teuthology.orchestra.run.burnupi10.stderr:terminate called after throwing an instance of 'ceph::FailedAssertion' 2015-03-22T09:29:11.942 INFO:teuthology.orchestra.run.burnupi10.stderr:*** Caught signal (Aborted) ** 2015-03-22T09:29:11.942 INFO:teuthology.orchestra.run.burnupi10.stderr: in thread 7f3a863f7700 2015-03-22T09:29:11.944 INFO:teuthology.orchestra.run.burnupi10.stderr: ceph version 0.93-164-g120300f (120300febebdc34c23fc24633edfdc8faf65e93c) 2015-03-22T09:29:11.944 INFO:teuthology.orchestra.run.burnupi10.stderr: 1: ceph-objectstore-tool() [0xaf09ea] 2015-03-22T09:29:11.944 INFO:teuthology.orchestra.run.burnupi10.stderr: 2: (()+0x10340) [0x7f3a90850340] 2015-03-22T09:29:11.944 INFO:teuthology.orchestra.run.burnupi10.stderr: 3: (gsignal()+0x39) [0x7f3a8c675f89] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 4: (abort()+0x148) [0x7f3a8c679398] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f3a8cf816b5] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 6: (()+0x5e836) [0x7f3a8cf7f836] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 7: (()+0x5e863) [0x7f3a8cf7f863] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 8: (()+0x5eaa2) [0x7f3a8cf7faa2] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0xbd2588] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int, ThreadPool::TPHandle*)+0xa52) [0x99aeb2] 2015-03-22T09:29:11.945 INFO:teuthology.orchestra.run.burnupi10.stderr: 11: (FileStore::_do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long, ThreadPool::TPHandle*)+0x64) [0x9a1544] 2015-03-22T09:29:11.946 INFO:teuthology.orchestra.run.burnupi10.stderr: 12: (FileStore::_do_op(FileStore::OpSequencer*, ThreadPool::TPHandle&)+0x16a) [0x9a16ca] 2015-03-22T09:29:11.946 INFO:teuthology.orchestra.run.burnupi10.stderr: 13: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa5e) [0xbc36ce] 2015-03-22T09:29:11.946 INFO:teuthology.orchestra.run.burnupi10.stderr: 14: (ThreadPool::WorkThread::entry()+0x10) [0xbc4770] 2015-03-22T09:29:11.946 INFO:teuthology.orchestra.run.burnupi10.stderr: 15: (()+0x8182) [0x7f3a90848182] 2015-03-22T09:29:11.946 INFO:teuthology.orchestra.run.burnupi10.stderr: 16: (clone()+0x6d) [0x7f3a8c73a38d]
Actions