Bug #6284: client: failed xlist assert on put_inode() - CephFS - Ceph

Actions

Copy link

Bug #6284

closed

client: failed xlist assert on put_inode()

Added by Greg Farnum over 10 years ago. Updated almost 8 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

Client

Labels (FS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

teuthology archives: /a/teuthology-2013-09-11_01:30:26-upgrade-fs-next-testing-basic-plana/29583/remote/ubuntu@plana37.front.sepia.ceph.com/log/ceph-client.0.4245.log
Blogbench, update and reboot the servers, run fsstress, and it crashed with

     0> 2013-09-11 08:23:19.091279 7f30e95d0700 -1 ./include/xlist.h: In function 'xlist<T>::~xlist() [with T = ObjectCacher::Object*]' thread 7f30e95d0700 time 2013-09-11 08:23:19.089433
./include/xlist.h: 69: FAILED assert(_size == 0)

 ceph version 0.61.8-27-gb16f812 (b16f812362ccb1d9bdd4900d155e248d695ef0d7)
 1: (Inode::~Inode()+0x28a) [0x4d44ca]
 2: (Client::put_inode(Inode*, int)+0x2c7) [0x48a697]
 3: (Context::complete(int)+0xa) [0x4cbe7a]
 4: (C_Gather::sub_finish(Context*, int)+0x117) [0x4d1497]
 5: (C_Gather::C_GatherSub::finish(int)+0x12) [0x4d1662]
 6: (Context::complete(int)+0xa) [0x4cbe7a]
 7: (finish_contexts(CephContext*, std::list<Context*, std::allocator<Context*> >&, int)+0x95) [0x600255]
 8: (ObjectCacher::bh_write_commit(long, sobject_t, long, unsigned long, unsigned long, int)+0x59f) [0x5fc95f]
 9: (ObjectCacher::C_WriteCommit::finish(int)+0x6b) [0x60379b]
 10: (Objecter::handle_osd_op_reply(MOSDOpReply*)+0xe55) [0x60f0c5]
 11: (Client::ms_dispatch(Message*)+0x8b) [0x4cb89b]
 12: (DispatchQueue::entry()+0x3f1) [0x646f81]
 13: (DispatchQueue::DispatchThread::entry()+0xd) [0x5860cd]
 14: (()+0x7e9a) [0x7f30eeb32e9a]
 15: (clone()+0x6d) [0x7f30ed2e8ccd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

I assume the xlist in question is the xlist of Objects in the oset associated with the Inode (at least, I can't find any other xlists that get destroyed via the Inode destructor). Perhaps we've got an issue with the ObjectCacher write holding the last reference to the Inode, and then when the write is completed the Client drops the Inode, but the ObjectCacher hasn't dropped its own references yet? The bit of log we have doesn't contradict this, but I'm not very familiar with the ObjectCacher right now.