Bug #22157
Updated by Patrick Donnelly over 6 years ago
<pre>
0> 2017-11-10 17:04:37.887573 7f7bc2d64700 -1 /build/ceph-12.2.1/src/include/xlist.h: In function 'xlist<T>::iterator& xlist<T>::iterator::operator++() [with T = Cap*]' thread 7f7bc2d64700 time 2017-11-10 17:04:37.674883
/build/ceph-12.2.1/src/include/xlist.h: 166: FAILED assert(cur->_list)
ceph version 12.2.1 (3e7492b9ada8bdc9a5cd0feafd42fbca27f9c38e) luminous (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x7f7bcc73aa4e]
2: (Client::trim_caps(MetaSession*, int)+0x7cd) [0x7f7bcc6acadd]
3: (Client::handle_client_session(MClientSession*)+0x3a1) [0x7f7bcc6ace91]
4: (Client::ms_dispatch(Message*)+0x54f) [0x7f7bcc6d9faf]
5: (DispatchQueue::entry()+0x78b) [0x7f7bcca2cf2b]
6: (DispatchQueue::DispatchThread::entry()+0xd) [0x7f7bcc7cb2dd]
7: (()+0x8184) [0x7f7bcb1b8184]
8: (clone()+0x6d) [0x7f7bca085bed]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
</pre>
There are two ways I can see this happening:
This is caused by:
1) We are trimming the auth_cap for the inode and there are other caps for the inode:
https://github.com/ceph/ceph/blob/f870729aa15ad1156fcd86b66ed601e659d3cf7a/src/client/Client.cc#L4069
The iterator points to one of those other caps which are removed during trim_dentry here:
https://github.com/ceph/ceph/blob/f870729aa15ad1156fcd86b66ed601e659d3cf7a/src/client/Client.cc#L4093
2) (Most likely) We are trimming the cap for the last dentry in a directory and the trim_caps iterator is currently pointing to the directory inode's cap. Client::trim_dentry will drop the dir inode/cap when calling Client::unlink:
https://github.com/ceph/ceph/blob/f870729aa15ad1156fcd86b66ed601e659d3cf7a/src/client/Client.cc#L672
This is probably caused by this refactoring patch:
<pre>
commit d6a6b03a0dd239a889332fd641cc6a3e99b961c1 HEAD (upstream/pull/12161/head)
Author: John Spray <john.spray@redhat.com>
Date: Tue Nov 29 17:13:29 2016 +0000
client: simplify remove_cap interface
All this complexity only existed to handle the
case where trim wanted to iterate over an xlist
without getting disturbed by a deletion. We
can simply increment the iterator before
maybe-deleting the Cap.
Signed-off-by: John Spray <john.spray@redhat.com>
</pre>
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1514298