Bug #11381
closedmessenger: double clear of pipe in reaper
0%
Description
2015-04-13 06:04:05.049191 7fa3f518a700 -1 msg/simple/SimpleMessenger.cc: In function 'void SimpleMessenger::reaper()' thread 7fa3f518a700 time 2015-04-13 06:04:05.047092 msg/simple/SimpleMessenger.cc: 237: FAILED assert(!cleared) ceph version 0.94-5-g7338647 (733864738fa93979727e480e403293e079bb51e9) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0xaebf4f] 2: (SimpleMessenger::reaper()+0x9b1) [0xac5381] 3: (SimpleMessenger::reaper_entry()+0x134) [0xac6554] 4: (SimpleMessenger::ReaperThread::entry()+0xd) [0xaccc2d] 5: (()+0x7e9a) [0x7fa3fa556e9a] 6: (clone()+0x6d) [0x7fa3f8cff3fd] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Updated by Haomai Wang about 9 years ago
Hi Greg, I'm not sure whether you have solved this problem. I just remember that in Pipe::fault we have potential probability to cause this.
In pipe.cc:1353 we stop this connection and we will let reader and write threads stop. If now reader and writer quit ASAP and we call queue_reap to trigger the reap progress. Now we haven't call "connection_state->clear_pipe(this)" in pipe.cc:1379, so we may assert cleared here.
Is it true?
Updated by Kefu Chai about 9 years ago
- Status changed from New to Fix Under Review
- Assignee set to Haomai Wang
Updated by Greg Farnum about 9 years ago
- Status changed from Fix Under Review to Pending Backport
- Backport set to hammer
Merged to master in commit:927105b0213e26abb3468624ba1bd85abb8ff429
Do we need/want to backport this to anything other than hammer? :/
Updated by Greg Farnum about 9 years ago
Wow, we hit this in a hammer test run! http://pulpito-rdu.front.sepia.ceph.com/teuthology-2015-04-23_16:08:01-kcephfs-hammer-testing-basic-typica/3783/
Updated by Loïc Dachary about 9 years ago
- hammer backport https://github.com/ceph/ceph/pull/4506
Updated by Greg Farnum about 9 years ago
- Status changed from Pending Backport to Resolved
Merged to hammer in commit:abc0741d57f30a39a18106bf03576e980ad89177
Updated by Greg Farnum about 5 years ago
- Project changed from Ceph to Messengers
- Category deleted (
msgr)