Bug #15150: librbd: AioImageRequestWQ::aio_write triggered helgrind assert - rbd - Ceph

Actions

Copy link

Bug #15150

closed

librbd: AioImageRequestWQ::aio_write triggered helgrind assert

Added by Josh Durgin about 8 years ago. Updated almost 8 years ago.

Status:

Duplicate

Priority:

Normal

Assignee:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

http://teuthology.ovh.sepia.ceph.com/teuthology/joshd-2016-03-15_20:25:10-rbd-master---basic-openstack/16261/teuthology.log

2016-03-15T20:49:50.628 INFO:tasks.workunit.client.0.target072210.stdout:[ RUN      ] TestLibRBD.FlushAio
2016-03-15T20:49:50.639 INFO:tasks.workunit.client.0.target072210.stdout:using new format!
2016-03-15T20:49:52.610 INFO:tasks.workunit.client.0.target072210.stderr:==11160== ---Thread-Announcement------------------------------------------
2016-03-15T20:49:52.610 INFO:tasks.workunit.client.0.target072210.stderr:==11160==
2016-03-15T20:49:52.611 INFO:tasks.workunit.client.0.target072210.stderr:==11160== Thread #1 is the program's root thread
2016-03-15T20:49:52.611 INFO:tasks.workunit.client.0.target072210.stderr:==11160==
2016-03-15T20:49:52.611 INFO:tasks.workunit.client.0.target072210.stderr:==11160== ----------------------------------------------------------------
2016-03-15T20:49:52.612 INFO:tasks.workunit.client.0.target072210.stderr:==11160==
2016-03-15T20:49:52.612 INFO:tasks.workunit.client.0.target072210.stderr:==11160== Thread #1: pthread_rwlock_{rd,rw}lock with a pthread_mutex_t* argument
2016-03-15T20:49:52.612 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    at 0xA955E16: ??? (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
2016-03-15T20:49:52.612 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x43EBDE: RWLock::get_read() const (RWLock.h:100)
2016-03-15T20:49:52.612 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x49AD60: RLocker (RWLock.h:161)
2016-03-15T20:49:52.719 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x49AD60: librbd::AioImageRequestWQ::start_in_flight_op(librbd::AioCompletion*) (AioImageRequestWQ.cc:376)
2016-03-15T20:49:52.719 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x49BA02: librbd::AioImageRequestWQ::aio_write(librbd::AioCompletion*, unsigned long, unsigned long, char const*, int, bool) (AioImageRequestWQ.cc:146)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x4867E2: rbd_aio_write (librbd.cc:2398)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x3A3613: TestLibRBD_FlushAio_Test::TestBody() (test_librbd.cc:2213)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8AF7E2: HandleSehExceptionsInMethodIfSupported<testing::Test, void> (gtest.cc:2078)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8AF7E2: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2114)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A2046: testing::Test::Run() (gtest.cc:2151)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A20ED: testing::TestInfo::Run() (gtest.cc:2326)
2016-03-15T20:49:52.720 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A21F4: testing::TestCase::Run() (gtest.cc:2444)
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A24A7: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4315)
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A2753: HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (gtest.cc:2078)
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A2753: HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (gtest.cc:2114)
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:==11160==    by 0x8A2753: testing::UnitTest::Run() (gtest.cc:3929)
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:==11160==
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:
2016-03-15T20:49:52.721 INFO:tasks.workunit.client.0.target072210.stderr:Helgrind: hg_main.c:295 (lockN_acquire_reader): Assertion 'lk->kind == LK_rdwr' failed.

Actions

Copy link

Updated by Jason Dillaman about 8 years ago

This is that fun issue where if a spin lock isn't properly destroyed and later a rwlock is placed in the same memory address, helgrind get's upset (it internally treats spinlocks as mutexes). So most likely someone outside of librbd is using a spinlock and not properly freeing it (again).

Actions

Copy link