Bug #17724: SubsystemMap.h: 62: FAILED assert(sub < m_subsys.size()) - Ceph - Ceph

Actions

Copy link

Bug #17724

open

SubsystemMap.h: 62: FAILED assert(sub < m_subsys.size())

Added by David Zafman over 7 years ago. Updated over 4 years ago.

Status:

New

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

http://pulpito.ceph.com/yuriw-2016-10-25_16:08:46-rados-master-distro-basic-smithi/494555

suites: ['1thrash/none.yaml', 'clusters/{fixed-2.yaml', 'fs/btrfs.yaml', 'mon_kv_backend/rocksdb.yaml', 'msgr-failures/few.yaml', 'msgr/simple.yaml', 'openstack.yaml}', 'rados/verify/{rados.yaml', 'tasks/rados_api_tests.yaml', 'validater/valgrind.yaml}’]

There may be 2 problems. First why did set_pg_num() fail. Because of the failure we went to RadosTestNS::TearDown(). The set_snap_read() does ldout() that calls should_gather() which hit the assert.

2016-10-25T18:58:44.892 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: /srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILD/ceph-11.0.2/src/test/librados/list.cc:689: Failure
2016-10-25T18:58:44.892 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: Value of: err_str.empty()
2016-10-25T18:58:44.892 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: Actual: false
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: Expected: true
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: /srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILD/ceph-11.0.2/src/log/SubsystemMap.h: In function 'bool ceph::log::SubsystemMap::should_gather(unsigned int, int)' thread 7fc8b4da1300 time 2016-10-25 18:58:44.875361
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: /srv/autobuild-ceph/gitbuilder.git/build/rpmbuild/BUILD/ceph-11.0.2/src/log/SubsystemMap.h: 62: FAILED assert(sub < m_subsys.size())
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: ceph version v11.0.2-682-gbbdfd13 (bbdfd1375e8d2d0b8db59fd805aeb2a5c8ea2563)
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x7fc8b50d8ae5]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 2: (()+0x1c08ac) [0x7fc8b4f808ac]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 3: (librados::IoCtxImpl::set_snap_read(snapid_t)+0x2ee) [0x7fc8b4fdae4e]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 4: (rados_ioctx_snap_set_read()+0x26) [0x7fc8b4fadb66]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 5: (RadosTestNS::cleanup_all_objects(void*)+0x33) [0x7fc8b5047e83]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 6: (RadosTestNS::TearDown()+0xd) [0x7fc8b504832d]
2016-10-25T18:58:44.893 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 7: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x7fc8b5036e43]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 8: (testing::Test::Run()+0x80) [0x7fc8b5029870]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 9: (testing::TestInfo::Run()+0x9e) [0x7fc8b502994e]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 10: (testing::TestCase::Run()+0xa5) [0x7fc8b5029a55]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 11: (testing::internal::UnitTestImpl::RunAllTests()+0x278) [0x7fc8b5029d38]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 12: (testing::UnitTest::Run()+0x68) [0x7fc8b502a008]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 13: (main()+0x156) [0x7fc8b4f84456]
2016-10-25T18:58:44.894 INFO:tasks.workunit.client.0.smithi092.stdout: api_list: 14: (__libc_start_main()+0xf5) [0x7fc8b159bb15]

Actions

Copy link

Updated by Josh Durgin almost 7 years ago

Status changed from New to Can't reproduce

Actions

Copy link

Updated by Sage Weil almost 7 years ago

Status changed from Can't reproduce to 12

2017-07-01T09:22:54.111 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:    -13> 2017-07-01 08:20:39.497294 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command dump_mempools hook 0x7fcfb7d4b538
2017-07-01T09:22:54.115 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:    -12> 2017-07-01 08:20:39.501615 7fcfb6c53a00 -1 WARNING: all dangerous and experimental features are enabled.
2017-07-01T09:22:54.119 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:    -11> 2017-07-01 08:20:39.501898 7fcfb6c53a00 -1 WARNING: all dangerous and experimental features are enabled.
2017-07-01T09:22:54.122 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:    -10> 2017-07-01 08:20:39.533270 7fcfb6c53a00 -1 WARNING: all dangerous and experimental features are enabled.
2017-07-01T09:22:54.124 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -9> 2017-07-01 08:20:39.533282 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) init /var/run/ceph/ceph-client.admin.31875.asok
2017-07-01T09:22:54.126 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -8> 2017-07-01 08:20:39.533295 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) bind_and_listen /var/run/ceph/ceph-client.admin.31875.asok
2017-07-01T09:22:54.129 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -7> 2017-07-01 08:20:39.533409 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command 0 hook 0x7fcfb7d9cac0
2017-07-01T09:22:54.132 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -6> 2017-07-01 08:20:39.533427 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command version hook 0x7fcfb7d9cac0
2017-07-01T09:22:54.138 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -5> 2017-07-01 08:20:39.533435 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command git_version hook 0x7fcfb7d9cac0
2017-07-01T09:22:54.141 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -4> 2017-07-01 08:20:39.533442 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command help hook 0x7fcfb7d510b0
2017-07-01T09:22:54.145 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -3> 2017-07-01 08:20:39.533449 7fcfb6c53a00  5 asok(0x7fcfb7d142f0) register_command get_command_descriptions hook 0x7fcfb7d9db10
2017-07-01T09:22:54.149 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -2> 2017-07-01 08:20:39.540514 7fcfa8d7b700  5 asok(0x7fcfb7d142f0) entry start
2017-07-01T09:22:54.152 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:     -1> 2017-07-01 08:20:39.541396 7fcfa8d7b700  5 asok(0x7fcfb7d142f0) AdminSocket: request '0' '' to 0x7fcfb7d9cac0 returned 1 bytes
2017-07-01T09:22:54.155 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:      0> 2017-07-01 09:22:53.648405 7fcfb6c53a00 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.3-2476-gd117
81c/rpm/el7/BUILD/ceph-12.0.3-2476-gd11781c/src/log/SubsystemMap.h: In function 'bool ceph::logging::SubsystemMap::should_gather(unsigned int, int)' thread 7fcfb6c53a00 time 2017-07-01 09:22:53.645781
2017-07-01T09:22:54.158 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.3-2476-gd11781c/rpm/el7/BUILD/ceph-12.0.3-2476-gd11781c/src/log
/SubsystemMap.h: 62: FAILED assert(sub < m_subsys.size())
2017-07-01T09:22:54.161 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:
2017-07-01T09:22:54.164 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  ceph version 12.0.3-2476-gd11781c (d11781c78570c982c0e20118cad7769056cc5c35) luminous (rc)
2017-07-01T09:22:54.166 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7fcfadcd0d10]
2017-07-01T09:22:54.169 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  2: (()+0x8bdd2) [0x7fcfb6cfddd2]
2017-07-01T09:22:54.172 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  3: (librados::IoCtxImpl::set_snap_read(snapid_t)+0x275) [0x7fcfb6d63545]
2017-07-01T09:22:54.177 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  4: (rados_ioctx_snap_set_read()+0x26) [0x7fcfb6d325d6]
2017-07-01T09:22:54.180 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  5: (RadosTestNS::cleanup_all_objects(void*)+0x33) [0x7fcfb6dcbf43]
2017-07-01T09:22:54.183 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  6: (RadosTestNS::TearDown()+0x13) [0x7fcfb6dcc433]
2017-07-01T09:22:54.189 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  7: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x7fcfb6dc1513]
2017-07-01T09:22:54.193 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  8: (testing::Test::Run()+0x80) [0x7fcfb6db3f40]
2017-07-01T09:22:54.199 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  9: (testing::TestInfo::Run()+0x9e) [0x7fcfb6db401e]
2017-07-01T09:22:54.208 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  10: (testing::TestCase::Run()+0xa5) [0x7fcfb6db4125]
2017-07-01T09:22:54.211 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  11: (testing::internal::UnitTestImpl::RunAllTests()+0x278) [0x7fcfb6db4408]
2017-07-01T09:22:54.214 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  12: (testing::UnitTest::Run()+0x68) [0x7fcfb6db46d8]
2017-07-01T09:22:54.217 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  13: (main()+0x11e) [0x7fcfb6cffcce]
2017-07-01T09:22:54.223 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  14: (__libc_start_main()+0xf5) [0x7fcfab3d0b35]
2017-07-01T09:22:54.228 INFO:tasks.workunit.client.0.smithi149.stdout:                 api_list:  15: (()+0x8f831) [0x7fcfb6d01831]

/a/sage-2017-07-01_06:55:28-rados-wip-sage-testing2-distro-basic-smithi/1351326

Actions

Copy link