Actions
Bug #44078
closedcentos 8.1: ceph-mon: assertion "z >= signedspan::zero()" failed in ceph::to_timespan(
Status:
Duplicate
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
{
"crash_id": "2020-02-11T00:59:33.440700Z_9fb04b6c-75f3-4c74-94b4-c4d5727a57b5",
"timestamp": "2020-02-11T00:59:33.440700Z",
"process_name": "ceph-mon",
"entity_name": "mon.a",
"ceph_version": "15.0.0-10072-gec05e8b",
"utsname_hostname": "smithi026",
"utsname_sysname": "Linux",
"utsname_release": "4.18.0-147.5.1.el8_1.x86_64",
"utsname_version": "#1 SMP Wed Feb 5 02:00:39 UTC 2020",
"utsname_machine": "x86_64",
"os_name": "CentOS Linux",
"os_id": "centos",
"os_version_id": "8",
"os_version": "8 (Core)",
"assert_condition": "z >= signedspan::zero()",
"assert_func": "ceph::time_detail::timespan ceph::to_timespan(ceph::time_detail::signedspan)",
"assert_file": "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/huge/release/15.0.0-10072-gec05e8b/rpm/el8/BUILD/ceph-15.0.0-10072-gec05e8b/src/common/ceph_time.h",
"assert_line": 485,
"assert_thread_name": "safe_timer",
"assert_msg": "/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/huge/release/15.0.0-10072-gec05e8b/rpm/el8/BUILD/ceph-15.0.0-10072-gec05e8b/src/common/ceph_time.h: In function 'ceph::time_detail::timespan ceph::to_timespan(ceph::time_detail::signedspan)' thread 7efcb2f5c700 time 2020-02-11T00:59:33.437560+0000\n/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/huge/release/15.0.0-10072-gec05e8b/rpm/el8/BUILD/ceph-15.0.0-10072-gec05e8b/src/common/ceph_time.h: 485: FAILED ceph_assert(z >= signedspan::zero())\n",
"backtrace": [
"(()+0x12dc0) [0x7efcbbb2edc0]",
"(gsignal()+0x10f) [0x7efcba8098df]",
"(abort()+0x127) [0x7efcba7f3cf5]",
"(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a7) [0x7efcbdde4485]",
"(()+0x27b64e) [0x7efcbdde464e]",
"(Paxos::begin(ceph::buffer::v14_2_0::list&)+0xddb) [0x5603abac840b]",
"(Paxos::propose_pending()+0x12b) [0x5603abac96eb]",
"(Paxos::trigger_propose()+0x339) [0x5603abacc8e9]",
"(PaxosService::propose_pending()+0x26e) [0x5603abad215e]",
"(()+0x38d4b9) [0x5603abad24b9]",
"(Context::complete(int)+0xd) [0x5603ab9dd2fd]",
"(SafeTimer::timer_thread()+0x1b7) [0x7efcbdebfec7]",
"(SafeTimerThread::entry()+0x11) [0x7efcbdec14a1]",
"(()+0x82de) [0x7efcbbb242de]",
"(clone()+0x43) [0x7efcba8ce133]"
]
}
http://pulpito.ceph.com/rzarzynski_bug43903_more_pgnum_changes_osdmapfix/4752857/
Updated by Radoslaw Zarzynski about 4 years ago
- Related to Bug #43365: Nautilus: Random mon crashes in failed assertion at ceph::time_detail::signedspan added
Updated by Sage Weil about 4 years ago
- Priority changed from Normal to High
I hit this on centos 8.1
-2> 2020-02-24T04:19:43.311+0000 7fc757e12700 10 mon.b@0(leader).paxos(paxos updating c 1507..2229) begin for 2230 33815 bytes
-1> 2020-02-24T04:19:43.312+0000 7fc757e12700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/huge/release/15.1.0-1143-gcc83a6f/rpm/el8/BUILD/ceph-15.1.0-1143-gcc83a6f/src/common/ceph_time.h: In function 'ceph::time_detail::timespan ceph::to_timespan(ceph::time_detail::signedspan)' thread 7fc757e12700 time 2020-02-24T04:19:43.311855+0000
/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/huge/release/15.1.0-1143-gcc83a6f/rpm/el8/BUILD/ceph-15.1.0-1143-gcc83a6f/src/common/ceph_time.h: 485: FAILED ceph_assert(z >= signedspan::zero())
ceph version 15.1.0-1143-gcc83a6f (cc83a6f3ce8c0775bad30a6ea9c366e1bdcb99aa) octopus (rc)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7fc762c95f42]
2: (()+0x27915c) [0x7fc762c9615c]
3: (Paxos::begin(ceph::buffer::v14_2_0::list&)+0xf8c) [0x5614864f0e8c]
4: (Paxos::propose_pending()+0x10f) [0x5614864f23ef]
5: (Paxos::trigger_propose()+0x339) [0x5614864f5849]
6: (PaxosService::propose_pending()+0x26e) [0x5614864fb08e]
7: (()+0x38c3e9) [0x5614864fb3e9]
8: (Context::complete(int)+0xd) [0x56148640751d]
9: (SafeTimer::timer_thread()+0x1b7) [0x7fc762d6fdd7]
10: (SafeTimerThread::entry()+0x11) [0x7fc762d713b1]
11: (()+0x82de) [0x7fc7609d82de]
12: (clone()+0x43) [0x7fc75f782133]
code is
auto start = ceph::coarse_mono_clock::now(); get_store()->apply_transaction(t); auto end = ceph::coarse_mono_clock::now(); logger->tinc(l_paxos_begin_latency, to_timespan(end - start));
kernel is 4.18.0-147.el8.x86_64
Updated by Sage Weil about 4 years ago
- Subject changed from ceph-mon: assertion "z >= signedspan::zero()" failed in ceph::to_timespan() to centos 8.1: ceph-mon: assertion "z >= signedspan::zero()" failed in ceph::to_timespan(
- Priority changed from High to Urgent
centos 8.1 again
/a/sage-2020-02-25_15:51:04-rados-wip-sage2-testing-2020-02-25-0704-distro-basic-smithi/4801785
Actions