Project

General

Profile

Actions

Bug #57915

open

LibRadosWatchNotify.AioNotify - error callback ceph_assert(ref > 0)

Added by Nitzan Mordechai over 1 year ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a//nmordech-2022-10-23_05:26:13-rados:verify-wip-nm-51282-distro-default-smithi/7077932

2022-10-23T08:20:28.499 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify: [       OK ] LibRadosWatchNotify.AioWatchNotify2 (95329 ms)
2022-10-23T08:20:28.499 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify: [ RUN      ] LibRadosWatchNotify.AioNotify
2022-10-23T08:20:28.499 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-563-gbf5a2d02/rpm/el8/BUILD/ceph-18.0.0-563-gbf5a2d02/src/librados/AioCompletionImpl.h: In function 'void librados::AioCompletionImpl::put_unlock()' thread 7fed0d92f700 time 2022-10-23T08:20:28.464902+0000
2022-10-23T08:20:28.499 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-563-gbf5a2d02/rpm/el8/BUILD/ceph-18.0.0-563-gbf5a2d02/src/librados/AioCompletionImpl.h: 120: FAILED ceph_assert(ref > 0)
2022-10-23T08:20:28.500 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  ceph version 18.0.0-563-gbf5a2d02 (bf5a2d02701e3618fc3fa5d6f7127ac5ac5dd65a) reef (dev)
2022-10-23T08:20:28.500 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fed1783edc5]
2022-10-23T08:20:28.500 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  2: /usr/lib64/ceph/libceph-common.so.2(+0x2a6f8b) [0x7fed1783ef8b]
2022-10-23T08:20:28.501 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  3: rados_aio_release()
2022-10-23T08:20:28.501 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  4: (LibRadosWatchNotify::watch_notify2_test_errcb_aio_reconnect(void*, unsigned long, int)+0x2bd) [0x55627291a9bd]
2022-10-23T08:20:28.501 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  5: /lib64/librados.so.2(+0x138ed0) [0x7fed18773ed0]
2022-10-23T08:20:28.501 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  6: /lib64/librados.so.2(+0xd5995) [0x7fed18710995]
2022-10-23T08:20:28.501 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  7: /lib64/librados.so.2(+0xd612a) [0x7fed1871112a]
2022-10-23T08:20:28.502 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  8: /lib64/librados.so.2(+0xc4ab1) [0x7fed186ffab1]
2022-10-23T08:20:28.502 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  9: /lib64/librados.so.2(+0xdaa4c) [0x7fed18715a4c]
2022-10-23T08:20:28.502 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  10: /lib64/libstdc++.so.6(+0xc2ba3) [0x7fed1572eba3]
2022-10-23T08:20:28.502 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  11: /lib64/libpthread.so.0(+0x81ca) [0x7fed183e01ca]
2022-10-23T08:20:28.503 INFO:tasks.workunit.client.0.smithi071.stdout:         api_watch_notify:  12: clone()
2022-10-23T08:20:28.557 INFO:tasks.workunit.client.0.smithi071.stderr:bash: line 1: 117866 Aborted                 (core dumped) ceph_test_rados_api_watch_notify 2>&1
2022-10-23T08:20:28.557 INFO:tasks.workunit.client.0.smithi071.stderr:     117867 Done                    | tee ceph_test_rados_api_watch_notify.log
2022-10-23T08:20:28.557 INFO:tasks.workunit.client.0.smithi071.stderr:     117868 Done                    | sed "s/^/         api_watch_notify: /" 

Actions #1

Updated by Radoslaw Zarzynski over 1 year ago

Nitzan, I recall you mentioned about some watch-related tests on today's stand-up. Is this one of them?

Actions #2

Updated by Nitzan Mordechai over 1 year ago

Yes, this is one of the Notify bugs that i hit during my tests

Actions #3

Updated by Nitzan Mordechai 9 months ago

didn't reoccur yet

Actions

Also available in: Atom PDF