Project

General

Profile

Actions

Bug #58173

closed

api_aio_pp: failure on LibRadosAio.SimplePoolEIOFlag and LibRadosAio.PoolEIOFlag

Added by Adam King over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Urgent
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The workunits/rados/test.sh script is run in the orch suite on some tests. In a few of them, these two tests were failing.
The run did include some unmerged PRs but they all only touched code within mgr modules so considering it unlikely they caused the failures.
Additionally, it was inconsistent whether one, the other, or both tests referenced in the tracker title failed.

https://pulpito.ceph.com/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101924/

2022-12-03T02:55:13.492 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.SimplePoolEIOFlag
2022-12-03T02:55:13.493 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: [       OK ] LibRadosAio.SimplePoolEIOFlag (3986 ms)
2022-12-03T02:55:13.493 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.PoolEIOFlag
2022-12-03T02:55:13.493 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: setting pool EIO
2022-12-03T02:55:13.494 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: max_success 100, min_failed 0
2022-12-03T02:55:13.494 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-1172-g20d17790/rpm/el8/BUILD/ceph-18.0.0-1172-g20d17790/src/test/librados/aio_cxx.cc:2376: Failure
2022-12-03T02:55:13.494 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: Value of: max_success + 1 == min_failed
2022-12-03T02:55:13.495 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp:   Actual: false
2022-12-03T02:55:13.495 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: Expected: true
2022-12-03T02:55:13.495 INFO:tasks.workunit.client.0.smithi120.stdout:               api_aio_pp: [  FAILED  ] LibRadosAio.PoolEIOFlag (4017 ms)

https://pulpito.ceph.com/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101936/

2022-12-03T03:12:42.349 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.SimplePoolEIOFlag
2022-12-03T03:12:42.350 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: /build/ceph-18.0.0-1172-g20d17790/src/test/librados/aio_cxx.cc:2317: Failure
2022-12-03T03:12:42.350 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: Expected equality of these values:
2022-12-03T03:12:42.350 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp:   -5
2022-12-03T03:12:42.350 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp:   my_completion->get_return_value()
2022-12-03T03:12:42.351 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp:     Which is: 0
2022-12-03T03:12:42.351 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: [  FAILED  ] LibRadosAio.SimplePoolEIOFlag (3574 ms)
2022-12-03T03:12:42.351 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.PoolEIOFlag
2022-12-03T03:12:42.351 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: setting pool EIO
2022-12-03T03:12:42.352 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: waiting for 99,100
2022-12-03T03:12:42.352 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: max_success 100, min_failed 0
2022-12-03T03:12:42.352 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: /build/ceph-18.0.0-1172-g20d17790/src/test/librados/aio_cxx.cc:2376: Failure
2022-12-03T03:12:42.352 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: Value of: max_success + 1 == min_failed
2022-12-03T03:12:42.353 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp:   Actual: false
2022-12-03T03:12:42.353 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: Expected: true
2022-12-03T03:12:42.353 INFO:tasks.workunit.client.0.smithi090.stdout:               api_aio_pp: [  FAILED  ] LibRadosAio.PoolEIOFlag (5013 ms)

https://pulpito.ceph.com/adking-2022-12-02_21:56:44-orch:cephadm-wip-adk-testing-2022-12-02-0932-distro-default-smithi/7101923/

2022-12-03T02:56:59.081 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.SimplePoolEIOFlag
2022-12-03T02:56:59.082 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-1172-g20d17790/rpm/el8/BUILD/ceph-18.0.0-1172-g20d17790/src/test/librados/aio_cxx.cc:2317: Failure
2022-12-03T02:56:59.082 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: Expected equality of these values:
2022-12-03T02:56:59.082 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp:   -5
2022-12-03T02:56:59.082 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp:   my_completion->get_return_value()
2022-12-03T02:56:59.083 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp:     Which is: 0
2022-12-03T02:56:59.083 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: [  FAILED  ] LibRadosAio.SimplePoolEIOFlag (3563 ms)
2022-12-03T02:56:59.083 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: [ RUN      ] LibRadosAio.PoolEIOFlag
2022-12-03T02:56:59.083 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: setting pool EIO
2022-12-03T02:56:59.084 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: max_success 95, min_failed 96
2022-12-03T02:56:59.084 INFO:tasks.workunit.client.0.smithi047.stdout:               api_aio_pp: [       OK ] LibRadosAio.PoolEIOFlag (4018 ms)


Related issues 1 (0 open1 closed)

Has duplicate crimson - Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlagDuplicateSamuel Just

Actions
Actions #1

Updated by Adam King over 1 year ago

  • Project changed from Ceph to RADOS
Actions #2

Updated by Neha Ojha over 1 year ago

  • Priority changed from Normal to Urgent
Actions #3

Updated by Samuel Just over 1 year ago

  • Related to Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag added
Actions #4

Updated by Samuel Just over 1 year ago

  • Related to deleted (Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag)
Actions #5

Updated by Samuel Just over 1 year ago

  • Has duplicate Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag added
Actions #6

Updated by Samuel Just over 1 year ago

Matan added that test within the last two weeks: https://github.com/ceph/ceph/pull/49029

Actions #7

Updated by Laura Flores over 1 year ago

  • Related to Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag added
Actions #8

Updated by Laura Flores over 1 year ago

  • Related to deleted (Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag)
Actions #9

Updated by Laura Flores over 1 year ago

Excuse my update Sam, I see you already added it as a duplicate.

Actions #10

Updated by Laura Flores over 1 year ago

Building a branch here with https://github.com/ceph/ceph/pull/49029 reverted, which can be used to verify whether it is breaking tests.

https://shaman.ceph.com/builds/ceph/wip-pr-49029-reverted/6c5f7b3b7baab53895e2a3b6f8b6802851cc5cc3/

Actions #11

Updated by Matan Breizman over 1 year ago

  • Status changed from New to Fix Under Review
  • Assignee set to Matan Breizman
  • Pull request ID set to 49278
Actions #12

Updated by Laura Flores over 1 year ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF