Project

General

Profile

Actions

Bug #64343

open

Expected warnings that need to be whitelisted cause rados/cephadm tests to fail

Added by Laura Flores 3 months ago. Updated about 1 month ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
backport_processed
Backport:
pacific,quincy,reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Follows the merge of https://github.com/ceph/ceph/pull/41479 and https://github.com/ceph/ceph/pull/54312, there are some warnings that we expect to see in testing environments that are causing some tests to fail. They need to be whitelisted.

See https://pulpito.ceph.com/yuriw-2024-02-06_00:23:51-rados-wip-yuri10-testing-2024-02-02-1149-pacific-distro-default-smithi/7548075/ for examples.

Warnings include:
- MON_DOWN
- PG_AVAILABILITY
- OSD_DOWN


Related issues 4 (3 open1 closed)

Related to Dashboard - Bug #64344: rados/cephadm/dashboard: test that expects a HOST_MAINTENANCE_MODE scenario fails due to warning in cluster logNew

Actions
Copied to RADOS - Backport #64407: pacific: Expected warnings that need to be whitelisted cause rados/cephadm tests to failRejectedActions
Copied to RADOS - Backport #64408: quincy: Expected warnings that need to be whitelisted cause rados/cephadm tests to failNewActions
Copied to RADOS - Backport #64409: reef: Expected warnings that need to be whitelisted cause rados/cephadm tests to failNewActions
Actions #1

Updated by Laura Flores 3 months ago

  • Related to Bug #64344: rados/cephadm/dashboard: test that expects a HOST_MAINTENANCE_MODE scenario fails due to warning in cluster log added
Actions #3

Updated by Laura Flores 3 months ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 55498
Actions #4

Updated by Laura Flores 3 months ago

An alternative to reverting: https://github.com/ceph/ceph/pull/55507

Actions #5

Updated by Mark Nelson 3 months ago

I grepped through the example suite 'yuriw-2024-02-07_00:12:44-rados-wip-yuri2-testing-2024-02-06-1154-distro-default-smithi':

find . -name "teuthology.log" -exec grep -l -H 'FAIL' {} \; | xargs grep -l -H 'MON_DOWN' | wc -l
35

Those should be covered hopefully by #55507. Then I took the list of failures that did not have MON_DOWN in them and looked for the other two keywords:

cat ~/whitelist-testing/other-failures.txt | xargs grep -l -H 'OSD_DOWN'
./7549140/teuthology.log
./7549184/teuthology.log
./7549497/teuthology.log
./7549527/teuthology.log
./7549106/teuthology.log
./7549481/teuthology.log
./7549519/teuthology.log
./7549369/teuthology.log
./7549611/teuthology.log
./7549330/teuthology.log
./7549381/teuthology.log
./7549284/teuthology.log
./7549427/teuthology.log
./7549665/teuthology.log
./7549588/teuthology.log
./7549625/teuthology.log

and

cat ~/whitelist-testing/other-failures.txt | xargs grep -l -H 'PG_AVAILABILITY'
./7549519/teuthology.log
./7549164/teuthology.log

I think Laura has already looked through these, but I thought it might be helpful to have a record of potential other failure conditions.

Actions #6

Updated by Laura Flores 3 months ago

  • Backport set to pacific,quincy,reef
Actions #7

Updated by Radoslaw Zarzynski 3 months ago

  • Pull request ID changed from 55498 to 55507
Actions #8

Updated by Radoslaw Zarzynski 3 months ago

  • Status changed from Fix Under Review to Pending Backport
Actions #9

Updated by Backport Bot 3 months ago

  • Copied to Backport #64407: pacific: Expected warnings that need to be whitelisted cause rados/cephadm tests to fail added
Actions #10

Updated by Backport Bot 3 months ago

  • Copied to Backport #64408: quincy: Expected warnings that need to be whitelisted cause rados/cephadm tests to fail added
Actions #11

Updated by Backport Bot 3 months ago

  • Copied to Backport #64409: reef: Expected warnings that need to be whitelisted cause rados/cephadm tests to fail added
Actions #12

Updated by Backport Bot 3 months ago

  • Tags set to backport_processed
Actions #13

Updated by Konstantin Shalygin about 1 month ago

  • Assignee set to Laura Flores
  • Source set to Development
Actions

Also available in: Atom PDF