Project

General

Profile

Bug #57935

all test jobs get stuck at "Running task ansible.cephlab..."

Added by Ilya Dryomov 3 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

https://pulpito.ceph.com/dis-2022-10-25_23:10:37-rbd-wip-dis-testing-distro-default-smithi/

2022-10-26T12:33:24.459 INFO:teuthology.run_tasks:Running task internal.timer...
2022-10-26T12:33:24.470 INFO:teuthology.task.internal:Starting timer...
2022-10-26T12:33:24.470 INFO:teuthology.run_tasks:Running task pcp...
2022-10-26T12:33:24.486 INFO:teuthology.run_tasks:Running task selinux...
2022-10-26T12:33:24.510 DEBUG:teuthology.task.selinux:Excluding smithi027: OS 'ubuntu' does not support SELinux
2022-10-26T12:33:24.511 DEBUG:teuthology.task.selinux:Excluding smithi190: OS 'ubuntu' does not support SELinux
2022-10-26T12:33:24.511 DEBUG:teuthology.task.selinux:Getting current SELinux state
2022-10-26T12:33:24.511 DEBUG:teuthology.task.selinux:Existing SELinux modes: {}
2022-10-26T12:33:24.511 INFO:teuthology.task.selinux:Putting SELinux into permissive mode
2022-10-26T12:33:24.512 INFO:teuthology.run_tasks:Running task ansible.cephlab...

History

#1 Updated by Dan Mick 3 months ago

It's conceivable this is related to the RHEV problems, currently still affecting quay.ceph,io, git.ceph.com, labdashboard.front, and nagios.front. Focusing on quay at the moment.

#2 Updated by Laura Flores 3 months ago

Note from Dan:

It's looking more and more like we're just going to have to rebuild these broken VMs from scratch. Luckily in many cases that won't involve complete data loss.

#3 Updated by Christian Rohmann about 2 months ago

Could you kindly give an update on the state of the restoration process?

Mostly I would love to know what the implications for testing of upcoming point releases (e.g. 16.2.11) still are.

#4 Updated by Laura Flores about 2 months ago

Hi Christian,

The lab was fixed a few weeks ago, and we are close to cutting the RC for v16.2.11. We expect to finish testing over the holidays and release at the beginning of January.

#5 Updated by Laura Flores about 2 months ago

  • Status changed from New to Resolved

Also available in: Atom PDF