Project

General

Profile

Actions

Feature #45732

open

Unlock testnode on 'Could not find an image for' FOG error

Added by David Galloway almost 4 years ago. Updated almost 4 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:

Description

e.g., http://pulpito.ceph.com/pnawracay-2020-05-27_16:08:24-rados:dashboard-wip-pse-fix-root-volume-alert-not-firing-distro-basic-mira/5095629/

2020-05-27T16:08:45.750 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pse-fix-root-volume-alert-not-firing/qa; will attempt to use it
2020-05-27T16:08:45.752 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pse-fix-root-volume-alert-not-firing/qa/tasks
2020-05-27T16:08:45.752 INFO:teuthology.run_tasks:Running task internal.check_packages...
2020-05-27T16:08:45.754 INFO:teuthology.task.internal:Checking packages...
2020-05-27T16:08:45.777 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash '39d2bcad0b5d1731fcee366c57c545ceb82c657e'
2020-05-27T16:08:45.777 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch
2020-05-27T16:08:45.778 INFO:teuthology.packaging:ref: None
2020-05-27T16:08:45.778 INFO:teuthology.packaging:tag: None
2020-05-27T16:08:45.779 INFO:teuthology.packaging:branch: wip-pse-fix-root-volume-alert-not-firing
2020-05-27T16:08:45.779 INFO:teuthology.packaging:sha1: 39d2bcad0b5d1731fcee366c57c545ceb82c657e
2020-05-27T16:08:45.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-pse-fix-root-volume-alert-not-firing
2020-05-27T16:08:45.969 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-1914.g39d2bcad0b5
2020-05-27T16:08:45.969 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep...
2020-05-27T16:08:45.986 INFO:teuthology.task.internal:no buildpackages task found
2020-05-27T16:08:45.986 INFO:teuthology.run_tasks:Running task internal.lock_machines...
2020-05-27T16:08:46.001 INFO:teuthology.task.internal.lock_machines:Locking machines...
2020-05-27T16:08:46.002 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/
2020-05-27T16:08:46.054 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pnawracay@teuthology', 'count': 2, 'machine_type': 'mira', 'description': '/home/teuthworker/archive/pnawracay-2020-05-27_16:08:24-rados:dashboard-wip-pse-fix-root-volume-alert-not-firing-distro-basic-mira/5095629'}
2020-05-27T16:08:46.076 DEBUG:teuthology.lock.ops:locked ubuntu@mira045.front.sepia.ceph.com, ubuntu@mira109.front.sepia.ceph.com
2020-05-27T16:08:46.132 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x7f35e98a7978>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x7f35e7b23840>, signals=[15])
2020-05-27T16:08:46.270 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 94, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/lock_machines.py", line 78, in lock_machines
    os_version, arch)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/lock/ops.py", line 145, in lock_many
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 87, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 101, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 37, in resurrect_traceback
    reraise(*exc_info)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 24, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 78, in create
    self.set_image(host_id)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 168, in set_image
    image_data = self.get_image_data()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 160, in get_image_data
    (self.os_type, self.os_version))
RuntimeError: Could not find an image for centos 8.1
2020-05-27T16:08:46.294 ERROR:teuthology.run_tasks: Sentry event: http://sentry.ceph.com/sepia/teuthology/?q=b86022d8d7fe4465bda62352e2fdc144
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 94, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/lock_machines.py", line 78, in lock_machines
    os_version, arch)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/lock/ops.py", line 145, in lock_many
    reimaged[machine] = machines[machine]
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 87, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 101, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 37, in resurrect_traceback
    reraise(*exc_info)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/six.py", line 703, in reraise
    raise value
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 24, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/__init__.py", line 39, in reimage
    return obj.create()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 78, in create
    self.set_image(host_id)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 168, in set_image
    image_data = self.get_image_data()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/provision/fog.py", line 160, in get_image_data
    (self.os_type, self.os_version))
RuntimeError: Could not find an image for centos 8.1
2020-05-27T16:08:46.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.lock_machines
2020-05-27T16:08:46.358 INFO:teuthology.run:Summary data:
description: rados:dashboard/{clusters/{2-node-mgr.yaml} debug/mgr.yaml objectstore/bluestore-bitmap.yaml
  supported-random-distro$/{centos_8.yaml} tasks/dashboard.yaml}
failure_reason: Could not find an image for centos 8.1
owner: scheduled_pnawracay@teuthology
sentry_event: http://sentry.ceph.com/sepia/teuthology/?q=b86022d8d7fe4465bda62352e2fdc144
status: dead
success: false

There are no RHEL/CentOS 8 images for mira because those OSes don't support the storage hardware. There's no need to keep the testnode locked when this happens.

Actions #2

Updated by Brad Hubbard almost 4 years ago

I'll see if I can work out what's going on here.

Actions

Also available in: Atom PDF