Project

General

Profile

Actions

Bug #65236

open

upgrade/cephfs/mds_upgrade_sequence: cephadm refresh fails; podman times out

Added by Laura Flores about 1 month ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119

2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:HEALTH_WARN failed to probe daemons or devices
2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices
2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:    host smithi152 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/mon.smithi152/config
2024-03-29T05:10:54.537 INFO:teuthology.orchestra.run.smithi142.stdout:/usr/bin/podman: timeout after 895 seconds
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 -e NODE_NAME=smithi152 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/run/ceph:z -v /var/log/ceph/9f96302c-ed85-11ee-95d0-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpw7p6g95h:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a6b710b62a1696018e5d2e3dd0dc6f55c1b3bad43842a897f9227eec10772f28 inventory --format=json-pretty --filter-for-batch
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:Traceback (most recent call last):
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:    return _run_code(code, main_globals, None,
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:    exec(code, run_globals)
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 5528, in <module>
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 5516, in main
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 406, in _infer_config
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 321, in _infer_fsid
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 434, in _infer_image
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 308, in _validate_fsid
2024-03-29T05:10:54.538 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/__main__.py", line 3303, in command_ceph_volume
2024-03-29T05:10:54.539 INFO:teuthology.orchestra.run.smithi142.stdout:  File "/var/lib/ceph/9f96302c-ed85-11ee-95d0-87774f69a715/cephadm.1f56a973b95b087e9844a535e7ba89f3cd4b929bf241ebfb24a12607b44365f4/cephadmlib/call_wrappers.py", line 310, in call_throws


Related issues 1 (1 open0 closed)

Related to Orchestrator - Bug #65233: upgrade/cephfs/mds_upgrade_sequence: 'ceph orch ps' command times outNew

Actions
Actions #1

Updated by Laura Flores about 1 month ago

  • Backport set to squid
Actions #2

Updated by Adam King about 1 month ago

  • Related to Bug #65233: upgrade/cephfs/mds_upgrade_sequence: 'ceph orch ps' command times out added
Actions

Also available in: Atom PDF