Project

General

Profile

Actions
Copy link

Bug #63894

closed

qa: cephadm failed with an error code 1, alertmanager container not found.

Added by Kotresh Hiremath Ravishankar 5 months ago. Updated 3 months ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Seen in quincy run:

Job: https://pulpito.ceph.com/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501828

Log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-12-26_19:48:51-fs-wip-yuri-testing-2023-12-26-0957-quincy-distro-default-smithi/7501828/teuthology.log

<prev>
2023-12-26T22:01:55.771 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 761f1ff0-a439-11ee-95a5-87774f69a715 - ceph mon dump -f json
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: cephadm 2023-12-26T22:01:54.120883+0000 mgr.smithi059.gwkiyc (mgr.14156) 157 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-761f1ff0-a439-11ee-95a5-87774f69a715-alertmanager.smithi059
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: /bin/podman: stderr Error: no such container ceph-761f1ff0-a439-11ee-95a5-87774f69a715-alertmanager.smithi059
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Deploy daemon alertmanager.smithi059 ...
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi059 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0...
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:57:32Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:58:10Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.296 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:58:47Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi059 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0...
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:00:01Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:00:38Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:01:16Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Traceback (most recent call last):
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8230, in <module>
2023-12-26T22:01:56.297 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: main()
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 8218, in main
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: r = ctx.func(ctx)
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 1759, in _default_image
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: return func(ctx)
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4344, in command_deploy
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: uid, gid = extract_uid_gid_monitoring(ctx, daemon_type)
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 4275, in extract_uid_gid_monitoring
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: uid, gid = extract_uid_gid(ctx, file_path=['/etc/alertmanager', '/etc/prometheus'])
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/var/lib/ceph/761f1ff0-a439-11ee-95a5-87774f69a715/cephadm.d4237e4639c108308fe13147b1c08af93c3d5724d9ff21ae797eb4b78fea3931", line 2548, in extract_uid_gid
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: raise RuntimeError('uid/gid not found')
2023-12-26T22:01:56.298 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: RuntimeError: uid/gid not found
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Traceback (most recent call last):
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1347, in _remote_connection
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: yield (conn, connr)
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1244, in _run_cephadm
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: code, '\n'.join(err)))
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Non-zero exit code 125 from /bin/podman container inspect --format {{.State.Status}} ceph-761f1ff0-a439-11ee-95a5-87774f69a715-alertmanager.smithi059
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: /bin/podman: stderr Error: no such container ceph-761f1ff0-a439-11ee-95a5-87774f69a715-alertmanager.smithi059
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Deploy daemon alertmanager.smithi059 ...
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi059 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/alertmanager
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0...
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:57:32Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.299 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:58:10Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T21:58:47Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: Non-zero exit code 125 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/prom/alertmanager:v0.20.0 -e NODE_NAME=smithi059 -e CEPH_USE_RANDOM_NONCE=1 docker.io/prom/alertmanager:v0.20.0 -c %u %g /etc/prometheus
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Trying to pull docker.io/prom/alertmanager:v0.20.0...
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:00:01Z" level=warning msg="Failed, retrying in 1s ... (1/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:00:38Z" level=warning msg="Failed, retrying in 1s ... (2/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr time="2023-12-26T22:01:16Z" level=warning msg="Failed, retrying in 1s ... (3/3). Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get \"https://docker-mirror.front.sepia.ceph.com:5000/v2/\": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"
2023-12-26T22:01:56.300 INFO:journalctl@ceph.mon.smithi059.smithi059.stdout:Dec 26 22:01:55 smithi059 ceph-761f1ff0-a439-11ee-95a5-87774f69a715-mon-smithi059111931: stat: stderr Error: initializing source docker://prom/alertmanager:v0.20.0: (Mirrors also failed: [docker-mirror.front.sepia.ceph.com:5000/prom/alertmanager:v0.20.0: pinging container registry docker-mirror.front.sepia.ceph.com:5000: Get "https://docker-mirror.front.sepia.ceph.com:5000/v2/": dial tcp 172.21.0.79:5000: connect: no route to host]): docker.io/prom/alertmanager:v0.20.0: reading manifest v0.20.0 in docker.io/prom/alertmanager: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Related issues 1 (1 open — 0 closed)

Is duplicate of Orchestrator - Bug #58658: Error: initializing source docker://prom/alertmanager:v0.20.0New

Actions
Actions
Copy link
 #2

Updated by Nitzan Mordechai 5 months ago

/a/yuriw-2023-12-26_16:04:49-rados-wip-yuri5-testing-2023-12-15-0747-pacific-distro-default-smithi/7501359

Actions
Copy link
 #3

Updated by Laura Flores 4 months ago

  • Related to Bug #58658: Error: initializing source docker://prom/alertmanager:v0.20.0 added
Actions
Copy link
 #4

Updated by Nitzan Mordechai 4 months ago

/a/yuriw-2024-01-10_19:18:18-rados-wip-yuri3-testing-2024-01-10-0735-distro-default-smithi/7512326

Actions
Copy link
 #5

Updated by Laura Flores 4 months ago

/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532443
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532465
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532542
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532593
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532323
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532397
/a/yuriw-2024-01-25_17:23:47-rados-pacific-release-distro-default-smithi/7532327

Actions
Copy link
 #6

Updated by Matan Breizman 3 months ago

/a/yuriw-2024-02-11_15:26:36-rados-wip-yuri8-testing-2024-02-07-0820-pacific-distro-default-smithi/7556218
/a/yuriw-2024-02-11_15:26:36-rados-wip-yuri8-testing-2024-02-07-0820-pacific-distro-default-smithi/7556226
/a/yuriw-2024-02-11_15:26:36-rados-wip-yuri8-testing-2024-02-07-0820-pacific-distro-default-smithi/7556242
...

Actions
Copy link
 #7

Updated by Laura Flores 3 months ago

  • Status changed from New to Duplicate
Actions
Copy link
 #8

Updated by Laura Flores 3 months ago

  • Related to deleted (Bug #58658: Error: initializing source docker://prom/alertmanager:v0.20.0)
Actions
Copy link
 #9

Updated by Laura Flores 3 months ago

  • Is duplicate of Bug #58658: Error: initializing source docker://prom/alertmanager:v0.20.0 added
Actions
Copy link

Also available in: Atom PDF