Project

General

Profile

Actions
Copy link

Bug #46299

closed

Trying to pull docker.io/prom/prometheus:v2.18.1: too many request to registry

Added by Sebastian Wagner almost 4 years ago. Updated 11 months ago.

Status:
Closed
Priority:
High
Assignee:
David Galloway
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

https://pulpito.ceph.com/swagner-2020-07-01_09:26:21-rados:cephadm-wip-swagner-testing-2020-07-01-0956-distro-basic-smithi/5194228/

2020-07-01T10:04:28.253 INFO:tasks.cephadm:Adding local image mirror vossi04.front.sepia.ceph.com:5000
2020-07-01T10:04:28.301 DEBUG:teuthology.orchestra.remote:smithi189:/etc/containers/registries.conf is 4KB
2020-07-01T10:04:28.340 INFO:teuthology.orchestra.run.smithi189:> sudo sh -c 'cat > /etc/containers/registries.conf'
2020-07-01T10:04:28.400 DEBUG:teuthology.orchestra.remote:smithi205:/etc/containers/registries.conf is 4KB
2020-07-01T10:04:28.447 INFO:teuthology.orchestra.run.smithi205:> sudo sh -c 'cat > /etc/containers/registries.conf'

...

2020-07-01T10:14:26.759 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: debug 2020-07-01T10:14:26.700+0000 7f9405ffb700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:INFO:cephadm:Deploy daemon prometheus.a ...
2020-07-01T10:14:26.759 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:Verifying port 9095 ...
2020-07-01T10:14:26.760 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:Non-zero exit code 125 from /bin/podman run --rm --net=host --ipc=host -e CONTAINER_IMAGE=prom/prometheus:v2.18.1 -e NODE_NAME=smithi205 --entrypoint stat prom/prometheus:v2.18.1 -c %u %g /etc/prometheus
2020-07-01T10:14:26.760 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr Trying to pull registry.access.redhat.com/prom/prometheus:v2.18.1...
2020-07-01T10:14:26.760 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr   name unknown: Repo not found
2020-07-01T10:14:26.760 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr Trying to pull registry.fedoraproject.org/prom/prometheus:v2.18.1...
2020-07-01T10:14:26.760 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr   manifest unknown: manifest unknown
2020-07-01T10:14:26.761 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr Trying to pull registry.centos.org/prom/prometheus:v2.18.1...
2020-07-01T10:14:26.761 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr   manifest unknown: manifest unknown
2020-07-01T10:14:26.761 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr Trying to pull docker.io/prom/prometheus:v2.18.1...
2020-07-01T10:14:26.761 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr time="2020-07-01T10:10:18Z" level=error msg="HEADER map[Cache-Control:[no-cache] Content-Type:[application/json] Retry-After:[60]]" 
2020-07-01T10:14:26.761 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr time="2020-07-01T10:11:20Z" level=error msg="HEADER map[Cache-Control:[no-cache] Content-Type:[application/json] Retry-After:[60]]" 
2020-07-01T10:14:26.762 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr time="2020-07-01T10:12:22Z" level=error msg="HEADER map[Cache-Control:[no-cache] Content-Type:[application/json] Retry-After:[60]]" 
2020-07-01T10:14:26.763 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr time="2020-07-01T10:13:24Z" level=error msg="HEADER map[Cache-Control:[no-cache] Content-Type:[application/json] Retry-After:[60]]" 
2020-07-01T10:14:26.763 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr   too many request to registry
2020-07-01T10:14:26.763 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr Error: unable to pull prom/prometheus:v2.18.1: 4 errors occurred:
2020-07-01T10:14:26.763 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr         * Error initializing source docker://registry.access.redhat.com/prom/prometheus:v2.18.1: Error reading manifest v2.18.1 in registry.access.redhat.com/prom/prometheus: name unknown: Repo not found
2020-07-01T10:14:26.763 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr         * Error initializing source docker://registry.fedoraproject.org/prom/prometheus:v2.18.1: Error reading manifest v2.18.1 in registry.fedoraproject.org/prom/prometheus: manifest unknown: manifest unknown
2020-07-01T10:14:26.764 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr         * Error initializing source docker://registry.centos.org/prom/prometheus:v2.18.1: Error reading manifest v2.18.1 in registry.centos.org/prom/prometheus: manifest unknown: manifest unknown
2020-07-01T10:14:26.764 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr         * Error parsing image configuration: too many request to registry
2020-07-01T10:14:26.764 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: INFO:cephadm:stat:stderr
2020-07-01T10:14:26.764 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: Traceback (most recent call last):
2020-07-01T10:14:26.764 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 4847, in <module>
2020-07-01T10:14:26.765 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 1187, in _default_image
2020-07-01T10:14:26.765 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 2886, in command_deploy
2020-07-01T10:14:26.765 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 2818, in extract_uid_gid_monitoring
2020-07-01T10:14:26.765 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 1803, in extract_uid_gid
2020-07-01T10:14:26.766 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 2280, in run
2020-07-01T10:14:26.766 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]:   File "<stdin>", line 866, in call_throws
2020-07-01T10:14:26.766 INFO:journalctl@ceph.mgr.x.smithi205.stdout:Jul 01 10:14:26 smithi205 bash[31753]: RuntimeError: Failed command: /bin/podman run --rm --net=host --ipc=host -e CONTAINER_IMAGE=prom/prometheus:v2.18.1 -e NODE_NAME=smithi205 --entrypoint stat prom/prometheus:v2.18.1 -c %u %g /etc/prometheus

Changing the registry mirror to `docker-mirror.front.sepia.ceph.com:5000` should work. I just can't do that myself.

Actions
Copy link
 #1

Updated by Sebastian Wagner almost 4 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Sebastian Wagner almost 4 years ago

as a workaround, I restarted the registry on vossi04 again.

Actions
Copy link
 #3

Updated by Sebastian Wagner almost 4 years ago

  • Priority changed from Normal to High
Actions
Copy link
 #4

Updated by Sebastian Wagner almost 4 years ago

I've restarted the vossi04 regstry again.

Actions
Copy link
 #5

Updated by David Galloway almost 4 years ago

  • Status changed from New to Need More Info
  • Assignee set to David Galloway

I thought docker-mirror.front.sepia.ceph.com:5000 was the resolution for this.

Actions
Copy link
 #6

Updated by Deepika Upadhyay over 3 years ago

http://qa-proxy.ceph.com/teuthology/yuriw-2020-09-16_23:57:37-rados-wip-yuri8-testing-2020-09-16-2220-octopus-distro-basic-smithi/5441329/teuthology.log
/a/yuriw-2020-09-16_23:57:37-rados-wip-yuri8-testing-2020-09-16-2220-octopus-distro-basic-smithi/5441343/teuthology.log

Error: unable to pull docker.io/ceph/daemon-base:latest-nautilus: unable to pull image: Error parsing image configuration: too many request to registry

Actions
Copy link
 #7

Updated by Laura Flores 11 months ago

  • Status changed from Need More Info to Closed

We don't use docker hub anymore in our CI to host images. This can be closed.

Actions
Copy link

Also available in: Atom PDF