Project

General

Profile

Actions

Bug #46066

closed

502 (Bad Gateway) when pulling from quay.ceph.io/ceph-ci/ceph

Added by Kefu Chai almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Category:
Infrastructure Service
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

2020-06-18T07:04:32.870 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:Running command: /sbin/ip route ls
2020-06-18T07:04:32.870 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/sbin/ip:stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 102
2020-06-18T07:04:32.871 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/sbin/ip:stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.183 metric 102
2020-06-18T07:04:32.871 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0007503032684326172, stop=False, exit=0, reads=[4]
2020-06-18T07:04:32.871 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/sbin/ip:profile rt=0.0008108615875244141, stop=True, exit=0, reads=[4]
2020-06-18T07:04:32.871 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:Mon IP 172.21.15.183 is in CIDR network 172.21.0.0/20
2020-06-18T07:04:32.872 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:Pulling latest quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9 container...
2020-06-18T07:04:32.872 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:Running command: /bin/podman pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9
2020-06-18T07:04:33.651 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:stderr Trying to pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9...
2020-06-18T07:04:33.651 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=0.7787401676177979, stop=False, exit=None, reads=[6]
2020-06-18T07:04:33.768 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:stderr Getting image source signatures
2020-06-18T07:04:33.768 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=0.8956148624420166, stop=False, exit=None, reads=[6]
2020-06-18T07:04:34.008 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:stderr   invalid status code from registry 502 (Bad Gateway)
2020-06-18T07:04:34.008 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=1.1359565258026123, stop=False, exit=None, reads=[6]
2020-06-18T07:04:34.052 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:stderr Error: error pulling image "quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9": una
ble to pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9: unable to pull image: Error reading blob sha256:d71c5eab96b2ab4ce9461534cf5e4beda459d7dd3049e9ee2a4c73a13c0629b4: invalid st
atus code from registry 502 (Bad Gateway)
2020-06-18T07:04:34.052 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=1.1795868873596191, stop=False, exit=None, reads=[6]
2020-06-18T07:04:34.056 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=1.1834368705749512, stop=False, exit=125, reads=[4, 6]
2020-06-18T07:04:34.056 INFO:teuthology.orchestra.run.smithi183.stderr:DEBUG:cephadm:/bin/podman:profile rt=1.1836388111114502, stop=True, exit=125, reads=[4, 6]
2020-06-18T07:04:34.056 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:Non-zero exit code 125 from /bin/podman pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9
2020-06-18T07:04:34.057 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:/bin/podman:stderr Trying to pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9...
2020-06-18T07:04:34.057 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:/bin/podman:stderr Getting image source signatures
2020-06-18T07:04:34.057 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:/bin/podman:stderr   invalid status code from registry 502 (Bad Gateway)
2020-06-18T07:04:34.057 INFO:teuthology.orchestra.run.smithi183.stderr:INFO:cephadm:/bin/podman:stderr Error: error pulling image "quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9": unab
le to pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9: unable to pull image: Error reading blob sha256:d71c5eab96b2ab4ce9461534cf5e4beda459d7dd3049e9ee2a4c73a13c0629b4: invalid sta
tus code from registry 502 (Bad Gateway)
2020-06-18T07:04:34.058 INFO:teuthology.orchestra.run.smithi183.stderr:Traceback (most recent call last):
2020-06-18T07:04:34.058 INFO:teuthology.orchestra.run.smithi183.stderr:  File "/sbin/cephadm", line 4789, in <module>
2020-06-18T07:04:34.058 INFO:teuthology.orchestra.run.smithi183.stderr:    r = args.func()
2020-06-18T07:04:34.058 INFO:teuthology.orchestra.run.smithi183.stderr:  File "/sbin/cephadm", line 1170, in _default_image
2020-06-18T07:04:34.059 INFO:teuthology.orchestra.run.smithi183.stderr:    return func()
2020-06-18T07:04:34.059 INFO:teuthology.orchestra.run.smithi183.stderr:  File "/sbin/cephadm", line 2391, in command_bootstrap
2020-06-18T07:04:34.059 INFO:teuthology.orchestra.run.smithi183.stderr:    call_throws([container_path, 'pull', args.image])
2020-06-18T07:04:34.059 INFO:teuthology.orchestra.run.smithi183.stderr:  File "/sbin/cephadm", line 849, in call_throws
2020-06-18T07:04:34.059 INFO:teuthology.orchestra.run.smithi183.stderr:    raise RuntimeError('Failed command: %s' % ' '.join(command))
2020-06-18T07:04:34.060 INFO:teuthology.orchestra.run.smithi183.stderr:RuntimeError: Failed command: /bin/podman pull quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9

/a//kchai-2020-06-18_05:54:35-rados-wip-kefu-testing-2020-06-18-0846-distro-basic-smithi/5159557/

but the image was pushed to quay.ceph.io

+ docker push quay.ceph.io/ceph-ci/ceph:0a9c34cc8dbea56a321a6ea258a0652395f208f9
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Getting image source signatures
Copying blob sha256:aff9833d464191900b4a3b6e68aad912f930de639b411124e433aeb3757001cd
Copying blob sha256:1eb6ef00e50b2ea18c37524899a66b6e9badfbe0f13a6c99d1335b8ca9d07c33
Copying config sha256:313b89959e5d8ece93abf0eadd46deaf742f63fa557499423cb9d0074e970e8c
Writing manifest to image destination
Copying config sha256:313b89959e5d8ece93abf0eadd46deaf742f63fa557499423cb9d0074e970e8c
Writing manifest to image destination
Writing manifest to image destination
Storing signatures

see https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/41935//consoleFull

and it can be found at quay.ceph.io.

see https://quay.ceph.io/repository/ceph-ci/ceph/manifest/sha256:c87e08c732f1cfa73d3d9e7bd204645b897830beda036db487c2b5f4ebd088d1

Actions

Also available in: Atom PDF