Actions
Bug #46035
openReport the correct error when quay fails
Status:
New
Priority:
Normal
Assignee:
-
Category:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
2020-06-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi007.stderr:INFO:cephadm:Pulling latest quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd container... 2020-06-16T13:01:38.409 INFO:teuthology.orchestra.run.smithi007.stderr:DEBUG:cephadm:Running command: /usr/bin/docker pull quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd 2020-06-16T13:01:42.716 INFO:teuthology.orchestra.run.smithi007.stderr:DEBUG:cephadm:/usr/bin/docker:stderr Error response from daemon: manifest for quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd not found: manifest unknown: manifest unknown 2020-06-16T13:01:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:INFO:cephadm:Non-zero exit code 1 from /usr/bin/docker pull quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd 2020-06-16T13:01:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:INFO:cephadm:/usr/bin/docker:stderr Error response from daemon: manifest for quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd not found: manifest unknown: manifest unknown 2020-06-16T13:01:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:Traceback (most recent call last): 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: File "/home/ubuntu/cephtest/cephadm", line 4761, in <module> 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: r = args.func() 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: File "/home/ubuntu/cephtest/cephadm", line 1162, in _default_image 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: return func() 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: File "/home/ubuntu/cephtest/cephadm", line 2366, in command_bootstrap 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: call_throws([container_path, 'pull', args.image]) 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: File "/home/ubuntu/cephtest/cephadm", line 846, in call_throws 2020-06-16T13:01:42.724 INFO:teuthology.orchestra.run.smithi007.stderr: raise RuntimeError('Failed command: %s' % ' '.join(command)) 2020-06-16T13:01:42.725 INFO:teuthology.orchestra.run.smithi007.stderr:RuntimeError: Failed command: /usr/bin/docker pull quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd 2020-06-16T13:01:42.729 INFO:teuthology.orchestra.run.smithi007.stderr:DEBUG:cephadm:Releasing lock 140539053150448 on /run/cephadm/83387b56-afd1-11ea-a06d-001a4aab830c.lock 2020-06-16T13:01:42.729 INFO:teuthology.orchestra.run.smithi007.stderr:DEBUG:cephadm:Lock 140539053150448 released on /run/cephadm/83387b56-afd1-11ea-a06d-001a4aab830c.lock 2020-06-16T13:01:42.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-06-16T13:01:42.743 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2020-06-16T13:01:42.744 INFO:teuthology.orchestra.run.smithi007:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2020-06-16T13:01:42.749 INFO:teuthology.orchestra.run.smithi008:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2020-06-16T13:01:42.755 INFO:tasks.cephadm:Stopping all daemons... 2020-06-16T13:01:42.756 INFO:tasks.cephadm.mon.smithi007:Stopping mon.smithi007... 2020-06-16T13:01:42.756 INFO:teuthology.orchestra.run.smithi007:> sudo systemctl stop ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007 2020-06-16T13:01:42.812 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007.service: Unit ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007.service not loaded. 2020-06-16T13:01:42.813 DEBUG:teuthology.orchestra.run:got remote process result: 5 2020-06-16T13:01:42.813 ERROR:tasks.cephadm:Failed to stop "ceph.mon.smithi007" Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri2-testing-2020-06-15-1645-octopus/qa/tasks/cephadm.py", line 462, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/daemon/cephadmunit.py", line 149, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/remote.py", line 250, in sh proc=self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/remote.py", line 206, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 475, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 162, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 184, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi007 with status 5: 'sudo systemctl stop ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007' 2020-06-16T13:01:42.814 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/contextutil.py", line 32, in nested vars.append(enter()) File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri2-testing-2020-06-15-1645-octopus/qa/tasks/cephadm.py", line 462, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/daemon/cephadmunit.py", line 149, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/remote.py", line 250, in sh proc=self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/remote.py", line 206, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 475, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 162, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 184, in _raise_for_status node=self.hostname, label=self.label CommandFailedError: Command failed on smithi007 with status 5: 'sudo systemctl stop ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007'
sudo systemctl stop ceph-83387b56-afd1-11ea-a06d-001a4aab830c@mon.smithi007
at first glance would lead me to believe a Ceph code change is to blame but the real error is Error response from daemon: manifest for quay.io/ceph-ci/ceph:32f62798ee9a2145abdafd82203a5740d87c5dbd not found: manifest unknown: manifest unknown
which is kind of my responsibility.
Actions