Bug #45420
cephadmunit.py: teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 125: 'sudo docker kill -s 1 ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1'
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
teuthology
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
/a/bhubbard-2020-05-01_23:30:27-rados:thrash-old-clients-master-distro-basic-smithi/5014156
2020-05-02T23:16:25.929 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:25 smithi094 systemd[1]: Stopping Ceph osd.1 for d8648236-8cc8-11ea-a068-001a4aab830c... 2020-05-02T23:16:26.039 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 bash[17770]: debug 2020-05-02T23:16:26.051+0000 7fa396728700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2020-05-02T23:16:26.039 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 bash[17770]: debug 2020-05-02T23:16:26.051+0000 7fa396728700 -1 osd.1 455 *** Got signal Terminated *** 2020-05-02T23:16:26.204 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 bash[17770]: debug 2020-05-02T23:16:26.051+0000 7fa396728700 -1 osd.1 455 *** Immediate shutdown (osd_fast_shutdown=true) *** 2020-05-02T23:16:26.204 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 podman[143190]: 2020-05-02 23:16:26.216427372 +0000 UTC m=+0.257080594 container died 900d37e94a3796ff360f9007f1d215afb583884e61cec3e809cbc59fde518bde (image=quay.io/ceph-ci/ceph:beaa4b04bc57ed43e98602e493e8a787a014b4e6, name=ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1) 2020-05-02T23:16:26.247 INFO:teuthology.orchestra.run.smithi094.stderr:Error: can only kill running containers. 900d37e94a3796ff360f9007f1d215afb583884e61cec3e809cbc59fde518bde is in state stopped: container state improper 2020-05-02T23:16:26.257 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 podman[143190]: 2020-05-02 23:16:26.257171524 +0000 UTC m=+0.297824636 container stop 900d37e94a3796ff360f9007f1d215afb583884e61cec3e809cbc59fde518bde (image=quay.io/ceph-ci/ceph:beaa4b04bc57ed43e98602e493e8a787a014b4e6, name=ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1) 2020-05-02T23:16:26.257 INFO:ceph.osd.1.smithi094.stdout:May 02 23:16:26 smithi094 podman[143190]: 900d37e94a3796ff360f9007f1d215afb583884e61cec3e809cbc59fde518bde 2020-05-02T23:16:26.266 DEBUG:teuthology.orchestra.run:got remote process result: 125 2020-05-02T23:16:26.267 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 122, in wrapper return func(self) File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 1095, in do_sighup self.ceph_manager.signal_osd(osd, signal.SIGHUP, silent=True) File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 2797, in signal_osd self.cluster).signal(sig, silent=silent) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/daemon/cephadmunit.py", line 123, in signal self.remote.sh(self.kill_cmd(sig)) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 247, in sh proc=self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 203, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 473, in run r.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status node=self.hostname, label=self.label teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 125: 'sudo docker kill -s 1 ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1'
Related issues
History
#1 Updated by Sebastian Wagner almost 4 years ago
- Subject changed from teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 125: 'sudo docker kill -s 1 ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1' to cephadmunit.py: teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 125: 'sudo docker kill -s 1 ceph-d8648236-8cc8-11ea-a068-001a4aab830c-osd.1'
#2 Updated by Deepika Upadhyay over 3 years ago
- Related to Bug #46247: cephadm mon failure: Error: no container with name or ID ... no such container added
#3 Updated by Sebastian Wagner over 3 years ago
- Category set to teuthology
#4 Updated by Sebastian Wagner about 3 years ago
- Priority changed from High to Normal
#5 Updated by Sebastian Wagner over 2 years ago
- Status changed from New to Can't reproduce