Actions
Bug #44399
openqa: /usr/libexec/platform-python fails / core dumps
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2020-03-03T16:28:22.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive
2020-03-03T16:28:22.320 INFO:teuthology.task.internal:Transferring archived files...
2020-03-03T16:28:22.320 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2020-03-03_14:22:02-kcephfs-master-distro-basic-smithi/4820744/remote/smithi063
2020-03-03T16:28:22.320 INFO:teuthology.orchestra.run.smithi063:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- .
2020-03-03T16:28:24.154 INFO:teuthology.task.internal:Transferring binaries for coredumps...
2020-03-03T16:28:24.193 INFO:teuthology.orchestra.run.smithi063:> which /usr/libexec/platform-python
2020-03-03T16:28:24.232 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/libexec/platform-python
2020-03-03T16:28:24.289 DEBUG:teuthology.orchestra.remote:smithi063:/usr/libexec/platform-python is 12KB
2020-03-03T16:28:24.335 ERROR:teuthology.run_tasks:Manager failed: internal.archive
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 159, in run_tasks
suppress = manager.__exit__(*exc_info)
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 363, in archive
fetch_binaries_for_coredumps(path, rem)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 318, in fetch_binaries_for_coredumps
dump_program))
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 320, in _sftp_get_file
sftp.get(remote_path, local_path)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/sftp_client.py", line 801, in get
with open(localpath, "wb") as fl:
IOError: [Errno 13] Permission denied: '/usr/libexec/platform-python'
2020-03-03T16:28:24.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload
2020-03-03T16:28:24.360 INFO:teuthology.task.internal:Not uploading archives.
2020-03-03T16:28:24.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.base
2020-03-03T16:28:24.374 INFO:teuthology.task.internal:Tidying up after the test...
2020-03-03T16:28:24.374 INFO:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest
2020-03-03T16:28:24.381 INFO:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest
2020-03-03T16:28:24.384 INFO:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest
2020-03-03T16:28:24.396 INFO:teuthology.orchestra.run.smithi063.stdout: 262149 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 3 16:26 /home/ubuntu/cephtest
2020-03-03T16:28:24.396 INFO:teuthology.orchestra.run.smithi063.stdout: 262150 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive
2020-03-03T16:28:24.396 INFO:teuthology.orchestra.run.smithi063.stdout: 262170 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit
2020-03-03T16:28:24.396 INFO:teuthology.orchestra.run.smithi063.stdout: 262174 92 -rw------- 1 ubuntu ubuntu 91917 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit/audit.log.gz
2020-03-03T16:28:24.396 INFO:teuthology.orchestra.run.smithi063.stdout: 262152 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog
2020-03-03T16:28:24.397 INFO:teuthology.orchestra.run.smithi063.stdout: 262154 4 -rw-rw-rw- 1 ubuntu ubuntu 678 Mar 3 16:19 /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2020-03-03T16:28:24.397 INFO:teuthology.orchestra.run.smithi063.stdout: 262172 68 -rw-rw-rw- 1 ubuntu ubuntu 65894 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2020-03-03T16:28:24.397 INFO:teuthology.orchestra.run.smithi063.stdout: 262151 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 3 15:54 /home/ubuntu/cephtest/archive/coredump
2020-03-03T16:28:24.397 INFO:teuthology.orchestra.run.smithi063.stdout: 262159 47596 -rw------- 1 root root 57581568 Mar 3 15:54 /home/ubuntu/cephtest/archive/coredump/1583250867.14575.core
2020-03-03T16:28:24.398 INFO:teuthology.orchestra.run.smithi063.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty
2020-03-03T16:28:24.401 INFO:teuthology.orchestra.run.smithi099.stdout: 262153 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 3 16:26 /home/ubuntu/cephtest
2020-03-03T16:28:24.401 INFO:teuthology.orchestra.run.smithi099.stdout: 262154 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive
2020-03-03T16:28:24.401 INFO:teuthology.orchestra.run.smithi099.stdout: 262170 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit
2020-03-03T16:28:24.402 INFO:teuthology.orchestra.run.smithi099.stdout: 262174 100 -rw------- 1 ubuntu ubuntu 100166 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit/audit.log.gz
2020-03-03T16:28:24.402 INFO:teuthology.orchestra.run.smithi099.stdout: 262156 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog
2020-03-03T16:28:24.402 INFO:teuthology.orchestra.run.smithi099.stdout: 262158 4 -rw-rw-rw- 1 ubuntu ubuntu 686 Mar 3 16:19 /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2020-03-03T16:28:24.402 INFO:teuthology.orchestra.run.smithi099.stdout: 262172 72 -rw-rw-rw- 1 ubuntu ubuntu 69870 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2020-03-03T16:28:24.404 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty
2020-03-03T16:28:24.407 INFO:teuthology.orchestra.run.smithi040.stdout: 393248 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 3 16:26 /home/ubuntu/cephtest
2020-03-03T16:28:24.407 INFO:teuthology.orchestra.run.smithi040.stdout: 393249 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive
2020-03-03T16:28:24.407 INFO:teuthology.orchestra.run.smithi040.stdout: 524332 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit
2020-03-03T16:28:24.407 INFO:teuthology.orchestra.run.smithi040.stdout: 524609 92 -rw------- 1 ubuntu ubuntu 90692 Mar 3 16:28 /home/ubuntu/cephtest/archive/audit/audit.log.gz
2020-03-03T16:28:24.408 INFO:teuthology.orchestra.run.smithi040.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog
2020-03-03T16:28:24.408 INFO:teuthology.orchestra.run.smithi040.stdout: 393237 4 -rw-rw-rw- 1 ubuntu ubuntu 2921 Mar 3 16:02 /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2020-03-03T16:28:24.408 INFO:teuthology.orchestra.run.smithi040.stdout: 393236 68 -rw-rw-rw- 1 ubuntu ubuntu 66499 Mar 3 16:28 /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2020-03-03T16:28:24.409 INFO:teuthology.orchestra.run.smithi040.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty
2020-03-03T16:28:24.409 DEBUG:teuthology.orchestra.run:got remote process result: 1
2020-03-03T16:28:24.409 ERROR:teuthology.run_tasks:Manager failed: internal.base
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 159, in run_tasks
suppress = manager.__exit__(*exc_info)
File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/internal/__init__.py", line 49, in base
wait=False,
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 457, in wait
proc.wait()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 158, in wait
self._raise_for_status()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 180, in _raise_for_status
node=self.hostname, label=self.label
CommandFailedError: Command failed on smithi040 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest'
2020-03-03T16:28:24.411 DEBUG:teuthology.run_tasks:Unwinding manager console_log
2020-03-03T16:28:24.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.lock_machines
2020-03-03T16:28:24.443 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: IOError: [Errno 13] Permission denied: '/usr/libexec/platform-python'
From: /ceph/teuthology-archive/pdonnell-2020-03-03_14:22:02-kcephfs-master-distro-basic-smithi/4820744/teuthology.log
Core: /ceph/teuthology-archive/pdonnell-2020-03-03_14:22:02-kcephfs-master-distro-basic-smithi/4820744/remote/smithi063/coredump/1583250867.14575.core
./remote/smithi063/coredump/1583250867.14575.core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from '/usr/libexec/platform-python /usr/bin/dnf makecache --timer'
Actions