Bug #45592
Updated by Patrick Donnelly almost 4 years ago
<pre>
2020-05-16T09:59:44.263 INFO:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf
2020-05-16T09:59:55.308 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at
2020-05-16T10:15:14.402 ERROR:paramiko.transport:Socket exception: No route to host (113)
2020-05-16T10:15:14.407 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules...
2020-05-16T10:15:14.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host
2020-05-16T10:15:14.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2020-05-16T10:15:14.412 DEBUG:tasks.ceph:Missed logrotate, SSHException
2020-05-16T10:15:14.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s
2020-05-16T10:15:14.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s
2020-05-16T10:15:17.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19
2020-05-16T10:15:17.475 INFO:tasks.cephfs_test_runner:test_up_less_than_max (tasks.cephfs.test_failover.TestClusterResize) ... ERROR
2020-05-16T10:15:17.476 INFO:tasks.cephfs_test_runner:ERROR
2020-05-16T10:15:17.476 INFO:tasks.cephfs_test_runner:
2020-05-16T10:15:17.476 INFO:tasks.cephfs_test_runner:======================================================================
2020-05-16T10:15:17.477 INFO:tasks.cephfs_test_runner:ERROR: test_up_less_than_max (tasks.cephfs.test_failover.TestClusterResize)
2020-05-16T10:15:17.477 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2020-05-16T10:15:17.478 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2020-05-16T10:15:17.478 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/cephfs/test_failover.py", line 248, in test_up_less_than_max
2020-05-16T10:15:17.478 INFO:tasks.cephfs_test_runner: self.shrink(2)
2020-05-16T10:15:17.478 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/cephfs/test_failover.py", line 213, in shrink
2020-05-16T10:15:17.479 INFO:tasks.cephfs_test_runner: status = self.fs.wait_for_daemons(timeout=60+grace*2)
2020-05-16T10:15:17.479 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/cephfs/filesystem.py", line 918, in wait_for_daemons
2020-05-16T10:15:17.479 INFO:tasks.cephfs_test_runner: status = self.status()
2020-05-16T10:15:17.480 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/cephfs/filesystem.py", line 333, in status
2020-05-16T10:15:17.480 INFO:tasks.cephfs_test_runner: return FSStatus(self.mon_manager)
2020-05-16T10:15:17.480 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/cephfs/filesystem.py", line 68, in __init__
2020-05-16T10:15:17.480 INFO:tasks.cephfs_test_runner: self.map = json.loads(self.mon.raw_cluster_cmd("fs", "dump", "--format=json"))
2020-05-16T10:15:17.481 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20200516.030215/qa/tasks/ceph_manager.py", line 1363, in raw_cluster_cmd
2020-05-16T10:15:17.481 INFO:tasks.cephfs_test_runner: stdout=BytesIO(),
2020-05-16T10:15:17.481 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 206, in run
2020-05-16T10:15:17.482 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2020-05-16T10:15:17.482 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 468, in run
2020-05-16T10:15:17.482 INFO:tasks.cephfs_test_runner: r.execute()
2020-05-16T10:15:17.482 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 102, in execute
2020-05-16T10:15:17.483 INFO:tasks.cephfs_test_runner: self.client.exec_command(self.command)
2020-05-16T10:15:17.483 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 508, in exec_command
2020-05-16T10:15:17.483 INFO:tasks.cephfs_test_runner: chan = self._transport.open_session(timeout=timeout)
2020-05-16T10:15:17.483 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 879, in open_session
2020-05-16T10:15:17.484 INFO:tasks.cephfs_test_runner: timeout=timeout,
2020-05-16T10:15:17.484 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 1006, in open_channel
2020-05-16T10:15:17.484 INFO:tasks.cephfs_test_runner: raise e
2020-05-16T10:15:17.485 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/transport.py", line 2055, in run
2020-05-16T10:15:17.485 INFO:tasks.cephfs_test_runner: ptype, m = self.packetizer.read_message()
2020-05-16T10:15:17.485 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 459, in read_message
2020-05-16T10:15:17.485 INFO:tasks.cephfs_test_runner: header = self.read_all(self.__block_size_in, check_rekey=True)
2020-05-16T10:15:17.486 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/paramiko/packet.py", line 301, in read_all
2020-05-16T10:15:17.486 INFO:tasks.cephfs_test_runner: x = self.__socket.recv(n)
2020-05-16T10:15:17.487 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/lib/python3.6/site-packages/gevent/_socket3.py", line 378, in recv
2020-05-16T10:15:17.487 INFO:tasks.cephfs_test_runner: return _socket.socket.recv(self._sock, *args)
2020-05-16T10:15:17.487 INFO:tasks.cephfs_test_runner:OSError: [Errno 113] No route to host
</pre>
From: /ceph/teuthology-archive/pdonnell-2020-05-16_06:07:05-fs-wip-pdonnell-testing-20200516.030215-distro-basic-smithi/5060351/teuthology.log