Tools » teuthology

1) we also need to add some sleep before reconnect to avoid immediate reconnect ( should be fixed in teuthology )

2017-05-12T05:52:01.414 INFO:teuthology.orchestra.run.vpm091:Running: '/bin/echo -e \'cat <<EOF\\nset default="Advanced options for Ubuntu>Ubuntu, with Linux 4.11.0-ceph-g6f58448462f5"\\nEOF\\n\' | sudo tee -- \'/etc/grub.d/01_ceph_kernel.tmp~\' >/dev/null && sudo chmod a+x -- \'/etc/grub.d/01_ceph_kernel.tmp~\' && sudo mv -- \'/etc/grub.d/01_ceph_kernel.tmp~\' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & )'
2017-05-12T05:52:01.482 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@vpm091.front.sepia.ceph.com to complete...
2017-05-12T05:52:01.630 INFO:teuthology.orchestra.run.vpm091.stderr:Generating grub configuration file ...
2017-05-12T05:52:01.689 INFO:teuthology.orchestra.run.vpm091.stderr:Found linux image: /boot/vmlinuz-4.11.0-ceph-g6f58448462f5
2017-05-12T05:52:01.695 INFO:teuthology.orchestra.run.vpm091.stderr:Found initrd image: /boot/initrd.img-4.11.0-ceph-g6f58448462f5
2017-05-12T05:52:01.808 INFO:teuthology.orchestra.run.vpm091.stderr:Found linux image: /boot/vmlinuz-4.4.0-24-generic
2017-05-12T05:52:01.815 INFO:teuthology.orchestra.run.vpm091.stderr:Found initrd image: /boot/initrd.img-4.4.0-24-generic
2017-05-12T05:52:01.928 INFO:teuthology.orchestra.run.vpm091.stderr:done
2017-05-12T05:52:02.983 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@vpm165.front.sepia.ceph.com to complete...
2017-05-12T05:52:03.291 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@vpm131.front.sepia.ceph.com to complete...
2017-05-12T05:52:03.291 INFO:teuthology.misc:Re-opening connections...
2017-05-12T05:52:03.291 INFO:teuthology.misc:trying to connect to ubuntu@vpm165.front.sepia.ceph.com
2017-05-12T05:52:03.292 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm165.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:03.569 INFO:teuthology.orchestra.run.vpm165:Running: 'true'
2017-05-12T05:52:03.882 INFO:teuthology.misc:trying to connect to ubuntu@vpm091.front.sepia.ceph.com
2017-05-12T05:52:03.883 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm091.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:03.884 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.91
2017-05-12T05:52:03.884 DEBUG:teuthology.misc:waited 0.592638969421
2017-05-12T05:52:04.885 INFO:teuthology.misc:trying to connect to ubuntu@vpm131.front.sepia.ceph.com
2017-05-12T05:52:04.885 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm131.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:04.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.131
2017-05-12T05:52:04.887 INFO:teuthology.misc:trying to connect to ubuntu@vpm091.front.sepia.ceph.com
2017-05-12T05:52:04.887 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm091.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:04.888 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.91
2017-05-12T05:52:04.888 DEBUG:teuthology.misc:waited 1.5967900753
2017-05-12T05:52:05.889 INFO:teuthology.misc:trying to connect to ubuntu@vpm131.front.sepia.ceph.com
2017-05-12T05:52:05.890 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm131.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:05.891 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.131
2017-05-12T05:52:05.891 INFO:teuthology.misc:trying to connect to ubuntu@vpm091.front.sepia.ceph.com
2017-05-12T05:52:05.891 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm091.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}
2017-05-12T05:52:21.255 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.91
2017-05-12T05:52:21.255 DEBUG:teuthology.misc:waited 17.9639761448
2017-05-12T05:52:22.257 INFO:teuthology.misc:trying to connect to ubuntu@vpm131.front.sepia.ceph.com
2017-05-12T05:52:22.258 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'vpm131.front.sepia.ceph.com', 'key_filename': ['/home/teuthworker/.ssh/id_rsa'], 'timeout': 60}