Bug #14840
closedmira091 is not accessible
Added by Yuri Weinstein about 8 years ago. Updated about 7 years ago.
0%
Description
via ssh and/or ipmitool
Files
| Screenshot from 2016-02-22 17_56_59.png (31.2 KB) Screenshot from 2016-02-22 17_56_59.png | David Galloway, 02/22/2016 10:57 PM | ||
| mira005.png (24.6 KB) mira005.png | David Galloway, 03/09/2017 05:29 PM |
Updated by David Galloway about 8 years ago
Updated by David Galloway about 8 years ago
- Category set to Infrastructure Hardware
- Status changed from New to Resolved
I ran memtest on this machine without issue.
I updated its BIOS and set the VPSes back up. If the issue persists, we may have to retire the VPSHOST.
Updated by Yuri Weinstein about 8 years ago
Reopening this as:
ubuntu@teuthology:~$ ssh mira091 ssh: connect to host mira091 port 22: No route to host
and also in job http://qa-proxy.ceph.com/teuthology/teuthology-2016-03-11_10:42:14-upgrade:client-upgrade-jewel-distro-basic-vps/53265/teuthology.log
2016-03-11T10:52:19.046 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:28.369 INFO:teuthology.provision:Downburst created ubuntu@vpm019.front.sepia.ceph.com: Fetching default SSH key from http://ceph.com/git/?p=keys.git;a=blob_plain;f=ssh/teuthology-ubuntu.pub;hb=HEAD
2016-03-11T10:52:28.410 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:32.709 INFO:teuthology.provision:Downburst created ubuntu@vpm012.front.sepia.ceph.com: Fetching default SSH key from http://ceph.com/git/?p=keys.git;a=blob_plain;f=ssh/teuthology-ubuntu.pub;hb=HEAD
2016-03-11T10:52:32.751 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:37.028 INFO:teuthology.provision:Downburst failed on ubuntu@vpm192.front.sepia.ceph.com: libvirt: XML-RPC error : Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Traceback (most recent call last):
File "/home/ubuntu/src/downburst/virtualenv/bin/downburst", line 9, in <module>
load_entry_point('downburst==0.0.1', 'console_scripts', 'downburst')()
File "/home/ubuntu/src/downburst/downburst/cli.py", line 59, in main
return args.func(args)
File "/home/ubuntu/src/downburst/downburst/create.py", line 22, in create
conn = libvirt.open(args.connect)
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 252, in open
if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
2016-03-11T10:52:37.029 ERROR:teuthology.lock:Unable to create virtual machine: ubuntu@vpm192.front.sepia.ceph.com
2016-03-11T10:52:40.050 ERROR:teuthology.provision:Error destroying vpm192.front.sepia.ceph.com: libvirt: XML-RPC error : Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Traceback (most recent call last):
File "/home/ubuntu/src/downburst/virtualenv/bin/downburst", line 9, in <module>
load_entry_point('downburst==0.0.1', 'console_scripts', 'downburst')()
File "/home/ubuntu/src/downburst/downburst/cli.py", line 59, in main
return args.func(args)
File "/home/ubuntu/src/downburst/downburst/destroy.py", line 42, in destroy
conn = libvirt.open(args.connect)
File "/usr/lib/python2.7/dist-packages/libvirt.py", line 252, in open
if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Updated by Dan Mick about 8 years ago
cycled power and it came back up; syslog doesn't seem to have anything very useful in it, nor does kern.log.
I do notice this, which is weird, but surely unconnected:
Mar 11 22:43:11 mira091 ntpd1976: i/o error on routing socket No buffer space available - disabling
Updated by David Galloway about 8 years ago
- Status changed from New to Resolved
Machine seems to be stable now?
ubuntu@mira091:~$ uptime 18:58:27 up 53 days, 20:16, 2 users, load average: 0.64, 1.17, 1.50
Updated by David Galloway almost 8 years ago
- Status changed from Resolved to In Progress
This system's got at least 1 bad DIMM according to SEL. Will have lab team diagnose and replace.
I've marked down its VPSes in the meantime.
Updated by David Galloway over 7 years ago
Lab team didn't find any bad DIMMs. I'm reimaging the host now.
Updated by David Galloway over 7 years ago
- Status changed from In Progress to Resolved
Reimaged this host on 4OCT and brought VMs back up today.
Updated by David Galloway over 7 years ago
- Status changed from Resolved to In Progress
This machine locked up again. We'll have to retire it.
Updated by David Galloway about 7 years ago
- File mira005.png mira005.png added
I moved the VPSes to mira005 but it MCE'd last night. I updated BIOS firmware and am running memtest now.
Updated by David Galloway about 7 years ago
- Status changed from In Progress to Closed
System is on the list to be e-wasted.