Project

General

Profile

Actions
Copy link

Bug #14840

closed

mira091 is not accessible

Added by Yuri Weinstein about 8 years ago. Updated about 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
David Galloway
Category:
Infrastructure Hardware
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

via ssh and/or ipmitool

Files

Download all files
Screenshot from 2016-02-22 17_56_59.png (31.2 KB) Screenshot from 2016-02-22 17_56_59.png David Galloway, 02/22/2016 10:57 PM
mira005.png (24.6 KB) mira005.png David Galloway, 03/09/2017 05:29 PM
Actions
Copy link
 #2

Updated by David Galloway about 8 years ago

  • Category set to Infrastructure Hardware
  • Status changed from New to Resolved

I ran memtest on this machine without issue.

I updated its BIOS and set the VPSes back up. If the issue persists, we may have to retire the VPSHOST.

Actions
Copy link
 #3

Updated by Yuri Weinstein about 8 years ago

Reopening this as:

ubuntu@teuthology:~$ ssh mira091
ssh: connect to host mira091 port 22: No route to host

and also in job http://qa-proxy.ceph.com/teuthology/teuthology-2016-03-11_10:42:14-upgrade:client-upgrade-jewel-distro-basic-vps/53265/teuthology.log

2016-03-11T10:52:19.046 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:28.369 INFO:teuthology.provision:Downburst created ubuntu@vpm019.front.sepia.ceph.com: Fetching default SSH key from http://ceph.com/git/?p=keys.git;a=blob_plain;f=ssh/teuthology-ubuntu.pub;hb=HEAD
2016-03-11T10:52:28.410 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:32.709 INFO:teuthology.provision:Downburst created ubuntu@vpm012.front.sepia.ceph.com: Fetching default SSH key from http://ceph.com/git/?p=keys.git;a=blob_plain;f=ssh/teuthology-ubuntu.pub;hb=HEAD
2016-03-11T10:52:32.751 INFO:teuthology.provision:Provisioning a ubuntu 14.04 vps
2016-03-11T10:52:37.028 INFO:teuthology.provision:Downburst failed on ubuntu@vpm192.front.sepia.ceph.com: libvirt: XML-RPC error : Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Traceback (most recent call last):
  File "/home/ubuntu/src/downburst/virtualenv/bin/downburst", line 9, in <module>
    load_entry_point('downburst==0.0.1', 'console_scripts', 'downburst')()
  File "/home/ubuntu/src/downburst/downburst/cli.py", line 59, in main
    return args.func(args)
  File "/home/ubuntu/src/downburst/downburst/create.py", line 22, in create
    conn = libvirt.open(args.connect)
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 252, in open
    if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
2016-03-11T10:52:37.029 ERROR:teuthology.lock:Unable to create virtual machine: ubuntu@vpm192.front.sepia.ceph.com
2016-03-11T10:52:40.050 ERROR:teuthology.provision:Error destroying vpm192.front.sepia.ceph.com: libvirt: XML-RPC error : Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Traceback (most recent call last):
  File "/home/ubuntu/src/downburst/virtualenv/bin/downburst", line 9, in <module>
    load_entry_point('downburst==0.0.1', 'console_scripts', 'downburst')()
  File "/home/ubuntu/src/downburst/downburst/cli.py", line 59, in main
    return args.func(args)
  File "/home/ubuntu/src/downburst/downburst/destroy.py", line 42, in destroy
    conn = libvirt.open(args.connect)
  File "/usr/lib/python2.7/dist-packages/libvirt.py", line 252, in open
    if ret is None:raise libvirtError('virConnectOpen() failed')
libvirt.libvirtError: Cannot recv data: ssh: connect to host mira091.front.sepia.ceph.com port 22: No route to host: Connection reset by peer
Actions
Copy link
 #4

Updated by Yuri Weinstein about 8 years ago

  • Status changed from Resolved to New
Actions
Copy link
 #5

Updated by Dan Mick about 8 years ago

cycled power and it came back up; syslog doesn't seem to have anything very useful in it, nor does kern.log.

I do notice this, which is weird, but surely unconnected:

Mar 11 22:43:11 mira091 ntpd1976: i/o error on routing socket No buffer space available - disabling

Actions
Copy link
 #6

Updated by David Galloway almost 8 years ago

  • Status changed from New to Resolved

Machine seems to be stable now?

ubuntu@mira091:~$ uptime
 18:58:27 up 53 days, 20:16,  2 users,  load average: 0.64, 1.17, 1.50
Actions
Copy link
 #7

Updated by David Galloway almost 8 years ago

  • Status changed from Resolved to In Progress

This system's got at least 1 bad DIMM according to SEL. Will have lab team diagnose and replace.

I've marked down its VPSes in the meantime.

Actions
Copy link
 #8

Updated by David Galloway over 7 years ago

Lab team didn't find any bad DIMMs. I'm reimaging the host now.

Actions
Copy link
 #9

Updated by David Galloway over 7 years ago

  • Status changed from In Progress to Resolved

Reimaged this host on 4OCT and brought VMs back up today.

Actions
Copy link
 #10

Updated by David Galloway over 7 years ago

  • Status changed from Resolved to In Progress

This machine locked up again. We'll have to retire it.

Actions
Copy link
 #11

Updated by David Galloway about 7 years ago

I moved the VPSes to mira005 but it MCE'd last night. I updated BIOS firmware and am running memtest now.

Actions
Copy link
 #12

Updated by David Galloway about 7 years ago

  • Status changed from In Progress to Closed

System is on the list to be e-wasted.

Actions
Copy link

Also available in: Atom PDF