Project

General

Profile

Bug #12872

failed to install new kernel version within timeout

Added by Greg Farnum almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
08/31/2015
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:

Description

2015-08-28T15:58:01.792 INFO:teuthology.misc:trying to connect to ubuntu@plana81.front.sepia.ceph.com
2015-08-28T15:58:01.793 INFO:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'plana81.front.sepia.ceph.com', 'timeout': 60}
2015-08-28T15:58:03.792 DEBUG:teuthology.orchestra.remote:[Errno 113] No route to host
2015-08-28T15:58:03.792 DEBUG:teuthology.misc:waited 106.124459982
2015-08-28T15:58:04.793 INFO:teuthology.misc:trying to connect to ubuntu@plana81.front.sepia.ceph.com
2015-08-28T15:58:04.793 INFO:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'plana81.front.sepia.ceph.com', 'timeout': 60}
2015-08-28T15:59:04.793 DEBUG:teuthology.orchestra.remote:timed out
2015-08-28T15:59:04.793 DEBUG:teuthology.misc:waited 167.125650883
2015-08-28T15:59:05.794 INFO:teuthology.misc:trying to connect to ubuntu@plana81.front.sepia.ceph.com
2015-08-28T15:59:05.794 INFO:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'plana81.front.sepia.ceph.com', 'timeout': 60}
2015-08-28T15:59:06.016 INFO:teuthology.orchestra.run.plana81:Running: 'true'
2015-08-28T15:59:06.408 DEBUG:teuthology.misc:waited 168.7398839
2015-08-28T15:59:07.407 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version...
2015-08-28T15:59:07.407 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want 4.2.0-rc2-ceph-00013-ga7ef36f...
2015-08-28T15:59:07.408 INFO:teuthology.orchestra.run.burnupi49:Running: 'uname -r'
2015-08-28T15:59:07.416 DEBUG:teuthology.task.kernel:current kernel version is 4.2.0-rc2-ceph-00013-ga7ef36f
2015-08-28T15:59:07.417 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install
2015-08-28T15:59:07.417 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version...
2015-08-28T15:59:07.417 INFO:teuthology.task.kernel:Checking kernel version of client.0, want 4.2.0...
2015-08-28T15:59:07.417 INFO:teuthology.orchestra.run.plana81:Running: 'uname -r'
2015-08-28T15:59:07.590 DEBUG:teuthology.task.kernel:current kernel version is 4.2.0-rc2-ceph-00013-ga7ef36f
2015-08-28T15:59:07.590 ERROR:teuthology.task.kernel:Saw exception
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/task/kernel.py", line 633, in wait_for_reboot
    'failed to install new kernel version within timeout'
AssertionError: failed to install new kernel version within timeout

http://pulpito.ceph.com/teuthology-2015-08-23_23:18:02-multimds-next-testing-basic-multi/1028561/ is plana45
http://pulpito.ceph.com/teuthology-2015-08-23_23:18:02-multimds-next-testing-basic-multi/1028548/ is plana81
1028544 is plana53
1028572 is nonobvious, either plana81 or plana47 (seen elsewhere)
1028559 looks like plana65
1028571 is unclear (planas 75 and 84, burnupi13) is plana17
1028552 is unclear
...and there are even more, I don't think this is individual machines being bad.

History

#1 Updated by Andrew Schoen almost 4 years ago

  • Status changed from New to Resolved
  • Assignee set to Andrew Schoen

This was fixed in: https://github.com/ceph/teuthology/commit/65e5569023b6f8a1ab1572694baa3d46f4ed6ab3

See where it's expecting the shorter version number? The longer version is what's actually installed. This bug was caused by some code the does some ceph specific mangling to to version number so that we can verify it's installed correctly. That codepath is not needed on other packages, only ceph.

2015-08-28T15:59:07.417 INFO:teuthology.task.kernel:Checking kernel version of client.0, want 4.2.0...
2015-08-28T15:59:07.417 INFO:teuthology.orchestra.run.plana81:Running: 'uname -r'
2015-08-28T15:59:07.590 DEBUG:teuthology.task.kernel:current kernel version is 4.2.0-rc2-ceph-00013-ga7ef36f

Also available in: Atom PDF