Project

General

Profile

Actions

Bug #16053

closed

hammer/infernalis: IndexError: list index out of range

Added by John Spray almost 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

Seeing this on fs-hammer nightlies. All the roles do have a period in them, so no idea where it's coming from:

2016-05-27T03:51:51.299 INFO:teuthology.run_tasks:Running task ceph.stop...
2016-05-27T03:51:51.309 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 69, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/ceph-qa-suite_hammer/tasks/ceph.py", line 1102, in stop
    id_ = i.split('.', 1)[1]
IndexError: list index out of range

http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-05-26_19:10:01-fs-hammer---basic-openstack/13140/teuthology.log
http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-05-26_19:10:01-fs-hammer---basic-openstack/13174/teuthology.log

Actions #3

Updated by John Spray almost 8 years ago

  • Subject changed from hammer: IndexError: list index out of range to hammer/infernalis: IndexError: list index out of range
Actions #4

Updated by Anonymous almost 8 years ago

  • Assignee set to Anonymous
Actions #5

Updated by Anonymous almost 8 years ago

I reproduced this on a non ovh run. Using the latest upstream ceph-qa-suite, the problem is different due to changes in ceph.py. The difference in the code is:

-    daemons = ctx.daemons.resolve_role_list(config.get('daemons', None), CEPH_ROLE_TYPES, True)
-    for role in daemons:
-        cluster, type_, id_ = teuthology.split_role(role)
-        ctx.daemons.get_daemon(type_, id_, cluster).stop()
+    daemons = ctx.daemons.resolve_role_list(config.get('daemons', None), CEPH_ROLE_TYPES)
+    for i in daemons:
+        type_ = i.split('.', 1)[0]
+        id_ = i.split('.', 1)[1]
+        ctx.daemons.get_daemon(type_, id_).stop()

However, this is still failing because ctx.daemons.get_daemon is returning None.
The get_daemon code is trying to match ceph.mdsa against ceph.mds and failing, I suspect. I am not sure what is changing either mds.a or mds.a-s to mdsa yet.

Actions #6

Updated by Anonymous almost 8 years ago

  • Status changed from New to 12
Actions #7

Updated by Anonymous almost 8 years ago

Inside the yaml file:

- ceph.stop:
- mds.*

I am guessing that this is generating mdsa instead of mds.a

(just a guess, I am running a test now and leaving for the day soon).

Actions #8

Updated by Anonymous almost 8 years ago


FWIW, I changed:

- ceph.stop:
  - mds.*

to

- ceph.stop:
  - mds.a
  - mds.a-s

and

- ceph.restart:
  - mds.*

to

- ceph.restart:
  - mds.a
  - mds.a-s

and the run passed.
Actions #9

Updated by Anonymous almost 8 years ago

teuthology/orchestra/daemon.py code is broken... Will fix.

Actions #10

Updated by Anonymous almost 8 years ago

Actions #11

Updated by Zack Cerza over 7 years ago

  • Status changed from 12 to Resolved
Actions

Also available in: Atom PDF