Bug #14335
closedopenstack: 'NoneType' object has no attribute '__getitem__'
0%
Description
Some jobs fail with:
@2016-01-10T13:26:07.782 DEBUG:teuthology.misc::sh: neutron subnet-list -f json -c id -c ip_version
2016-01-10T13:26:08.525 DEBUG:teuthology.misc:[
2016-01-10T13:26:08.525 DEBUG:teuthology.misc:{
2016-01-10T13:26:08.526 DEBUG:teuthology.misc:"ip_version": 4,
2016-01-10T13:26:08.526 DEBUG:teuthology.misc:"id": "a92ff450-0b85-4f46-b98a-33d3cfe0dbe3"
2016-01-10T13:26:08.526 DEBUG:teuthology.misc:}
2016-01-10T13:26:08.559 DEBUG:teuthology.misc:]
2016-01-10T13:26:08.559 DEBUG:teuthology.misc::sh: neutron port-list -f json -c fixed_ips -c device_id
2016-01-10T13:26:33.841 DEBUG:teuthology.misc:[
2016-01-10T13:26:33.843 DEBUG:teuthology.misc:{
2016-01-10T13:26:33.843 DEBUG:teuthology.misc:"fixed_ips": "{\"subnet_id\": \"a92ff450-0b85-4f46-b98a-33d3cfe0dbe3\", \"ip_address\": \"149.202.167.162\"}",
2016-01-10T13:26:33.844 DEBUG:teuthology.misc:"device_id": "1ff96af2-4a33-4d21-9aab-4cc580dc9037"
2016-01-10T13:26:33.844 DEBUG:teuthology.misc:},
2016-01-10T13:26:33.844 DEBUG:teuthology.misc:{
2016-01-10T13:26:33.844 DEBUG:teuthology.misc:"fixed_ips": "{\"subnet_id\": \"a92ff450-0b85-4f46-b98a-33d3cfe0dbe3\", \"ip_address\": \"149.202.168.132\"}",
2016-01-10T13:26:33.844 DEBUG:teuthology.misc:"device_id": "63b811b5-69c2-4d2b-a9f6-bfff5b81fa49"
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:},
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:{
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:"fixed_ips": "{\"subnet_id\": \"a92ff450-0b85-4f46-b98a-33d3cfe0dbe3\", \"ip_address\": \"149.202.169.159\"}",
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:"device_id": "eec60001-87c9-4f80-a542-7967d5e288b0"
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:},
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:{
2016-01-10T13:26:33.845 DEBUG:teuthology.misc:"fixed_ips": "{\"subnet_id\": \"a92ff450-0b85-4f46-b98a-33d3cfe0dbe3\", \"ip_address\": \"149.202.167.15\"}",
2016-01-10T13:26:33.846 DEBUG:teuthology.misc:"device_id": "68e5bffe-735a-465f-aab3-fe9d054ef137"
2016-01-10T13:26:33.846 DEBUG:teuthology.misc:},
2016-01-10T13:26:33.850 DEBUG:teuthology.misc:{
2016-01-10T13:26:33.850 DEBUG:teuthology.misc:"fixed_ips": "{\"subnet_id\": \"a92ff450-0b85-4f46-b98a-33d3cfe0dbe3\", \"ip_address\": \"149.202.169.158\"}",
2016-01-10T13:26:33.850 DEBUG:teuthology.misc:"device_id": "c35ab4ad-b743-40ea-83d3-45a7cd653987"
2016-01-10T13:26:33.850 DEBUG:teuthology.misc:}
2016-01-10T13:26:33.877 DEBUG:teuthology.misc:]
2016-01-10T13:26:33.878 DEBUG:teuthology.openstack:ignoring get_ip_neutron exception 'NoneType' object has no attribute '__getitem__'
2016-01-10T13:26:33.878 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
File "/home/ubuntu/teuthology/teuthology/run_tasks.py", line 53, in run_tasks
manager = run_one_task(taskname, ctx=ctx, config=config)
File "/home/ubuntu/teuthology/teuthology/run_tasks.py", line 41, in run_one_task
return fn(**kwargs)
File "/home/ubuntu/src/ceph-qa-suite_master/tasks/buildpackages.py", line 189, in task
teuth_config.gitbuilder_host = openstack.get_ip('packages-repository', '')
File "/home/ubuntu/teuthology/teuthology/openstack/__init__.py", line 429, in get_ip
return OpenStackInstance(instance_id).get_ip(network)
File "/home/ubuntu/teuthology/teuthology/openstack/__init__.py", line 142, in get_ip
self.get_addresses())[0]
File "/home/ubuntu/teuthology/teuthology/openstack/__init__.py", line 99, in get_addresses
action="get ip " + self['id']) as proceed:
File "/home/ubuntu/teuthology/teuthology/openstack/__init__.py", line 75, in getitem
return self.info[name.lower()]
TypeError: 'NoneType' object has no attribute '__getitem__'
2016-01-10T13:26:33.880 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: TypeError: 'NoneType' object has no attribute '__getitem__'
2016-01-10T13:26:33.883 INFO:teuthology.run:Summary data:
{description: 'rados/thrash/{hobj-sort.yaml 0-size-min-size-overrides/3-size-2-min-size.yaml
1-pg-log-overrides/short_pg_log.yaml clusters/{fixed-2.yaml openstack.yaml} fs/xfs.yaml
msgr/simple.yaml msgr-failures/fastclose.yaml thrashers/pggrow.yaml workloads/cache-snaps.yaml}',
failure_reason: '''NoneType'' object has no attribute ''__getitem__''', owner: scheduled_ubuntu@teuthology,
status: fail, success: false}
@
This probably could be fixed by having a guard ion https://github.com/ceph/teuthology/blob/master/teuthology/openstack/__init__.py#L66.
Files
Updated by Piotr Dalek over 8 years ago
- File teuthology (2).log teuthology (2).log added
Attached is Teuthology log from job where this occurred.
Updated by Loïc Dachary about 8 years ago
- Status changed from New to Won't Fix
That happens when the "packages-repository" virtual machine is removed while the teuthology cluster still has a timestamp that claims it exists and contains packages for a given Ceph version. That can't happen unless the virtual machine has been manually destroyed. I don't think we can reasonably guard against the problems that would create, although we could fix this specific one in the way you suggest.