Tools » teuthology

I was able to manually install a brew-sourced kernel (https://brewweb.devel.redhat.com/buildinfo?buildID=416058) onto magna074 today using the steps teuthology takes. However, when I ran the kernel task against magna074 again with the kernel stanza set to "distro" it did not uninstall the brew-sourced kernel. Here's the log from that run: http://magna002.ceph.redhat.com/andrewschoen/test-kernel/teuthology.log

We might have a bug here in the teuthology code that decides where or not to re-install a "distro" kernel: https://github.com/ceph/teuthology/blob/master/teuthology/task/kernel.py#L571

If I run those checks need_to_install_distro does on a freshly nuked machine (magna067) it would consider kernel-3.18.0_ceph_11304_gc50def6-1.x86_64 a "distro" kernel which I'm not sure is correct.

I reran the kernel task against magna074 with the sha1: "distro" removed from the kernel stanza in the config. This time it did install a different kernel.

Here's the log: http://magna002.ceph.redhat.com/andrewschoen/test-kernel-no-sha1/teuthology.log

Before the run, this is what manga074 reported as the installed kernel.

[ubuntu@magna074 ~]$ uname -a
Linux magna074 3.10.0-123.20.1.el7.x86_64 #1 SMP Wed Jan 21 09:45:55 EST 2015 x86_64 x86_64 x86_64 GNU/Linux

After the run:

andrewschoen@magna002:~$ ssh magna074 'uname -a'
Linux magna074 3.18.0-rc4-ceph-00015-g388cfdc #1 SMP Thu Nov 13 15:52:32 EST 2014 x86_64 x86_64 x86_64 GNU/Linux

I've been running into connection errors with brewhub.devel.redhat.com/brewhub often. I'm not sure if there is a different endpoint I should be using (this is what mojo shows to use) or if we're being rate limited or something. Maybe brewhub just isn't that stable or we've got network issues in octo.

2015-01-23 10:03:55,819.819 INFO:teuthology.orchestra.run.magna067:Running: 'python -c \'import koji; hub = koji.ClientSession("http://brewhub.devel.redhat.com/brewhub"); print hub.
getBuild(416058)\''
2015-01-23 10:03:56,269.269 ERROR:teuthology.task.kernel:Failed to query koji for build 416058
2015-01-23 10:03:56,270.270 ERROR:teuthology.task.kernel:stdout:
2015-01-23 10:03:56,270.270 ERROR:teuthology.task.kernel:stderr: Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1555, in __call__
    return self.__func(self.__name,args,opts)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1898, in _callMethod
    return self._sendCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1809, in _sendCall
    return self._sendOneCall(handler, headers, request)
  File "/usr/lib/python2.7/site-packages/koji/__init__.py", line 1827, in _sendOneCall
    cnx.endheaders()
  File "/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 791, in send
    self.connect()
  File "/usr/lib64/python2.7/httplib.py", line 772, in connect
    self.timeout, self.source_address)
  File "/usr/lib64/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known

Isn't that a DNS failure? We have a known issue with DNS in that lab.

https://github.com/ceph/teuthology/pull/417

We've decided to use 'koji' everywhere we used the term 'brew'. So, now the yaml config will need a 'koji: <build_id>' stanza added to 'kernel' to enable this feature.