Tools » teuthology

I see the same error when trying with a CentOS-7

The error does not show when using https://github.com/dachary/teuthology/tree/openstack instead of https://github.com/dachary/teuthology/tree/openstack-new which is a rebase of the branch with master.

The openstack branch does not including the following commits which are found in master:

04ccc91 Set network-related hostvars if necessary
c68ae24 Add Remote.{ip_address, interface, cidr}
a8da245 Use misc.get_scratch_devices() when necessary
f4e3859 Refactor hosts file generation
3cfdef7 Add a warning when gitbuilder 404s
78e0751 get_distro_version: Always return a string
570b7ad Unpin requests version
1562145 Add ceph_ansible task
b323c0b ansible: Patch more aggressively in unit tests
059008a ansible: Don't modify plays
b4190d0 ansible: Run ansible-playbook in repo dir
d02dbfd ansible: Place generated playbooks in repo dir
44e0674 ansible: write hosts file in _write_hosts_file()
609ae1b worker: When running a job, log the full command
b845d5f Fix bug when running archiveless jobs
20a9340 task/install: only install debug packages if debuginfo: true
2ec4b8e Set config.archive_path if it's not set already
2dfe3fd Collect timing data for task execution
2564eb3 Add teuthology.timer
e194655 matrix: reimpliment Sum
1354a17 matrix: fix Product index offset and minscanlen value
6d81bb1 suite: add randomized tests verifying subset coverage
df5c562 matrix: add __str__
0dc74a6 suite: ensure that convolutions are cycled out to satisfy mincyclicity
853f4a4 suite: fix rounding bug in _build_matrix
c7e070b task/internal: list leftover testdir files
faf9ae6 Revert "misc: ignore some unusable block devices" 
5afcede lock_machines(): Use dead status for lock failures
b047ed6 ProvisionOpenStack: raise QuotaExceededError
e826506 nuke.py: Fix some spelling/capitalization errors.
2daa8de nuke.py: try to undo multipath devices
7799c4c Update for openstack volume field name changes
0698f85 Add QuotaExceededError
61ecddf Install package dependencies in TravisCI
7bb6c65 Drop --no-use-wheel when invoking pip
0f927dc Unpin setuptools
3e1ccb9 Drop obsolete jenkins/ dir
f1bc998 Require mysqlclient
9200192 Require libvirt devel package on Linux distros
574221e Add libffi for MacOS
f2863c5 Require libvirt-python
6142714 Rebuild virtualenvs that use system site-packages
c43be2c Drop setuptools version restriction
20bf07a Always upgrade pip and setuptools
2a07746 Drop --system-site-packages
4843b92 Speed up these tests by ~20x
a3247a8 Remove RHEL from downburst version list
1dcaa6b openstack: a failure to rename a volume is not fatal
1a71f30 nuke: Expect json output from openstack commands
064a6ca Ignore few se denials in audit log parsing
69d85da Add user-data for CentOS 7.2
fda8c7e Correct CentOS 7.0/7.1 images and add 7.2
998dddb Use remote.inventory_info['os_version']
f22d0a3 valgrind.supp: suppress helgrind warnings for external libraries
0026c00 Fix regression in flock usage
7f294a4 doc: add descriptive comment at top of hammer.sh script
6e092e1 better logging in packaging._get_response
6b1e8a2 task.install: fix how the config option wait_for_package is set
de3773d packaging.GitbuilderProject will retry now when fetching package versions
8f171eb openstack: flock timeout on openstack server create
6b76d6b openstack: nuke must use ID and not name
343cfe6 README: fix rst syntax for sub-bullets
2e5f1c3 Downburst: Remove the test user's password
c33b30c Downburst: Make MAC setting persistent
019d09b openstack: support CentOS 7.1
dc3b842 doc: only OpenStack with OVH is supported
09da1aa Block requests==2.9.0
1446bcf clock: get the servers to use for ntpdate from ntp.conf
80b2241 ansible: include the contents of the failure log in the exception
57ce714 misc: ignore some unusable block devices
47adccd openstack: switch the default json output to dictionary
1d998e0 openstack: package data files
42bcfd3 suite: honor suite_verify_ceph_hash on --dry-run
6d3022a openstack: remove dead code
3a67e3f openstack: keep image private
fec70f9 openstack: remove bugous static IP regexp
8d79615 dnsmasq is not actually restarting
ae69ebb tests: openstack-integration is too agressive
ed8a15d tests: delegate integration tests to OpenStack
6df4902 ceph.conf: mon osd reporter subtree level = osd
a165cf2 amendment misunderstood lock server and results server address
e9e9898 suite: --wait to block until the suite finishes
6d77b4a misc: get_pkg_type to infer the package time from the os_type
361c6f7 openstack: update ~/.teuthology with --git-ceph{,qa-suite}-url
af0d1cd prune: Process log dirs in order of ctime

If I read correctly ansible-playbook

    # read vault_pass from a file
    if not options.ask_vault_pass and options.vault_password_file:
        vault_pass = utils.read_vault_file(options.vault_password_file)

(virtualenv)ubuntu@teuthology:~/src/ceph-cm-ansible_master$ git grep vault
ansible.cfg:vault_password_file = ~/.vault_pass.txt

This is the diff of what seems related to ansible: git diff openstack openstack-new (https://github.com/dachary/teuthology/compare/openstack...dachary:openstack-new). Where openstack works and openstack-new does not (with the error above). Whatever the problem is, it comes from this diff. Both pin the same version of ansible.

diff --git a/teuthology/task/ansible.py b/teuthology/task/ansible.py
index f95a7fe..f290682 100644
--- a/teuthology/task/ansible.py
+++ b/teuthology/task/ansible.py
@@ -199,28 +199,33 @@ class Ansible(Task):
             inventory.append('[{0}]'.format(self.inventory_group))
         inventory.extend(hostnames + [''])
         hosts_str = '\n'.join(inventory)
+        self.inventory = self._write_hosts_file(hosts_str)
+        self.generated_inventory = True
+
+    def _write_hosts_file(self, content):
+        """ 
+        Actually write the hosts file
+        """ 
         hosts_file = NamedTemporaryFile(prefix="teuth_ansible_hosts_",
                                         delete=False)
-        hosts_file.write(hosts_str)
+        hosts_file.write(content)
         hosts_file.flush()
-        self.generated_inventory = True
-        self.inventory = hosts_file.name
+        return hosts_file.name

     def generate_playbook(self):
         """ 
         Generate a playbook file to use. This should not be called if we're
         using an existing file.
         """ 
-        for play in self.playbook:
-            # Ensure each play is applied to all hosts mentioned in the --limit
-            # flag we specify later
-            play['hosts'] = 'all'
         pb_buffer = StringIO()
         pb_buffer.write('---\n')
         yaml.safe_dump(self.playbook, pb_buffer)
         pb_buffer.seek(0)
-        playbook_file = NamedTemporaryFile(prefix="teuth_ansible_playbook_",
-                                           delete=False)
+        playbook_file = NamedTemporaryFile(
+            prefix="teuth_ansible_playbook_",
+            dir=self.repo_path,
+            delete=False,
+        )
         playbook_file.write(pb_buffer.read())
         playbook_file.flush()
         self.playbook_file = playbook_file
@@ -248,6 +253,7 @@ class Ansible(Task):
         out_log = self.log.getChild('out')
         out, status = pexpect.run(
             command,
+            cwd=self.repo_path,
             logfile=_logfile or LoggerFile(out_log, logging.INFO),
             withexitstatus=True,
             timeout=None,
@@ -272,7 +278,7 @@ class Ansible(Task):
                         self.failure_log.name,
                     )
                 )
-                failures = fail_log
+                failures = fail_log.read().replace('\n', '')

         if failures:
             self._archive_failures()

It probably comes from this commit

commit b4190d05aae039c2a391b232a867f9fe653fc2b9
Author: Zack Cerza <zack@redhat.com>
Date:   Wed Feb 10 16:11:21 2016 -0700

    ansible: Run ansible-playbook in repo dir

    Signed-off-by: Zack Cerza <zack@redhat.com>

diff --git a/teuthology/task/ansible.py b/teuthology/task/ansible.py
index 7502aa5..708cc3b 100644
--- a/teuthology/task/ansible.py
+++ b/teuthology/task/ansible.py
@@ -257,6 +257,7 @@ class Ansible(Task):
         out_log = self.log.getChild('out')
         out, status = pexpect.run(
             command,
+            cwd=self.repo_path,
             logfile=_logfile or LoggerFile(out_log, logging.INFO),
             withexitstatus=True,
             timeout=None,
diff --git a/teuthology/test/task/test_ansible.py b/teuthology/test/task/test_ansible.py
index b35dcac..3c325b5 100644
--- a/teuthology/test/task/test_ansible.py
+++ b/teuthology/test/task/test_ansible.py
@@ -264,6 +264,7 @@ class TestAnsibleTask(TestTask):
                 task.execute_playbook(_logfile=logger)
             m_run.assert_called_once_with(
                 ' '.join(args),
+                cwd=task.repo_path,
                 logfile=logger,
                 withexitstatus=True,
                 timeout=None,

The right fix is to create the vault_passwd file. I have to figure out how to do that.