Project

General

Profile

Bug #18069

rados suite upgrade tests fail on xenial nodes

Added by John Spray over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
11/29/2016
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:

Description

For example:

info:   http://pulpito.ceph.com/teuthology-2016-11-28_11:00:15-rados-jewel-distro-basic-smithi/582105/
log:    http://qa-proxy.ceph.com/teuthology/teuthology-2016-11-28_11:00:15-rados-jewel-distro-basic-smithi/582105/
sentry: http://sentry.ceph.com/sepia/teuthology/?q=eb7ca1124d2945528e3886292faff070

    Failed to fetch package version from http://gitbuilder.ceph.com/ceph-deb-
    xenial-x86_64-basic/ref/v0.67.10/version

Presumably the Xenial gitbuilders don't have these historical tags

History

#1 Updated by John Spray over 2 years ago

Similar failures when other jobs try to get infernalis branch

#2 Updated by David Galloway over 2 years ago

Looks like somebody forced a rebuild on v0.67.10

infernalis is failing due to Error: Import No module named _markerlib
http://gitbuilder.sepia.ceph.com/gitbuilder-ceph-deb-xenial-amd64-basic/log.cgi?log=baf17c90980f3c61f75775f561ced5b2a1d2141c

#3 Updated by Sage Weil over 2 years ago

I forced the rebuild bc (I assumed) it wasn't actually built before. I suspect the workaround here is just to not run these ancient upgrade tests on xenial, though (os_type: centos). Right now on master we only do jewel -> kraken/master (hammer->jewel->x isn't merged yet). I don't think we need anything older than that?

#4 Updated by David Galloway over 2 years ago

Sage Weil wrote:

I forced the rebuild bc (I assumed) it wasn't actually built before.

Nah, that gitbuilder never built v0.67.10. When a new gitbuilder is set up, we trick the gitbuilder into thinking all builds have passed so it doesn't go back and try to rebuild every branch. We then go through and force rebuild on branches that make sense (master, hammer, etc.)

Is it even sane to expect v0.67.10 (released in 2014) to build on Xenial (released this year)? It's failing because ceph.com doesn't listen to git requests anymore. I guess it was when v0.67.10 was released. Either way, I don't think this build failure is distro-specific. It'd fail on any gitbuilder if we forced a rebuild.

I suspect the workaround here is just to not run these ancient upgrade tests on xenial, though (os_type: centos). Right now on master we only do jewel -> kraken/master (hammer->jewel->x isn't merged yet). I don't think we need anything older than that?

Sounds right to me but not really my call.

#5 Updated by David Galloway over 2 years ago

  • Related to Bug #18089: Various official builds missing from CI/Shaman added

#6 Updated by David Galloway over 2 years ago

distribute can be installed via easy_install but not pip.

(venv) root@gitbuilder-ceph-deb-xenial-amd64-basic:~/venv# pip install distribute
Collecting distribute
  Downloading distribute-0.7.3.zip (145kB)
    100% |████████████████████████████████| 153kB 2.8MB/s 
Requirement already satisfied: setuptools>=0.7 in ./lib/python2.7/site-packages (from distribute)
Building wheels for collected packages: distribute
  Running setup.py bdist_wheel for distribute ... error
  Complete output from command /root/venv/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-XVD1VT/distribute/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpMnUohqpip-wheel- --python-tag cp27:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-XVD1VT/distribute/setup.py", line 58, in <module>
      setuptools.setup(**setup_params)
    File "/usr/lib/python2.7/distutils/core.py", line 137, in setup
      ok = dist.parse_command_line()
    File "setuptools/dist.py", line 276, in parse_command_line
      result = _Distribution.parse_command_line(self)
    File "/usr/lib/python2.7/distutils/dist.py", line 467, in parse_command_line
      args = self._parse_command_opts(parser, args)
    File "setuptools/dist.py", line 602, in _parse_command_opts
      nargs = _Distribution._parse_command_opts(self, parser, args)
    File "/usr/lib/python2.7/distutils/dist.py", line 523, in _parse_command_opts
      cmd_class = self.get_command_class(command)
    File "setuptools/dist.py", line 406, in get_command_class
      ep.require(installer=self.fetch_build_egg)
    File "pkg_resources.py", line 2254, in require
      working_set.resolve(self.dist.requires(self.extras),env,installer)))
    File "pkg_resources.py", line 2471, in requires
      dm = self._dep_map
    File "pkg_resources.py", line 2682, in _dep_map
      self.__dep_map = self._compute_dependencies()
    File "pkg_resources.py", line 2699, in _compute_dependencies
      from _markerlib import compile as compile_marker
  ImportError: No module named _markerlib

  ----------------------------------------
  Failed building wheel for distribute
  Running setup.py clean for distribute
Failed to build distribute
Installing collected packages: distribute
  Running setup.py install for distribute ... done
Successfully installed distribute-0.7.3

(venv) root@gitbuilder-ceph-deb-xenial-amd64-basic:~/venv# easy_install distribute
Searching for distribute
Best match: distribute 0.7.3
Adding distribute 0.7.3 to easy-install.pth file

Using /root/venv/lib/python2.7/site-packages
Processing dependencies for distribute
Finished processing dependencies for distribute

(venv) root@gitbuilder-ceph-deb-xenial-amd64-basic:~/venv# pip install distribute
Requirement already satisfied: distribute in ./lib/python2.7/site-packages
Requirement already satisfied: setuptools>=0.7 in ./lib/python2.7/site-packages (from distribute)

#7 Updated by Sage Weil over 2 years ago

Yeah, I think the resolution is to just make upgardes from old versions avoid xenial. I've fixed the jewel hammer-x and clietn-upgrade tests in https://github.com/ceph/ceph-qa-suite/pull/1292.. if there are other tests just let me know, they're easy to fix!

#8 Updated by David Galloway over 2 years ago

  • Status changed from New to Resolved
  • Assignee set to Sage Weil

Also available in: Atom PDF