Project

General

Profile

Actions

Bug #16860

closed

'sudo yum install ceph-radosgw -y' fails jobs in rados run

Added by Yuri Weinstein over 7 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/yuriw-2016-07-29_08:07:15-rados-wip-yuri-testing2_2016_7_29-distro-basic-smithi/
Jobs: seems like all centos
Logs: http://qa-proxy.ceph.com/teuthology/yuriw-2016-07-29_08:07:15-rados-wip-yuri-testing2_2016_7_29-distro-basic-smithi/339542/teuthology.log

2016-07-29T11:03:39.664 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Skipping task due to undefined Error, in the future this 
2016-07-29T11:03:39.677 INFO:teuthology.task.ansible.out:

2016-07-29T11:03:39.678 INFO:teuthology.task.ansible.out:will be a fatal error.: 'ceph_dependency_packages_to_remove' is undefined.

....

2016-07-29T11:13:04.706 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Skipping task due to undefined Error, in the future this 
2016-07-29T11:13:04.752 INFO:teuthology.task.ansible.out:

2016-07-29T11:13:04.753 INFO:teuthology.task.ansible.out:will be a fatal error.: 'non_aarch64_packages_to_upgrade' is undefined.

.............

2016-07-29T11:15:29.986 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Skipping task due to undefined Error, in the future this 
2016-07-29T11:15:30.091 INFO:teuthology.task.ansible.out:
will be a fatal error.: 'non_aarch64_packages' is undefined.
This feature will 
be removed in a future release. Deprecation warnings can be disabled by setting 
deprecation_warnings=False in ansible.cfg.

.............

2016-07-29T11:23:07.688 INFO:teuthology.orchestra.run.smithi029.stderr:Error: Package: 1:ceph-base-11.0.0-1084.gf2349d2.x86_64 (Ceph)
2016-07-29T11:23:07.689 INFO:teuthology.orchestra.run.smithi029.stderr:           Requires: libliblua.so()(64bit)
2016-07-29T11:23:07.689 INFO:teuthology.orchestra.run.smithi029.stdout: You could try using --skip-broken to work around the problem
2016-07-29T11:23:08.018 INFO:teuthology.orchestra.run.smithi029.stdout: You could try running: rpm -Va --nofiles --nodigest
2016-07-29T11:23:08.298 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-base.
2016-07-29T11:23:08.306 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-base_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:08.357 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-base (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:13.553 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-mon.
2016-07-29T11:23:13.561 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-mon_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:13.611 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-mon (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:14.327 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-osd.
2016-07-29T11:23:14.333 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-osd_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:14.383 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-osd (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:15.529 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph.
2016-07-29T11:23:15.606 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:15.609 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:15.856 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-fuse.
2016-07-29T11:23:15.881 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-fuse_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:15.915 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-fuse (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:16.518 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-mds.
2016-07-29T11:23:16.527 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-mds_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:16.578 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-mds (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:17.171 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package ceph-test.
2016-07-29T11:23:17.182 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../ceph-test_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:17.230 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking ceph-test (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:19.959 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package libcephfs-jni.
2016-07-29T11:23:19.968 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../libcephfs-jni_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:20.017 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking libcephfs-jni (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:20.470 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package libcephfs-java.
2016-07-29T11:23:20.483 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../libcephfs-java_11.0.0-1084-gf2349d2-1trusty_all.deb ...
2016-07-29T11:23:20.583 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking libcephfs-java (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:20.900 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package librgw2.
2016-07-29T11:23:20.906 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../librgw2_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:20.956 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking librgw2 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:21.400 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package python-ceph.
2016-07-29T11:23:21.409 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../python-ceph_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:21.460 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking python-ceph (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:21.883 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package radosgw.
2016-07-29T11:23:21.887 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../radosgw_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:21.937 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking radosgw (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:22.566 INFO:teuthology.orchestra.run.smithi041.stdout:Selecting previously unselected package rbd-fuse.
2016-07-29T11:23:22.620 INFO:teuthology.orchestra.run.smithi041.stdout:Preparing to unpack .../rbd-fuse_11.0.0-1084-gf2349d2-1trusty_amd64.deb ...
2016-07-29T11:23:22.632 INFO:teuthology.orchestra.run.smithi041.stdout:Unpacking rbd-fuse (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:22.819 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
2016-07-29T11:23:23.195 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for ureadahead (0.100.0-16) ...
2016-07-29T11:23:23.425 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up librados2 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:23.666 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up librbd1 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:23.867 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up libradosstriper1 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:24.097 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up python-rados (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:24.245 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up libcephfs1 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:24.430 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up python-cephfs (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:24.673 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up python-rbd (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:24.816 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-common (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:25.376 INFO:teuthology.orchestra.run.smithi041.stderr:usermod: no changes
2016-07-29T11:23:25.376 INFO:teuthology.orchestra.run.smithi041.stdout:Setting system user ceph properties....done
2016-07-29T11:23:25.555 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-fuse (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:25.681 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up libcephfs-jni (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:25.861 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up libcephfs-java (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:26.008 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up librgw2 (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:26.210 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up python-ceph (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:26.361 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up rbd-fuse (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:26.512 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for ureadahead (0.100.0-16) ...
2016-07-29T11:23:26.647 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-test (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:26.789 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up radosgw (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:27.279 INFO:teuthology.orchestra.run.smithi041.stdout:radosgw-all start/running
2016-07-29T11:23:27.335 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-base (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:27.696 INFO:teuthology.orchestra.run.smithi041.stdout:ceph-all start/running
2016-07-29T11:23:27.873 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for ureadahead (0.100.0-16) ...
2016-07-29T11:23:28.050 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-mds (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:28.420 INFO:teuthology.orchestra.run.smithi041.stdout:ceph-mds-all start/running
2016-07-29T11:23:28.469 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-mon (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:28.795 INFO:teuthology.orchestra.run.smithi041.stdout:ceph-mon-all start/running
2016-07-29T11:23:28.847 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph-osd (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:29.436 INFO:teuthology.orchestra.run.smithi041.stdout:ceph-osd-all start/running
2016-07-29T11:23:29.595 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for ureadahead (0.100.0-16) ...
2016-07-29T11:23:29.846 INFO:teuthology.orchestra.run.smithi041.stdout:Setting up ceph (11.0.0-1084-gf2349d2-1trusty) ...
2016-07-29T11:23:29.989 INFO:teuthology.orchestra.run.smithi041.stdout:Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
2016-07-29T11:23:30.964 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 242, in _update_rpm_package_list_and_install
    remote.run(args=['sudo', 'yum', 'install', cpack, '-y'])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 394, in run
    log.info("Running command with timeout %d", timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 121, in wait
    stream_log,
CommandFailedError: Command failed on smithi029 with status 1: 'sudo yum install ceph-radosgw -y'
2016-07-29T11:23:30.967 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/contextutil.py", line 30, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 606, in install
    install_packages(ctx, package_list, config)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 342, in install_packages
    ctx, remote, pkgs[system_type], config)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 242, in _update_rpm_package_list_and_install
    remote.run(args=['sudo', 'yum', 'install', cpack, '-y'])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 394, in run
    log.info("Running command with timeout %d", timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 121, in wait
    stream_log,
CommandFailedError: Command failed on smithi029 with status 1: 'sudo yum install ceph-radosgw -y'
2016-07-29T11:23:30.968 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 69, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 1269, in task
    lambda: ship_utilities(ctx=ctx, config=None),
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/contextutil.py", line 30, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 606, in install
    install_packages(ctx, package_list, config)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 342, in install_packages
    ctx, remote, pkgs[system_type], config)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 242, in _update_rpm_package_list_and_install
    remote.run(args=['sudo', 'yum', 'install', cpack, '-y'])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 394, in run
    log.info("Running command with timeout %d", timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 121, in wait
    stream_log,
CommandFailedError: Command failed on smithi029 with status 1: 'sudo yum install ceph-radosgw -y'
2016-07-29T11:23:32.259 ERROR:teuthology.run_tasks: Sentry event: http://sentry.ceph.com/sepia/teuthology/?q=e6a917e74a784bf78995f5d40bf94d08
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 69, in run_tasks
    manager.__enter__()
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 1269, in task
    lambda: ship_utilities(ctx=ctx, config=None),
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/contextutil.py", line 30, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 606, in install
    install_packages(ctx, package_list, config)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 342, in install_packages
    ctx, remote, pkgs[system_type], config)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 83, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 242, in _update_rpm_package_list_and_install
    remote.run(args=['sudo', 'yum', 'install', cpack, '-y'])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 196, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 394, in run
    log.info("Running command with timeout %d", timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 121, in wait
    stream_log,
CommandFailedError: Command failed on smithi029 with status 1: 'sudo yum install ceph-radosgw -y'

Actions #1

Updated by Yuri Weinstein over 7 years ago

  • Assignee set to Noah Watkins
Actions #2

Updated by Yuri Weinstein over 6 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF