Project

General

Profile

Bug #10893

Install failing on RHEL-family machines (due to mixed package sources?)

Added by Greg Farnum over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Immediate
Assignee:
-
Category:
-
Target version:
-
Start date:
02/16/2015
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:

Description

2015-02-14T16:01:50.828 INFO:teuthology.orchestra.run.plana77:Running: 'sudo yum install ceph-radosgw-0.92 -y'
2015-02-14T16:01:50.835 INFO:teuthology.orchestra.run.plana64.stdout:--> Finished Dependency Resolution
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stdout: You could try using --skip-broken to work around the problem
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:Error: Package: 1:python-rbd-0.80.7-0.4.el7.x86_64 (epel)
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:           Requires: librbd1 = 1:0.80.7
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:           Installing: 1:librbd1-0.92-1099.gf62fb72.el7.x86_64 (Ceph)
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:               librbd1 = 1:0.92-1099.gf62fb72.el7
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:Error: Package: 1:python-cephfs-0.80.7-0.4.el7.x86_64 (epel)
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:           Requires: libcephfs1 = 1:0.80.7
2015-02-14T16:01:50.839 INFO:teuthology.orchestra.run.plana64.stderr:           Available: 1:libcephfs1-0.92-1099.gf62fb72.el7.x86_64 (Ceph)
2015-02-14T16:01:50.840 INFO:teuthology.orchestra.run.plana64.stderr:               libcephfs1 = 1:0.92-1099.gf62fb72.el7
2015-02-14T16:01:50.840 INFO:teuthology.orchestra.run.plana64.stderr:Error: Package: 1:python-rados-0.80.7-0.4.el7.x86_64 (epel)
2015-02-14T16:01:50.840 INFO:teuthology.orchestra.run.plana64.stderr:           Requires: librados2 = 1:0.80.7
2015-02-14T16:01:50.840 INFO:teuthology.orchestra.run.plana64.stderr:           Installing: 1:librados2-0.92-1099.gf62fb72.el7.x86_64 (Ceph)
2015-02-14T16:01:50.840 INFO:teuthology.orchestra.run.plana64.stderr:               librados2 = 1:0.92-1099.gf62fb72.el7
2015-02-14T16:01:51.169 INFO:teuthology.orchestra.run.plana77.stdout:Loaded plugins: priorities, product-id, subscription-manager
2015-02-14T16:01:51.216 INFO:teuthology.orchestra.run.plana77.stdout:This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
2015-02-14T16:01:51.745 INFO:teuthology.orchestra.run.plana64.stdout: You could try running: rpm -Va --nofiles --nodigest
2015-02-14T16:01:51.905 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/task/install.py", line 474, in _update_rpm_package_list_and_install
    remote.run(args=['sudo', 'yum', 'install', pkg2add, '-y'])
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 137, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed on plana64 with status 1: 'sudo yum install ceph-radosgw-0.92 -y'

This looks to me like #10476, and caused a rash of failures in suites run over the weekend. For instance:
http://pulpito.ceph.com/teuthology-2015-02-11_23:10:01-knfs-master-testing-basic-multi/753693/
http://pulpito.ceph.com/teuthology-2015-02-12_23:04:02-fs-hammer-testing-basic-multi/755674/
http://pulpito.ceph.com/teuthology-2015-02-13_23:04:03-fs-master-testing-basic-multi/756522/
There are more, but I got tired of copying! :)

History

#1 Updated by Greg Farnum over 4 years ago

  • Priority changed from Normal to Immediate

#2 Updated by Ken Dreyer over 4 years ago

The downstream packages in Fedora and EPEL have split the main "python-ceph" RPM into separate modules: "python-rados", "python-rbd", "python-cephfs". This causes integration issues between EPEL and the upstream Ceph RPMs.

The change was proposed upstream, but it has not yet been merged: https://github.com/ceph/ceph/pull/3307

The immediate fix for Teuthology is to set check_obsoletes in yum's priorities.conf:

echo 'check_obsoletes = 1' >> /etc/yum/pluginconf.d/priorities.conf

The longer-term fix is to merge that pull request 3307 upstream so we are aligned with downstream.

#3 Updated by Ken Dreyer over 4 years ago

Note that if we add check_obsoletes to priorities.conf, I'm pretty sure that's not something we'd need to revert later. (Particularly since ceph-deploy is setting that flag for users, too)

#4 Updated by Zack Cerza over 4 years ago

We got together to discuss this, and here's what we decided:

Since this is blocking testing and therefore merges, Ken is going to take over the ceph PR. I'm going to implement check_obsoletes in teuthology; I'll likely file a separate ticket for that.

#5 Updated by Ken Dreyer over 4 years ago

I've filed https://bugzilla.redhat.com/1193182 "ceph has unversioned obsoletes", in order to get this fixed downstream in EPEL. Boris, if the patch in that Bugzilla ticket looks good to you, please feel free to merge it into EPEL 7 and push out a new build there.

As Zack said, I'm hoping you don't mind if I go ahead and fix up https://github.com/ceph/ceph/pull/3307 myself. It would be good to get it into a wip- branch in ceph.git so it can be tested through gitbuilder.

In that 3307 PR, Sage mentioned that we'll need to alter https://github.com/ceph/ceph-qa-chef.git as well. I'm not 100% sure what changes we need there for that. If someone beats me to that, or at least gives me clues, I'd appreciate it :)

#8 Updated by Zack Cerza over 4 years ago

  • Status changed from Verified to Testing

We think the teuthology change will be enough to fix this; the packaging work is ongoing

#9 Updated by Ken Dreyer over 4 years ago

In order to keep this Teuthology implementation issue separate from the packaging work that needs to get merged to ceph.git, I'm going to track the packaging work in these two tickets:

  • #10884 "ceph-devel should be split into separate -devel subpackages"
  • #10897 "python-ceph should be split out into separate python-* packages"

#10 Updated by Zack Cerza over 4 years ago

  • Status changed from Testing to Resolved

Also available in: Atom PDF