Ceph

Can you re-try with the wip-strict-debian-deps branch ( https://github.com/ceph/ceph/pull/7931 ) ?

2016-03-10T02:34:32.412 INFO:teuthology.orchestra.run.vpm133.stdout:Preparing to unpack .../ceph-common_10.0.4-1939-gdedcbc6-1trusty_amd64.deb ...
2016-03-10T02:34:33.553 INFO:teuthology.orchestra.run.vpm133.stdout:Unpacking ceph-common (10.0.4-1939-gdedcbc6-1trusty) over (9.2.1-8-g5d2693d-1trusty) ...
2016-03-10T02:34:34.228 INFO:teuthology.orchestra.run.vpm133.stderr:dpkg: error processing archive /var/cache/apt/archives/ceph-common_10.0.4-1939-gdedcbc6-1trusty_amd64.deb (--unpack):
2016-03-10T02:34:34.228 INFO:teuthology.orchestra.run.vpm133.stderr: trying to overwrite '/usr/lib/python2.7/dist-packages/ceph_daemon.py', which is also in package ceph 9.2.1-8-g5d2693d-1trusty

The errors seem to happen because files are in both ceph-common and ceph.

I just finished going over the teuthology.log. Please bear with me:

First we have the install task, which installs the 9.2.1 packages:

2016-03-10T02:33:15.085 INFO:teuthology.run_tasks:Running task install...

2016-03-10T02:33:15.511 INFO:teuthology.task.install:Installing packages: ceph, ceph-mds, ceph-common, ceph-fuse, ceph-test, radosgw, python-ceph, libcephfs1, libcephfs-java, libcephfs-jni, librados2, librbd1, rbd-fuse, librados2, librbd1 on remote deb x86_64
2016-03-10T02:33:15.511 INFO:teuthology.task.install:Pulling from http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/ref/infernalis

Running: u'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o
Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install
ceph=9.2.1-8-g5d2693d-1trusty ceph-mds=9.2.1-8-g5d2693d-1trusty
ceph-common=9.2.1-8-g5d2693d-1trusty ceph-fuse=9.2.1-8-g5d2693d-1trusty
ceph-test=9.2.1-8-g5d2693d-1trusty radosgw=9.2.1-8-g5d2693d-1trusty
python-ceph=9.2.1-8-g5d2693d-1trusty libcephfs1=9.2.1-8-g5d2693d-1trusty
libcephfs-java=9.2.1-8-g5d2693d-1trusty libcephfs-jni=9.2.1-8-g5d2693d-1trusty
librados2=9.2.1-8-g5d2693d-1trusty librbd1=9.2.1-8-g5d2693d-1trusty
rbd-fuse=9.2.1-8-g5d2693d-1trusty librados2=9.2.1-8-g5d2693d-1trusty
librbd1=9.2.1-8-g5d2693d-1trusty'

Then the ceph task is run (not shown).

Then the "install.upgrade" task is run:

2016-03-10T02:34:21.048 INFO:teuthology.run_tasks:Running task install.upgrade...

2016-03-10T02:34:21.188 INFO:teuthology.task.install:Upgrading ceph deb packages: ceph-mds, rbd-fuse, librbd1, ceph-fuse, python-ceph, ceph-common, libcephfs-java, ceph, libcephfs-jni, ceph-test, radosgw, librados2, libcephfs1

2016-03-10T02:34:26.522 INFO:teuthology.orchestra.run.vpm133:Running: u'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph-mds=10.0.4-1939-gdedcbc6-1trusty rbd-fuse=10.0.4-1939-gdedcbc6-1trusty librbd1=10.0.4-1939-gdedcbc6-1trusty ceph-fuse=10.0.4-1939-gdedcbc6-1trusty python-ceph=10.0.4-1939-gdedcbc6-1trusty ceph-common=10.0.4-1939-gdedcbc6-1trusty libcephfs-java=10.0.4-1939-gdedcbc6-1trusty ceph=10.0.4-1939-gdedcbc6-1trusty libcephfs-jni=10.0.4-1939-gdedcbc6-1trusty ceph-test=10.0.4-1939-gdedcbc6-1trusty radosgw=10.0.4-1939-gdedcbc6-1trusty librados2=10.0.4-1939-gdedcbc6-1trusty libcephfs1=10.0.4-1939-gdedcbc6-1trusty'

This would seem to be a problem. Why does the "install.upgrade" task do
"apt-get install" and not "apt-get upgrade" ?

It is installing the new versions over the old ones.

And here comes the first error:

2016-03-10T02:34:33.553 INFO:teuthology.orchestra.run.vpm133.stdout:Unpacking ceph-common (10.0.4-1939-gdedcbc6-1trusty) over (9.2.1-8-g5d2693d-1trusty) ...
2016-03-10T02:34:34.228 INFO:teuthology.orchestra.run.vpm133.stderr:dpkg: error processing archive /var/cache/apt/archives/ceph-common_10.0.4-1939-gdedcbc6-1trusty_amd64.deb (--unpack):
2016-03-10T02:34:34.228 INFO:teuthology.orchestra.run.vpm133.stderr: trying to overwrite '/usr/lib/python2.7/dist-packages/ceph_daemon.py', which is also in package ceph 9.2.1-8-g5d2693d-1trusty

This happens because the ceph 9.2.1 package is still installed when the ceph-common 10.0.4 package install attempt happens.

Looking at "debian.control" in jewel, the Replaces and Breaks for ceph-common are:

Replaces: ceph-common (<< 0.78-500), python-ceph (<< 0.92-1223),
          ceph-test (<< 0.94-1322)
Breaks: python-ceph (<< 0.92-1223), ceph-test (<< 0.94-1322)

I guess we need to add something like "Replaces: ceph (<< 10)" and "Breaks: ceph (<< 10)"

Why does the "install.upgrade" task do "apt-get install" and not "apt-get upgrade" ?

Excellent question! there's no good reason. it's just the way it's always been.

In run http://pulpito.ovh.sepia.ceph.com:8081/teuthology-2016-03-11_02:10:01-upgrade:infernalis-x-jewel-distro-basic-openstack/
Jobs: ['12817', '12829', '12819', '12826', '12823', '12827']
Logs: http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-03-11_02:10:01-upgrade:infernalis-x-jewel-distro-basic-openstack/12829/teuthology.log

2016-03-11T06:18:39.196 INFO:teuthology.orchestra.run.target089072.stderr:dpkg: error processing archive /var/cache/apt/archives/ceph-osd_10.0.4-2087-g740c52d-1trusty_amd64.deb (--unpack):
2016-03-11T06:18:39.196 INFO:teuthology.orchestra.run.target089072.stderr: trying to overwrite '/usr/share/man/man8/ceph-osd.8.gz', which is also in package ceph 9.2.1-8-g5d2693d-1trusty
....

2016-03-11T06:18:50.623 INFO:teuthology.orchestra.run.target089072.stderr:Errors were encountered while processing:
2016-03-11T06:18:50.624 INFO:teuthology.orchestra.run.target089072.stderr: /var/cache/apt/archives/ceph-common_10.0.4-2087-g740c52d-1trusty_amd64.deb
2016-03-11T06:18:50.624 INFO:teuthology.orchestra.run.target089072.stderr: /var/cache/apt/archives/ceph-base_10.0.4-2087-g740c52d-1trusty_amd64.deb
2016-03-11T06:18:50.624 INFO:teuthology.orchestra.run.target089072.stderr: /var/cache/apt/archives/ceph-mon_10.0.4-2087-g740c52d-1trusty_amd64.deb
2016-03-11T06:18:50.624 INFO:teuthology.orchestra.run.target089072.stderr: /var/cache/apt/archives/ceph-osd_10.0.4-2087-g740c52d-1trusty_amd64.deb
2016-03-11T06:18:50.891 INFO:teuthology.orchestra.run.target089072.stderr:E: Sub-process /usr/bin/dpkg returned an error code (1)