Changing the priority for this. kraken is completely broken even if we install differently. A ceph-deploy install breaks like this:
ceph-deploy install --release=kraken 2.node.a
[ceph_deploy.conf][DEBUG ] found configuration file at: /Users/alfredo/tmp/foo/cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.36): /Users/alfredo/.virtualenvs/ceph-deploy/bin/ceph-deploy install --release=kraken 2.node.a
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] testing : None
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1027ad638>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] dev_commit : None
[ceph_deploy.cli][INFO ] install_mds : False
[ceph_deploy.cli][INFO ] stable : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] adjust_repos : True
[ceph_deploy.cli][INFO ] func : <function install at 0x102708c80>
[ceph_deploy.cli][INFO ] install_all : False
[ceph_deploy.cli][INFO ] repo : False
[ceph_deploy.cli][INFO ] host : ['2.node.a']
[ceph_deploy.cli][INFO ] install_rgw : False
[ceph_deploy.cli][INFO ] install_tests : False
[ceph_deploy.cli][INFO ] repo_url : None
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] install_osd : False
[ceph_deploy.cli][INFO ] version_kind : stable
[ceph_deploy.cli][INFO ] install_common : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] dev : master
[ceph_deploy.cli][INFO ] nogpgcheck : False
[ceph_deploy.cli][INFO ] local_mirror : None
[ceph_deploy.cli][INFO ] release : kraken
[ceph_deploy.cli][INFO ] install_mon : False
[ceph_deploy.cli][INFO ] gpg_url : None
[ceph_deploy.install][DEBUG ] Installing stable version kraken on cluster ceph hosts 2.node.a
[ceph_deploy.install][DEBUG ] Detecting platform for host 2.node.a ...
[2.node.a][DEBUG ] connection detected need for sudo
[2.node.a][DEBUG ] connected to host: 2.node.a
[2.node.a][DEBUG ] detect platform information from remote host
[2.node.a][DEBUG ] detect machine type
[ceph_deploy.install][INFO ] Distro info: Ubuntu 16.04 xenial
[2.node.a][INFO ] installing Ceph on 2.node.a
[2.node.a][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q --no-install-recommends install ca-certificates apt-transport-https
[2.node.a][DEBUG ] Reading package lists...
[2.node.a][DEBUG ] Building dependency tree...
[2.node.a][DEBUG ] Reading state information...
[2.node.a][DEBUG ] ca-certificates is already the newest version (20160104ubuntu1).
[2.node.a][DEBUG ] The following packages will be upgraded:
[2.node.a][DEBUG ] apt-transport-https
[2.node.a][DEBUG ] 1 upgraded, 0 newly installed, 0 to remove and 129 not upgraded.
[2.node.a][DEBUG ] Need to get 25.7 kB of archives.
[2.node.a][DEBUG ] After this operation, 0 B of additional disk space will be used.
[2.node.a][DEBUG ] Get:1 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.12~ubuntu16.04.1 [25.7 kB]
[2.node.a][DEBUG ] Fetched 25.7 kB in 0s (57.8 kB/s)
(Reading database ... 61705 files and directories currently installed.)
[2.node.a][DEBUG ] Preparing to unpack .../apt-transport-https_1.2.12~ubuntu16.04.1_amd64.deb ...
[2.node.a][DEBUG ] Unpacking apt-transport-https (1.2.12~ubuntu16.04.1) over (1.2.10ubuntu1) ...
[2.node.a][DEBUG ] Setting up apt-transport-https (1.2.12~ubuntu16.04.1) ...
[2.node.a][INFO ] Running command: sudo wget -O release.asc https://download.ceph.com/keys/release.asc
[2.node.a][WARNIN] --2016-10-17 15:01:05-- https://download.ceph.com/keys/release.asc
[2.node.a][WARNIN] Resolving download.ceph.com (download.ceph.com)... 173.236.253.173, 2607:f298:6050:51f3:f816:3eff:fe71:9135
[2.node.a][WARNIN] Connecting to download.ceph.com (download.ceph.com)|173.236.253.173|:443... connected.
[2.node.a][WARNIN] HTTP request sent, awaiting response... 200 OK
[2.node.a][WARNIN] Length: 1645 (1.6K) [application/octet-stream]
[2.node.a][WARNIN] Saving to: 'release.asc'
[2.node.a][WARNIN]
[2.node.a][WARNIN] 0K . 100% 228M=0s
[2.node.a][WARNIN]
[2.node.a][WARNIN] 2016-10-17 15:01:06 (228 MB/s) - 'release.asc' saved [1645/1645]
[2.node.a][WARNIN]
[2.node.a][INFO ] Running command: sudo apt-key add release.asc
[2.node.a][DEBUG ] OK
[2.node.a][DEBUG ] add deb repo to /etc/apt/sources.list.d/
[2.node.a][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q update
[2.node.a][DEBUG ] Get:1 https://download.ceph.com/debian-kraken xenial InRelease [8,491 B]
[2.node.a][DEBUG ] Hit:2 http://us.archive.ubuntu.com/ubuntu xenial InRelease
[2.node.a][DEBUG ] Get:3 https://download.ceph.com/debian-kraken xenial/main amd64 Packages [11.6 kB]
[2.node.a][DEBUG ] Get:4 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]
[2.node.a][DEBUG ] Get:5 https://download.ceph.com/debian-kraken xenial/main i386 Packages [788 B]
[2.node.a][DEBUG ] Get:6 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
[2.node.a][DEBUG ] Hit:7 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
[2.node.a][DEBUG ] Fetched 211 kB in 1s (159 kB/s)
[2.node.a][DEBUG ] Reading package lists...
[2.node.a][WARNIN] W: https://download.ceph.com/debian-kraken/dists/xenial/InRelease: Signature by key 08B73419AC32B4E966C1A330E84AC2C0460F3994 uses weak digest algorithm (SHA1)
[2.node.a][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get --assume-yes -q --no-install-recommends install -o Dpkg::Options::=--force-confnew ceph-osd ceph-mds ceph-mon radosgw
[2.node.a][DEBUG ] Reading package lists...
[2.node.a][DEBUG ] Building dependency tree...
[2.node.a][DEBUG ] Reading state information...
[2.node.a][DEBUG ] The following additional packages will be installed:
[2.node.a][DEBUG ] ceph-base ceph-common libcephfs1 librados2 libradosstriper1 librbd1 librgw2
[2.node.a][DEBUG ] python-cephfs python-rados python-rbd
[2.node.a][DEBUG ] Recommended packages:
[2.node.a][DEBUG ] btrfs-tools ntp | time-daemon ceph-fuse
[2.node.a][DEBUG ] The following packages will be upgraded:
[2.node.a][DEBUG ] ceph-base ceph-common ceph-mds ceph-mon ceph-osd libcephfs1 librados2
[2.node.a][DEBUG ] libradosstriper1 librbd1 librgw2 python-cephfs python-rados python-rbd
[2.node.a][DEBUG ] radosgw
[2.node.a][DEBUG ] 14 upgraded, 0 newly installed, 0 to remove and 129 not upgraded.
[2.node.a][DEBUG ] Need to get 79.6 MB of archives.
[2.node.a][DEBUG ] After this operation, 50.3 MB disk space will be freed.
[2.node.a][DEBUG ] Get:1 https://download.ceph.com/debian-kraken xenial/main amd64 ceph-osd amd64 11.0.2-1xenial [11.2 MB]
[2.node.a][DEBUG ] Get:2 https://download.ceph.com/debian-kraken xenial/main amd64 ceph-mon amd64 11.0.2-1xenial [2,841 kB]
[2.node.a][DEBUG ] Get:3 https://download.ceph.com/debian-kraken xenial/main amd64 ceph-mds amd64 11.0.2-1xenial [2,769 kB]
[2.node.a][DEBUG ] Get:4 https://download.ceph.com/debian-kraken xenial/main amd64 libradosstriper1 amd64 11.0.2-1xenial [1,515 kB]
[2.node.a][DEBUG ] Get:5 https://download.ceph.com/debian-kraken xenial/main amd64 radosgw amd64 11.0.2-1xenial [3,973 kB]
[2.node.a][DEBUG ] Get:6 https://download.ceph.com/debian-kraken xenial/main amd64 ceph-base amd64 11.0.2-1xenial [34.5 MB]
[2.node.a][DEBUG ] Get:7 https://download.ceph.com/debian-kraken xenial/main amd64 ceph-common amd64 11.0.2-1xenial [15.3 MB]
[2.node.a][DEBUG ] Get:8 https://download.ceph.com/debian-kraken xenial/main amd64 python-rados amd64 11.0.2-1xenial [694 kB]
[2.node.a][DEBUG ] Get:9 https://download.ceph.com/debian-kraken xenial/main amd64 librbd1 amd64 11.0.2-1xenial [1,235 kB]
[2.node.a][DEBUG ] Get:10 https://download.ceph.com/debian-kraken xenial/main amd64 librgw2 amd64 11.0.2-1xenial [1,805 kB]
[2.node.a][DEBUG ] Get:11 https://download.ceph.com/debian-kraken xenial/main amd64 librados2 amd64 11.0.2-1xenial [1,543 kB]
[2.node.a][DEBUG ] Get:12 https://download.ceph.com/debian-kraken xenial/main amd64 python-cephfs amd64 11.0.2-1xenial [230 kB]
[2.node.a][DEBUG ] Get:13 https://download.ceph.com/debian-kraken xenial/main amd64 libcephfs1 amd64 11.0.2-1xenial [1,638 kB]
[2.node.a][DEBUG ] Get:14 https://download.ceph.com/debian-kraken xenial/main amd64 python-rbd amd64 11.0.2-1xenial [338 kB]
[2.node.a][DEBUG ] Fetched 79.6 MB in 12s (6,382 kB/s)
(Reading database ... 61705 files and directories currently installed.)
[2.node.a][DEBUG ] Preparing to unpack .../ceph-osd_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking ceph-osd (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../ceph-mon_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking ceph-mon (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../ceph-mds_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking ceph-mds (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../libradosstriper1_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking libradosstriper1 (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../radosgw_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking radosgw (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../ceph-base_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking ceph-base (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../ceph-common_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking ceph-common (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../python-rados_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking python-rados (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../librbd1_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking librbd1 (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../librgw2_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking librgw2 (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../librados2_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking librados2 (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../python-cephfs_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking python-cephfs (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../libcephfs1_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking libcephfs1 (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Preparing to unpack .../python-rbd_11.0.2-1xenial_amd64.deb ...
[2.node.a][DEBUG ] Unpacking python-rbd (11.0.2-1xenial) over (10.2.3-1xenial) ...
[2.node.a][DEBUG ] Processing triggers for ureadahead (0.100.0-19) ...
[2.node.a][DEBUG ] Processing triggers for man-db (2.7.5-1) ...
[2.node.a][DEBUG ] Processing triggers for libc-bin (2.23-0ubuntu3) ...
[2.node.a][DEBUG ] Processing triggers for systemd (229-4ubuntu4) ...
[2.node.a][DEBUG ] Setting up librados2 (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up librbd1 (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up libradosstriper1 (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up python-rados (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up libcephfs1 (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up python-cephfs (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up python-rbd (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up ceph-common (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting system user ceph properties..usermod: no changes
[2.node.a][DEBUG ] ..done
[2.node.a][DEBUG ] Fixing /var/run/ceph ownership....done
[2.node.a][DEBUG ] [/usr/lib/tmpfiles.d/var.conf:14] Duplicate line for path "/var/log", ignoring.
[2.node.a][DEBUG ] rbdmap.service is a disabled or a static unit, not starting it.
[2.node.a][DEBUG ] Setting up ceph-base (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Installing new version of config file /etc/init.d/ceph ...
[2.node.a][DEBUG ] Setting up ceph-osd (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up ceph-mon (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up ceph-mds (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up librgw2 (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Setting up radosgw (11.0.2-1xenial) ...
[2.node.a][DEBUG ] Installing new version of config file /etc/init.d/radosgw ...
[2.node.a][DEBUG ] Processing triggers for libc-bin (2.23-0ubuntu3) ...
[2.node.a][DEBUG ] Processing triggers for ureadahead (0.100.0-19) ...
[2.node.a][DEBUG ] Processing triggers for systemd (229-4ubuntu4) ...
[2.node.a][INFO ] Running command: sudo ceph --version
[2.node.a][WARNIN] Traceback (most recent call last):
[2.node.a][WARNIN] File "/usr/bin/ceph", line 101, in <module>
[2.node.a][WARNIN] import rados
[2.node.a][WARNIN] ImportError: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
[2.node.a][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph --version
This is with 11.0.2 which was just released.
There are a lot of tools (e.g. ceph-ansible) that require correct output from `ceph --version`)
Also note that this time it wasn't just a single component getting installed