Project

General

Profile

Feature #9325

ceph-deploy: test public and private networks

Added by Tamilarasi muthamizhan over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Reviewed:
Affected Versions:

Description

ceph-deploy new --public-network 0.0.0.0/0 --cluster-network 0.0.0.0/0 node1 node2

test case : deploy a cluster with 2 nodes on two different networks.

History

#1 Updated by Tamilarasi muthamizhan over 9 years ago

  • Target version set to sprint13

#2 Updated by Tamilarasi muthamizhan over 9 years ago

  • Assignee changed from Sandon Van Ness to Tamilarasi muthamizhan

#3 Updated by Tamilarasi muthamizhan over 9 years ago

  • Assignee changed from Tamilarasi muthamizhan to Anonymous

#4 Updated by Anonymous over 9 years ago

  • Status changed from New to Need More Info
  • Assignee changed from Anonymous to Alfredo Deza

Running the following:

ceph-deploy purgedata plana57.front.sepia.ceph.com
ceph-deploy forgetkeys
ceph-deploy purge plana57.front.sepia.ceph.com
ceph-deploy new --cluster-network 10.214.132.21/1 plana57.front.sepia.ceph.com
ceph-deploy install plana57.front.sepia.ceph.com

Produces the following:

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/wusui/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.13): ./virtualenv/bin/ceph-deploy new --public-network 10.214.132.21/0 plana57.front.sepia.ceph.com
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO  ] making sure passwordless SSH succeeds
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: aardvark 
[plana57.front.sepia.ceph.com][INFO  ] Running command: ssh -CT -o BatchMode=yes plana57.front.sepia.ceph.com
[ceph_deploy.new][WARNIN] could not connect via SSH
[ceph_deploy.new][INFO  ] will connect again with password prompt
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[ceph_deploy.new][INFO  ] adding public keys to authorized_keys
[plana57.front.sepia.ceph.com][DEBUG ] append contents to file
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[plana57.front.sepia.ceph.com][DEBUG ] find the location of an executable
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip link show
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip addr show
[plana57.front.sepia.ceph.com][DEBUG ] IP addresses found: ['10.214.132.21', '10.214.145.21']
[ceph_deploy.new][DEBUG ] Resolving host plana57.front.sepia.ceph.com
[ceph_deploy.new][DEBUG ] Monitor plana57 at 10.214.132.21
[ceph_deploy.new][DEBUG ] Monitor initial members are ['plana57']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['10.214.132.21']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
wusui@aardvark:~/src/ceph-deploy$ ./virtualenv/bin/ceph-deploy install plana57.front.sepia.ceph.com
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/wusui/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.13): ./virtualenv/bin/ceph-deploy install plana57.front.sepia.ceph.com
[ceph_deploy.install][DEBUG ] Installing stable version firefly on cluster ceph hosts plana57.front.sepia.ceph.com
[ceph_deploy.install][DEBUG ] Detecting platform for host plana57.front.sepia.ceph.com ...
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[ceph_deploy.install][INFO  ] Distro info: Ubuntu 12.04 precise
[plana57.front.sepia.ceph.com][INFO  ] installing ceph on plana57.front.sepia.ceph.com
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo env DEBIAN_FRONTEND=noninteractive apt-get -q install --assume-yes ca-certificates
[plana57.front.sepia.ceph.com][DEBUG ] Reading package lists...
[plana57.front.sepia.ceph.com][DEBUG ] Building dependency tree...
[plana57.front.sepia.ceph.com][DEBUG ] Reading state information...
[plana57.front.sepia.ceph.com][DEBUG ] ca-certificates is already the newest version.
[plana57.front.sepia.ceph.com][DEBUG ] 0 upgraded, 0 newly installed, 0 to remove and 99 not upgraded.
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo wget -O release.asc https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc
[plana57.front.sepia.ceph.com][WARNIN] --2014-09-08 15:45:10--  https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc
[plana57.front.sepia.ceph.com][WARNIN] Resolving ceph.com (ceph.com)... 208.113.241.137, 2607:f298:4:147::b05:fe2a
[plana57.front.sepia.ceph.com][WARNIN] Connecting to ceph.com (ceph.com)|208.113.241.137|:443... connected.
[plana57.front.sepia.ceph.com][WARNIN] HTTP request sent, awaiting response... 200 OK
[plana57.front.sepia.ceph.com][WARNIN] Length: unspecified [text/plain]
[plana57.front.sepia.ceph.com][WARNIN] Saving to: `release.asc'
[plana57.front.sepia.ceph.com][WARNIN] 
[plana57.front.sepia.ceph.com][WARNIN]      0K .                                                      48.7M=0s
[plana57.front.sepia.ceph.com][WARNIN] 
[plana57.front.sepia.ceph.com][WARNIN] 2014-09-08 15:45:10 (48.7 MB/s) - `release.asc' saved [1752]
[plana57.front.sepia.ceph.com][WARNIN] 
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo apt-key add release.asc
[plana57.front.sepia.ceph.com][DEBUG ] OK
[plana57.front.sepia.ceph.com][DEBUG ] add deb repo to sources.list
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo apt-get -q update
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/main Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/restricted Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/universe Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/multiverse Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/restricted amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/universe amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/multiverse amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/restricted i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/universe i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/multiverse i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/multiverse TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/restricted TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/universe TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/multiverse Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/restricted Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise/universe Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://gitbuilder.ceph.com precise/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://gitbuilder.ceph.com precise/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise Release.gpg
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://gitbuilder.ceph.com precise/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://gitbuilder.ceph.com precise/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise Release
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://gitbuilder.ceph.com precise/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/main Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise Release
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/restricted Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/universe Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/multiverse Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/restricted amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/universe amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/multiverse amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/restricted i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/universe i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/multiverse i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/multiverse TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/restricted TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/universe TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/main Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/restricted Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/universe Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/multiverse Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/restricted amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/universe amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/multiverse amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/restricted i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/universe i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/multiverse i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/multiverse TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/restricted TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/universe TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/main Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/restricted Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/universe Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/multiverse Sources
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/restricted amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/universe amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/multiverse amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/restricted i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/universe i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/multiverse i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/multiverse TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/restricted TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/universe TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/multiverse Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/restricted Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-updates/universe Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/multiverse Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/restricted Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-backports/universe Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/multiverse Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/restricted Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://apt-mirror.front.sepia.ceph.com precise-security/universe Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://ceph.com precise/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise/main amd64 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Hit http://ceph.com precise/main i386 Packages
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://ceph.com precise/main TranslationIndex
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://ceph.com precise/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Ign http://ceph.com precise/main Translation-en
[plana57.front.sepia.ceph.com][DEBUG ] Reading package lists...
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get -q -o Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes install -- ceph ceph-mds ceph-common ceph-fs-common gdisk
[plana57.front.sepia.ceph.com][DEBUG ] Reading package lists...
[plana57.front.sepia.ceph.com][DEBUG ] Building dependency tree...
[plana57.front.sepia.ceph.com][DEBUG ] Reading state information...
[plana57.front.sepia.ceph.com][DEBUG ] gdisk is already the newest version.
[plana57.front.sepia.ceph.com][DEBUG ] Some packages could not be installed. This may mean that you have
[plana57.front.sepia.ceph.com][DEBUG ] requested an impossible situation or if you are using the unstable
[plana57.front.sepia.ceph.com][DEBUG ] distribution that some required packages have not yet been created
[plana57.front.sepia.ceph.com][DEBUG ] or been moved out of Incoming.
[plana57.front.sepia.ceph.com][DEBUG ] The following information may help to resolve the situation:
[plana57.front.sepia.ceph.com][DEBUG ] 
[plana57.front.sepia.ceph.com][DEBUG ] The following packages have unmet dependencies:
[plana57.front.sepia.ceph.com][DEBUG ]  ceph-common : Depends: librbd1 (= 0.80.5-1precise) but 0.84-1033-g1250c31-1precise is to be installed
[plana57.front.sepia.ceph.com][WARNIN] E: Unable to correct problems, you have held broken packages.
[plana57.front.sepia.ceph.com][ERROR ] RuntimeError: command returned non-zero exit status: 100
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get -q -o Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes install -- ceph ceph-mds ceph-common ceph-fs-common gdisk

This happened on both --cluster-network and --public-network, when I used either eth0 or eth2 IP address, and when I try ip-addr/0, and ip-addr/1. I was guessing that the number after the slash is a port number, and so I tried 6789, but in that case, ceph-deploy new --cluster-network 10.214.132.21/6789 plana57.front.sepia.ceph.com failed with the following:

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/wusui/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.13): ./virtualenv/bin/ceph-deploy new --public-network 10.214.132.21/6789 plana57.front.sepia.ceph.com
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO  ] making sure passwordless SSH succeeds
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: aardvark 
[plana57.front.sepia.ceph.com][INFO  ] Running command: ssh -CT -o BatchMode=yes plana57.front.sepia.ceph.com
[ceph_deploy.new][WARNIN] could not connect via SSH
[ceph_deploy.new][INFO  ] will connect again with password prompt
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[ceph_deploy.new][INFO  ] adding public keys to authorized_keys
[plana57.front.sepia.ceph.com][DEBUG ] append contents to file
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[plana57.front.sepia.ceph.com][DEBUG ] find the location of an executable
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip link show
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip addr show
[plana57.front.sepia.ceph.com][DEBUG ] IP addresses found: ['10.214.132.21', '10.214.145.21']
[ceph_deploy][ERROR ] Traceback (most recent call last):
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/util/decorators.py", line 69, in newfunc
[ceph_deploy][ERROR ]     return f(*a, **kw)
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/cli.py", line 160, in _main
[ceph_deploy][ERROR ]     return args.func(args)
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/new.py", line 150, in new
[ceph_deploy][ERROR ]     validate_host_ip(remote_ips, [args.public_network, args.cluster_network])
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/new.py", line 95, in validate_host_ip
[ceph_deploy][ERROR ]     if ip_in_one_subnet(ips, subnet):
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/new.py", line 90, in ip_in_one_subnet
[ceph_deploy][ERROR ]     if net.ip_in_subnet(ip, subnet):
[ceph_deploy][ERROR ]   File "/home/wusui/src/ceph-deploy/ceph_deploy/util/net.py", line 51, in ip_in_subnet
[ceph_deploy][ERROR ]     mask = (0xffffffff << (32 - int(bits))) & 0xffffffff
[ceph_deploy][ERROR ] ValueError: negative shift count
[ceph_deploy][ERROR ] 

#5 Updated by Alfredo Deza over 9 years ago

The '/XX' is not a port, it is meant to be the specification for the subnet of a given IP address, this is referred to as the CIDR (see: https://en.wikipedia.org/wiki/Subnetwork )

The package manager error has nothing to do with the specification of cluster subnets.

The logs seems to be very explicit about what exactly the problem is:

ceph-common : Depends: librbd1 (= 0.80.5-1precise) but 0.84-1033-g1250c31-1precise is to be installed

Who/what is installing librbd1 0.80.5-1 ?

Where is that package coming from?

#6 Updated by Alfredo Deza over 9 years ago

  • Assignee changed from Alfredo Deza to Anonymous

#7 Updated by Anonymous over 9 years ago

  • Status changed from Need More Info to 4
  • Assignee changed from Anonymous to Alfredo Deza

I have plana57 and burnupi61 locked.

I ran purgedata, forgetkeys, and purge on both machines, then tried:

./virtualenv/bin/ceph-deploy new --cluster-network 10.0.0.0/20 plana57.front.sepia.ceph.com

and got

.
.
.
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip link show
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip addr show
[plana57.front.sepia.ceph.com][DEBUG ] IP addresses found: ['10.214.132.21', '10.214.145.21']
[ceph_deploy][ERROR ] RuntimeError: subnet (10.0.0.0/20) is not valid for any of the ips found ['10.214.132.21', '10.214.145.21']

So I assume that these are the ip address I should use for cluster and public networks. So I
then tried:

./virtualenv/bin/ceph-deploy new --public-network 10.214.132.21/20 --cluster-network 10.214.145.21/20 plana57.front.sepia.ceph.com burnupi61.front.sepia.ceph.com

and got:

[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][INFO  ] making sure passwordless SSH succeeds
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: aardvark 
[plana57.front.sepia.ceph.com][INFO  ] Running command: ssh -CT -o BatchMode=yes plana57.front.sepia.ceph.com
[ceph_deploy.new][WARNIN] could not connect via SSH
[ceph_deploy.new][INFO  ] will connect again with password prompt
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[ceph_deploy.new][INFO  ] adding public keys to authorized_keys
[plana57.front.sepia.ceph.com][DEBUG ] append contents to file
[plana57.front.sepia.ceph.com][DEBUG ] connected to host: plana57.front.sepia.ceph.com 
[plana57.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[plana57.front.sepia.ceph.com][DEBUG ] detect machine type
[plana57.front.sepia.ceph.com][DEBUG ] find the location of an executable
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip link show
[plana57.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip addr show
[plana57.front.sepia.ceph.com][DEBUG ] IP addresses found: ['10.214.132.21', '10.214.145.21']
[ceph_deploy.new][DEBUG ] Resolving host plana57.front.sepia.ceph.com
[ceph_deploy.new][DEBUG ] Monitor plana57 at 10.214.132.21
[ceph_deploy.new][INFO  ] making sure passwordless SSH succeeds
[burnupi61.front.sepia.ceph.com][DEBUG ] connected to host: aardvark 
[burnupi61.front.sepia.ceph.com][INFO  ] Running command: ssh -CT -o BatchMode=yes burnupi61.front.sepia.ceph.com
[ceph_deploy.new][WARNIN] could not connect via SSH
[ceph_deploy.new][INFO  ] will connect again with password prompt
[burnupi61.front.sepia.ceph.com][DEBUG ] connected to host: burnupi61.front.sepia.ceph.com 
[burnupi61.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[burnupi61.front.sepia.ceph.com][DEBUG ] detect machine type
[ceph_deploy.new][INFO  ] adding public keys to authorized_keys
[burnupi61.front.sepia.ceph.com][DEBUG ] append contents to file
[burnupi61.front.sepia.ceph.com][DEBUG ] connected to host: burnupi61.front.sepia.ceph.com 
[burnupi61.front.sepia.ceph.com][DEBUG ] detect platform information from remote host
[burnupi61.front.sepia.ceph.com][DEBUG ] detect machine type
[burnupi61.front.sepia.ceph.com][DEBUG ] find the location of an executable
[burnupi61.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip link show
[burnupi61.front.sepia.ceph.com][INFO  ] Running command: sudo /bin/ip addr show
[burnupi61.front.sepia.ceph.com][DEBUG ] IP addresses found: ['10.214.149.8', '10.214.136.8']
[ceph_deploy.new][DEBUG ] Resolving host burnupi61.front.sepia.ceph.com
[ceph_deploy.new][DEBUG ] Monitor burnupi61 at 10.214.136.8
[ceph_deploy.new][DEBUG ] Monitor initial members are ['plana57', 'burnupi61']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['10.214.132.21', '10.214.136.8']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...

ceph.conf at this point is:

[global]
fsid = 8ebda0a4-aa82-4859-b03f-5857d4e1fe23
public_network = 10.214.132.21/20
cluster_network = 10.214.145.21/20
mon_initial_members = plana57, burnupi61
mon_host = 10.214.132.21,10.214.136.8
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
filestore_xattr_use_omap = true

I was able to ping both mon_host ip addresses but not ping the cluster_network. I suspect that I can't ping the cluster_network because it is private.

What else needs to be done to test this?

#8 Updated by Alfredo Deza over 9 years ago

  • Assignee changed from Alfredo Deza to Anonymous

Warren: you need to pass in a valid network range for this to work. I suggest using a network/ip calculator as sometimes it is a bit difficult to know what range of IPs you are getting for a given subnet definition.

In your case, the validation system in place for ceph-deploy is correctly preventing you from moving forward as the IP range you are giving does not match any of the IPs in the remote node.

A subnet of 10.0.0.0/20 will get you about 4000 IP addresses from 10.0.0.1 all the way up to 10.0.15.255. As you can see, within that range you will not find neither 10.214.132.21 nor 10.214.145.21

You must use a valid subnet, otherwise this will not work.

Specifying the actual IP with the CIDR notation to is also not what you need.

Here is an online calculator that might help: http://mxtoolbox.com/subnetcalculator.aspx

For this specific case, you may want to use the following subnets that are valid for both IPs you are using:

10.214.0.0/16

That subnet will give you IPs from 10.214.0.1 all the way up to 10.214.255.255

Again, you must pass in correct values and you need to make sure they are fully valid otherwise ceph-deploy will not allow you to continue.

#9 Updated by Anonymous over 9 years ago

Sandon recommended that I use --public-network 10.214.128.0/20 --cluster-network 10.214.144.0/20 for the new command. This seemed to work. After that, the ceph install on both machines seemed to work, followed by a mon create that also seemeed to work. When I
tried to gatherkeys after that, I got the following.

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ubuntu/.cephdeploy.conf
[ceph_deploy.cli][INFO  ] Invoked (1.5.15): ./virtualenv/bin/ceph-deploy gatherkeys plana57
[ceph_deploy.gatherkeys][DEBUG ] Checking plana57 for /etc/ceph/ceph.client.admin.keyring
[plana57][DEBUG ] connected to host: plana57 
[plana57][DEBUG ] detect platform information from remote host
[plana57][DEBUG ] detect machine type
[plana57][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on ['plana57']
[ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring
[ceph_deploy.gatherkeys][DEBUG ] Checking plana57 for /var/lib/ceph/bootstrap-osd/ceph.keyring
[plana57][DEBUG ] connected to host: plana57 
[plana57][DEBUG ] detect platform information from remote host
[plana57][DEBUG ] detect machine type
[plana57][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find /var/lib/ceph/bootstrap-osd/ceph.keyring on ['plana57']
[ceph_deploy.gatherkeys][DEBUG ] Checking plana57 for /var/lib/ceph/bootstrap-mds/ceph.keyring
[plana57][DEBUG ] connected to host: plana57 
[plana57][DEBUG ] detect platform information from remote host
[plana57][DEBUG ] detect machine type
[plana57][DEBUG ] fetch remote file
[ceph_deploy.gatherkeys][WARNIN] Unable to find /var/lib/ceph/bootstrap-mds/ceph.keyring on ['plana57']

It's unclear to me if the keyring problems are problem stemming from the initial configuration or if this is another issue. I tried running the ceph-deploy off of my machine, off of teuthology.front.sepia.ceph.com, and off of a vms that I brought up. In all three cases, it was unable to find the keyrings. I did purgedata, forgetkeys, and purge before trying each case as well.

#10 Updated by Tamilarasi muthamizhan over 9 years ago

  • Target version changed from sprint13 to sprint14

#11 Updated by Anonymous over 9 years ago

  • Status changed from 4 to In Progress

#12 Updated by Anonymous over 9 years ago

  • Status changed from In Progress to Resolved

Okay. I am guessing that I messed up the monitor setup previously. I ran ceph-deploy mon create-initial on both machines, created osds on both machines, and brought up ceph. Using iftop -i eth2, I saw traffic from plana57.back.sepia.ceph.com to burnupi61.back.sepia.ceph.com and back. (10kb range). I then ran rados -p data bench 180 write on plana57 and saw up to 400Kb of traffic on eth2 between plana57.back and burnupi61.back. So I assume that this setup is correct and the behavior here is what we want.

Also available in: Atom PDF