Ceph-deploy

I'm suffering from the same issue, working on ubunto 15.04 with ceph 0.94.1

Previously I had a problem that in Ubuntu 15.04 it can't find the initctl (because it uses the systemd) so i reinstall the upstart and now it stucks here.

[mon1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=mon1
[mon1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon1.asok mon_status
[mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[mon1][WARNIN] monitor: mon.mon1, might not be running yet

It's my first ceph installation.

I am not really an expert of ceph, but I manage to get it going.

is your ceph installed in /usr/bin directory? By default ceph is installed in /usr/local/bin, set your --prefix to /usr otherwise you have to modify upstart config files.

Also just to be sure, before installing, purge and purgedata at your host, ceph-deploy [purge|purgedata] $(hostname -s).
If it complains of "ceph" being installed while running purgedata, move ceph from /usr/bin to some temporary directory and then later move it back to /usr/bin.

I think this should do.
-krishan

Thanks for your quick reply,
The ceph is installed on /usr/bin, and I have no problems to purge|purgedata.
Ill put some more prints maybe someone will find something.
my steps was:
ceph-deploy --username ceph install mon1
ceph-deploy --username ceph new mon1

ceph-deploy --username ceph admin mon1 --> [ceph_deploy][ERROR ] RuntimeError: ceph.client.admin.keyring not found

ceph-deploy --username ceph mon create-initial

the create-initial fails with the following:

[mon1][DEBUG ] locating the `service` executable...
[mon1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=mon1
[mon1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon1.asok mon_status
[mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[mon1][WARNIN] monitor: mon.mon1, might not be running yet
[mon1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon1.asok mon_status
[mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[mon1][WARNIN] monitor mon1 does not exist in monmap
[mon1][WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[mon1][WARNIN] monitors may not be able to form quorum
[ceph_deploy.mon][INFO ] processing monitor mon.mon1
[mon1][DEBUG ] connection detected need for sudo
[mon1][DEBUG ] connected to host: ceph@mon1
[mon1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.mon1.asok mon_status
[mon1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[ceph_deploy.mon][WARNIN] mon.mon1 monitor is not yet in quorum, tries left: 5
[ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying
...

may be you want to try with default user, client.admin and without security. so do not specify any user. Also after "ceph-deploy new $(hostname -s)" open the cluster config, ceph.conf, and disable the authentication requirements, i.e replace cephx with none. and then create monitors.

try these at command line from current directory:
ceph-deploy purge $(hostname -s)
ceph-deploy purgedata $(hostname -s)
ceph-deploy install $(hostname -s)
cepph-deploy new $(hostname -s)
sed -i 's/cephx/none/g' ceph.conf
ceph-deploy mon create-initial

-

Thank you,
I've already tried it and got exactly the same results :-(

Thanks Travis,

root@cephDeployment:~/cephCluster# ceph-deploy --version
1.5.20

root@cephDeployment:~/cephCluster# cat /etc/apt/sources.list.d/ceph.list
deb http://ceph.com/debian/ vivid main

root@cephDeployment:~/cephCluster# apt-cache policy ceph |grep -m1 http| awk '{ print $2 " " $3 }'
http://il.archive.ubuntu.com/ubuntu/ vivid/main

I'm not using pip but I think I will try it later.

Hi Ariel,

I tried doing an installation on Vivid, because I was curious what would happen. The install I did is using all downstream packages (packaged/hosted by Ubuntu), and nothing from ceph.com.

For me, the install itself failed. Here are the commands I did. I was using a Vivid VM, and was starting by just doing a single node to get a Ceph monitor up (no OSDs). But I didn't get very far with the downstream packaging.

apt-get install ceph-deploy
mkdir ceph-config
cd ceph-config
ceph-deploy new `hostname`
ceph-deploy install --no-adjust-repos `hostname`

Note that the "--no-adjust-repos" option makes it such that it will not write a new ceph.list file in /etc/apt/sources.list.d. Since there are no Vivid packages hosted on ceph.com, there is no need to write a .list file that points to ceph.com.

The install failed for me with:

[vivid1][DEBUG ] dpkg: error processing archive /var/cache/apt/archives/ceph_0.94.1-0ubuntu1_amd64.deb (--unpack):
[vivid1][DEBUG ]  trying to overwrite '/usr/share/man/man8/ceph-deploy.8.gz', which is also in package ceph-deploy 1.5.20-0ubuntu1
[vivid1][DEBUG ] dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

So in this case, both 'ceph-deploy' and 'ceph' packages have packaged the same file, which will not work. This is an error with Ubuntu's packages.

I'm guessing others did not hit this because they don't have ceph-deploy installed on the same machine they were trying to deploy to. So, I fired up a second machine to try again. First I removed everything to do with the first cluster.

ceph-deploy uninstall `hostname`
ceph-deploy purgedata `hostname`
ceph-deploy forgetkeys

Then I started a second cluster that wouldn't use the node I was using ceph-deploy on

ceph-deploy new vivid2
ceph-deploy install --no-adjust-repos vivid2

This time the install was successful (not surprising). So then I tried to create the monitor.

ceph-deploy mon create-intial

At this point, I also hit the error that upstart (initctl) is not present. ceph-deploy only supports upstart right now, not systemd, so this isn't actually too surprising. The Ceph packages from Ubuntu do appear to ship with systemd unit files, so I was able to start the mon on my remote node (trusty2) by issuing "systemctl start ceph-mon" on that node. However, this doesn't help a whole lot with ceph-deploy. The "mon create-initial" command does a not of additional things once it starts mon and see they are in quorum, so starting them manually doesn't really help.

Unfortunately, I think the answer here is that ceph-deploy does not support systemd on Ubuntu, mostly because Ceph upstream is only targeting LTS releases (Precise and Trusty). That being said, Adding systemd support is certainly coming, and is not too difficult to add.

Travis, thank you for your try!
I switch to upstart so now initctl works for me, I used thik link:

http://sourcedigit.com/15820-use-upstart-in-ubuntu-15-04-how-to-switch-off-disable-systemd-in-ubuntu-15-04/

I run the command on my monitor

systemctl start ceph-mon

and rerrun the "ceph-deploy create-initial" and it works for me!!!!

so this should be the current work around to install ceph on ubunto 15.04.

1. switch systemd to upstart
2. run "systemctl start ceph-mon"
3. run ceph-deploy create-initial" from your ceph-deployment server

thank you!!! :-)

--Ariel