Documentation #8875
closed`ceph-deploy new` needs to be called for every node, not just the admin one
0%
Description
Hi guys,
Running into issue adding monitors. The initial monitor gets created successfully, but get below error adding additional ones.
Running ubuntu 14.04 and ceph firefly.
Error:
[DEBUG ] Starting ceph-create-keys on cwtcph002...
[WARNIN] No data was received after 7 seconds, disconnecting.
[INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.cwtcph002.asok mon_status
[ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[WARNIN] monitor: mon.cwtcph002, might not be running yet
[INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.cwtcph002.asok mon_status
[ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
[WARNIN] cwtcph002 is not defined in `mon initial members`
[WARNIN] monitor cwtcph002 does not exist in monmap
[WARNIN] neither `public_addr` nor `public_network` keys are defined for monitors
[WARNIN] monitors may not be able to form quorum
Updated by Alfredo Deza over 9 years ago
- Assignee set to Alfredo Deza
Can you paste the whole output of ceph-deploy?
Updated by Bobby Yakov over 9 years ago
Hi Alfredo,
Nodes were cleaned out, will re-run install today and get you the log files.
In the mean time, it appears that passwords-less SSH is required for me to run install.
I thought this version of Ceph, no longer requires that I go to each node and grant access,
can you confirm. Want to know if this is another issue I need to open.
Thanks
Updated by Bobby Yakov over 9 years ago
It appears I was able to get further this time, the steps are below.
Key difference is, when I did ceph-deploy new I included all the nodes not just the admin.
Will let you know how the rest of install went.
Thanks...
steps
1. On Each Node:
echo "ceph_user ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/ceph_user
sudo chmod 0440 /etc/sudoers.d/ceph_user
2. On admin node only
a. ssh-keygen
b. Create ~/.ssh/config
Host cwtcph001
Hostname cwtcph001
User ceph_user
Host cwtcph002
Hostname cwtcph002
User ceph_user
Host cwtcph003
Hostname cwtcph003
User ceph_user
c. On each Node:
(including admin node since it will do both admin and monitoring).
ssh-copy-id cewit_user@cwtcph001 (Admin + monitoring)
ssh-copy-id cewit_user@cwtcph002
ssh-copy-id cewit_user@cwtcph003
3. mkdir ceph-cluster cd ceph-cluster
4.
a. wget q -O 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -
b. echo deb http://ceph.com/debian-firefly/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list
c. sudo apt-get update && sudo apt-get install ceph-deploy (admin node)
5. ceph-deploy new cwtcph001 cwtcph002 cwtcph003
output files = ceph.conf ceph.log ceph.mon.keyring
Log file :
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/cewit_admin/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.9): /usr/bin/ceph-deploy new cwtcph001 cwtcph002 cwtcph003
[ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[ceph_deploy.new][DEBUG ] Resolving host cwtcph001
[ceph_deploy.new][DEBUG ] Monitor cwtcph001 at 10.1.0.13
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[ceph_deploy.new][DEBUG ] Resolving host cwtcph002
[ceph_deploy.new][DEBUG ] Monitor cwtcph002 at 10.1.0.14
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[cwtcph002][DEBUG ] connected to host: cwtcph001
[cwtcph002][INFO ] Running command: ssh -CT -o BatchMode=yes cwtcph002
[ceph_deploy.new][DEBUG ] Resolving host cwtcph003
[ceph_deploy.new][DEBUG ] Monitor cwtcph003 at 10.1.0.15
[ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[cwtcph003][DEBUG ] connected to host: cwtcph001
[cwtcph003][INFO ] Running command: ssh -CT -o BatchMode=yes cwtcph003
[ceph_deploy.new][DEBUG ] Monitor initial members are ['cwtcph001', 'cwtcph002', 'cwtcph003']
[ceph_deploy.new][DEBUG ] Monitor addrs are ['10.1.0.13', '10.1.0.14', '10.1.0.15']
[ceph_deploy.new][DEBUG ] Creating a random mon key...
[ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
6. ceph-deploy install cwtcph001 cwtcph002 cwtcph003 (completed with no errors on all 3 nodes).
Node Files:
a. cwtcph001 = ceph.conf ceph.log ceph.mon.keyring release.asc
b. cwtcph002 = home directory release.asc
c. cwtcph003 = home directory release.asc
7. ceph-deploy mon create cwtcph001 cwtcph002 cwtcph003
Appears to have worked this time.
8. ceph-deploy gatherkeys cwtcph001
Updated by Alfredo Deza over 9 years ago
- Status changed from New to 4
You still need a user that can call sudo without a password prompt on remote nodes.
And it looks like you only pasted the `ceph-deploy new` command log output. Would you mind getting the complete log output from ceph-deploy
of all the commands you run right until the point where you hit the error?
Updated by Bobby Yakov over 9 years ago
I was able to complete install.
The first step above granted sudo rights on each node.
The way I was able to get it to work was, when running command " ceph-deploy new " ,
I have to specify all the nodes. The instructions do not state that.
Originally only when running ceph-deploy new I only specified the primary admin node.
then ran command ceph-deploy install to all the nodes.
I appears that nodes specified in ceph-deploy new must match nodes specified in ceph-deploy install.
Updated by Alfredo Deza over 9 years ago
- Tracker changed from Bug to Documentation
- Subject changed from ceph-deploy mon create fails to create additional monitoring nodes. to `ceph-deploy new` needs to be called for every node, not just the admin one
- Status changed from 4 to In Progress
I noted the problem in the docs and will fix that shortly.
You are right, you need to run `ceph-deploy new {NODES}` not just use the admin node.
Updated by Alfredo Deza over 9 years ago
- Project changed from devops to Ceph
- Category deleted (
ceph-deploy)
Updated by Alfredo Deza over 9 years ago
- Status changed from In Progress to Resolved
PR https://github.com/ceph/ceph/pull/2206
and merged commit e6935dd into master