Bug #16036

ceph mon deamon startup side effect running ceph-create-keys

Added by Owen Synge over 6 years ago. Updated almost 6 years ago.

Target version:
% Done:


3 - minor
Affected Versions:
Pull request ID:
Crash signature (v1):
Crash signature (v2):


For full details on this bug please read:

In summary this bug has been blocked until ceph-deploy is no longer dependent on this behaviour.

Summary of issues:

(0) A ceph cluster should be able to be installed in any order. With the
current behavior if the mds, rgw, or osd nodes are deployed first (along
with the boot strap keyrings), the mon created must have all keys for
the admin, mds-bootstrap, rgw-bootstrap, and osd-boostrap deployed in
the correct path before the mon can safely be started, even if the
cluster does not need the mds or rgw service's.

(1) It is unfriendly to configuration being stored on the configuration
server as the server needs to be updated with the values from the
configured node keys, when people might want to store these keys centrally.

(2) Assuming the admin, rgw-bootstrap, mds-bootstrap and osd-boostrap
keys are always installed on all mon nodes is clearly increasing the
distribution of keys where they might not be needed. Hence reducing

(3) Using the current model adds an extra complication that these keys
then need to be distributed to each node from the configured node, if
generated by starting the mon, and not from the configuration server.

(4) If you wish to use a more devops approach, and generate keys
explicitly all the keys must be installed on all mon nodes before the
mon nodes are started.

(4.1) As a side effect we need to document why admins need the
mds-bootstrap keyring when they dont want this service it is confusing,
and requires an unnecessary process of migrating all keys to the
explicitly desired keys.

(5) I am developing a simple python library to configure ceph on each
node independently of all others, (think of it as a parallelism version
of ceph-deploy that can be called by any config management system) but
with the current side effect behavior starting the mon needs to fail if
the mds-bootstrap keyring is not created on the mon nodes before
starting the mon, otherwise we get ordering complications.

(5) The side effect is confusing, as no one expects this side effect,
hence this leads to ceph seeming complex to a first time user.

(6) I feel it is the responsibility of configuration management not the
mon demon to request creating these keys.

(7) I dont think this is clearly documented, hence this leads to ceph
seeming complex to a first time user.

(8) As more services like mds and rgw get added to ceph the problem gets

(9) Adding one more step to the by hand installation will clarify the
authentication process. This extra step would simply be:

/usr/sbin/ceph-create-keys --cluster ${CLUSTER} --id ${MON_NAME}


#1 Updated by Owen Synge over 6 years ago

ceph-deploy is now fixed in master of ceph-deploy

#2 Updated by Owen Synge over 6 years ago

We now have a Pull request for this fix.

#3 Updated by Jason Dillaman over 6 years ago

  • Project changed from rbd to Ceph

#4 Updated by Nathan Cutler over 6 years ago

  • Status changed from New to Fix Under Review
  • Priority changed from Normal to High

#5 Updated by Samuel Just about 6 years ago

  • Priority changed from High to Normal

#6 Updated by Nathan Cutler almost 6 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF