Bug #16255
closed
ceph-create-keys: sometimes blocks forever if mds "allow" is set
Added by Dietmar Maurer almost 8 years ago.
Updated over 7 years ago.
Category:
Administration/Usability
Description
The documentations at:
http://docs.ceph.com/docs/master/dev/mon-bootstrap/
tells to create the client.admin key with:
- ceph-authtool /path/to/keyring --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow'
Please note the mds permission is 'allow', and not 'allow *'.
I never understood why this need to be different, but I used this everywhere. So all our
existing installations use it.
But this commit:
https://github.com/ceph/ceph/commit/c7e905e7e232a973abf7c6fa71a2ffbad7aa0ffd
suddenly requires 'allow *' for mds permissions. So with my old keys,
ceph-create-keys simply runs an endless loop logging the following
error message:
Jun 13 06:10:17 elsa ceph-create-keys10083: Error EINVAL: key for client.admin
exists but cap mds does not match
Jun 13 06:10:17 elsa ceph-create-keys10083: INFO:ceph-create-keys:Cannot get
or create admin key
Jun 13 06:10:18 elsa ceph-create-keys10083: INFO:ceph-create-keys:Talking to
monitor...
So either the documentation is wrong, or the commit should be reverted.
- Project changed from Ceph to CephFS
The difference between "allow" and "allow *" is that the "*" is necessary in more recent versions to issue 'tell' commands. Having an existing key that only has "allow" isn't usually a problem until someone tries to do "ceph tell mds.foo ...".
The loop you're seeing presumably is only occurring when /etc/ceph/ceph.client-admin.keyring has been removed.
Folks on systemd-using systems will probably not be seeing this issue because the systemd service file checks for /var/lib/ceph/bootstrap-mds/ceph.keyring.
Anyway, this is a bug. ceph-create-keys should not loop forever if the admin key exists but has different caps than the default.
- Subject changed from ceph-create-keys: set mds "allow *" is wrong! to ceph-create-keys: sometimes blocks forever if mds "allow" is set
The loop you're seeing presumably is only occurring when /etc/ceph/ceph.client-admin.keyring has been removed.
exactly (we do not store the key there).
Folks on systemd-using systems will probably not be seeing this issue because the systemd service file checks for /var/lib/ceph/bootstrap-mds/ceph.keyring.
Anyway, this is a bug. ceph-create-keys should not loop forever if the admin key exists but has different caps than the default.
Yes - I thought it should simply use the existing key.
- Category set to Administration/Usability
We've encountered the same issue when upgrading. The " on systemd-using systems will probably not be seeing this" is not an issue, what we see on a systemd server is that upgrading will trigger it.
This might be because the initial keyring for the cluster was created with `mds allow` and now the newly upgraded cluster requests `mds allow *` which returns the:
Error EINVAL: moncap parse failed, stopped at 'allow' of 'allow'
Relevant BZ: https://bugzilla.redhat.com/show_bug.cgi?id=135729
Regardless of the deployment method (manual/ceph-ansible/ceph-deploy) a user who adds a new monitor to an upgraded cluster will hit this problem
- Status changed from New to Fix Under Review
- Assignee set to John Spray
I've had the same issue when use ceph-deploy gatherkeys(jewel)
if I change "mds 'allow'" to "mds 'allow *'", it's true.
- Status changed from Fix Under Review to Pending Backport
- Copied to Backport #17347: jewel: ceph-create-keys: sometimes blocks forever if mds "allow" is set added
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF