Osd - simple ceph-mon dm-crypt key management¶

Summary¶

ceph-disk can encrypt disks for you, but it stores keys in /etc/ceph/keys, which is only somewhat useful. Let's construct a scheme that lets you store disk encryption keys security on the monitor without external dependencies. (And perhaps make hooks along the way to use external key management too.)

Owners¶

• Sage Weil (Red Hat)
• Milan Broz (Red Hat)
• Name

Interested Parties¶

• Danny Al-Gaaf (Deutsche Telekom)
• Name (Affiliation)
• Name

Current Status¶

ceph-disk will

• transparently create osd volumes with dm-crypt
• start osds
• store keys in /etc/ceph/keys only

ceph-mon will

• store arbitrary key/value data on the mon
  • ceph config-key set <key> -i value
  • ceph config-key get <key> -o value
• protect this data with a ceph auth key
  • ceph -n client.osd.1.wrapper --key <secret> config-key get osd.1/dmcrypt-key -o ...
• let you restrict keys' access to specific subranges
  • e.g., client.osd.1.wrapper only has access to read osd.1/dmcrypt-key and nothing else

on any given disk we have

• a GPT partition table
• GPT parittion types for
  • osd data volume
  • encrypted osd data volume
  • osd journal volume
  • encrypted osd journal volume

Detailed Description¶

There are a few missing pieces.
Wrapper key
The first is that we need a place on the encrypted disk to store a few pieces of metadata:

• which osd id (e.g., 123) and uuid is this
• which cluster uuid is this
• what is the wrapper key

The wrapper key is used to fetch teh decryption key from the mon. It can be easily revoked if the disk goes missing without throwing out the encryption key.
Or, if we use LUKS, I think the encryption key is used to unlock the actual encryption key on the disk. EIther way, the wrapper key can be had by anyone who gets the disk. There is probably a real name for this ("token?")
Do we put a new, tiny partition on the disk to store this? With a new type?
Do we use the same wrapper key for the data and journal? Probably different ones for each, given that they are often on different devices (ssd for journals, hdd for disks, etc.)

ceph-disk changes
We need to do a bunch of new stuff here that may mean talking to the mon and be slower than before:
when creating a disk, we need to generate the encryption key, store it on the mon, and generate a wrapper key that can read it. we'll use a boostrap-osd type key to do this.
when activating a disk, we need to get the disk's wrapper key and fetch the encryption key from the mon. this requires the network, which means ceph-disk activate may be slower than before. i'm not sure if udev will mind or not.

streamline revocation
We'll want a tool or well-documented workflow to revoke a wrapper key. Maybe give it a unique name (client.osd.1.wrapper.$random) and revoke the key's permissions so we still ahve a record of the key itself for future reference?

Work items¶

Coding tasks¶

1. establish scheme for storing keys on disks
2. expand osd-bootstrap and or osd profiles on mon to allow all the command we need
3. ceph-disk: create keys and store them on mon
4. ceph-disk activate: fetch keys and start mon
5. revocation workflow
6. un-revocation worklow?