Project

General

Profile

Feature #14669

ceph-disk: support indirect dmcrypt key retrieval

Added by Loic Dachary over 3 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
Start date:
02/05/2016
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Associated revisions

Revision 1ec58fcf (diff)
Added by Loic Dachary over 3 years ago

ceph-disk: implement lockbox key management

Instead of storing the dmcrypt keys in the /etc/ceph/dmcrypt-keys
directory, they are stored in the monitor. If a machine with
OSDs created with ceph-disk prepare --dmcrypt is lost, it does
not contain the key that would allow to decrypt their content.

The dmcrypt key is retrieved from the monitor using a different keyring
for each OSD. It is stored in a small partition called the lockbox. At
boot time the lockbox is mounted

/var/lib/ceph/osd-lockbox/$uuid

and used when the $uuid partition is detected by udev to map it with
cryptsetup.

The OSDs that were prepared prior to the lockbox implementation are
supported by looking up the key found in /etc/ceph/dmcrypt-keys before
looking in /var/lib/ceph/osd-lockbox/$uuid.

http://tracker.ceph.com/issues/14669 Fixes: #14669

Signed-off-by: Loic Dachary <>

History

#1 Updated by Loic Dachary over 3 years ago

  • Status changed from Verified to In Progress

#2 Updated by Loic Dachary over 3 years ago

osd dm-crypt key management scheme
----------------------------------

- create new partition type OSD_LOCKBOX (or similar)
- populate with tiny file system
- automount, read-only, at /var/lib/ceph/osd-lockbox/$uuid (where uuid can be random, doesn't matter.. either unique to this device, or the osd uuid)
- 'km-mode' file indicates which key management scheme we are using.

creation
--------

- create lockbox partition on device (unencrypted), with tiny fs
- store luks key on monitor
- ceph config-key put dm-crypt/osd/$osd_uuid/luks $secret
- create a ceph user that can fetch it
- ceph auth get-or-create client.osd-lockbox.$osd_uuid mon 'allow command "config-key get" with key="dm-crypt/osd/$osd_uuid"' > /var/lib/ceph/osd-lockbox/$osd_uuid/keyring
- echo 'ceph-mon v1' > /var/lib/ceph/osd-lockbox/$osd_uuid/km-mode

activation
----------

- if km-mode == "ceph-mon v1" ...
- use user and key from 'keyring' file:
- 'ceph -n client.osd-lockbox.$osd_uuid -k /var/lib/ceph/osd-lockbox/$osd_uuid/keyring config-key get dm-crypt/osd/$osd_uuid' will write the secret to stdout

- when we encounter a dm-crypt device,
- first check legacy location (/etc/ceph/dmcrypt-keys/$osd_uuid)
- then check for lockbox (/var/lib/ceph/osd-lockbox/$osd_uuid)

- after we mount the lockbox, re-probe any dm-crypt devices with the same uuid (in case they tried before but lockbox wasn't mounted yet)

- if there are alternative key managers in use, indicate them in the lockbox, and adjust the "get key" method accordingly

#3 Updated by Loic Dachary over 3 years ago

  • Description updated (diff)

#4 Updated by Loic Dachary over 3 years ago

[root@target167114242062 ceph-disk]# cryptsetup remove /dev/mapper/b2ba96fe-e05e-49a0-818b-9ba459ad6256 
[root@target167114242062 ceph-disk]# cryptsetup --key-file /etc/ceph/dmcrypt-keys/b2ba96fe-e05e-49a0-818b-9ba459ad6256 create b2ba96fe-e05e-49a0-818b-9ba459ad6256 /dev/vdb1 --key-size 256
[root@target167114242062 ceph-disk]# mount -t xfs -o noatime,inode64 -- /dev/mapper/b2ba96fe-e05e-49a0-818b-9ba459ad6256 /mnt
[root@target167114242062 ceph-disk]# umount /mnt
[root@target167114242062 ceph-disk]# cryptsetup remove /dev/mapper/b2ba96fe-e05e-49a0-818b-9ba459ad6256 
[root@target167114242062 ceph-disk]# cat /etc/ceph/dmcrypt-keys/b2ba96fe-e05e-49a0-818b-9ba459ad6256 | cryptsetup --key-file - create b2ba96fe-e05e-49a0-818b-9ba459ad6256 /dev/vdb1 --key-size 256
[root@target167114242062 ceph-disk]# mount -t xfs -o noatime,inode64 -- /dev/mapper/b2ba96fe-e05e-49a0-818b-9ba459ad6256 /mnt
mount: wrong fs type, bad option, bad superblock on /dev/mapper/b2ba96fe-e05e-49a0-818b-9ba459ad6256,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

#5 Updated by Loic Dachary over 3 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF