Calling ceph-disk with zap-disk and dmcrypt fails
Attempting to initialise a fresh device with 'dmcrypt' and 'zap-disk' params set fails with:
# ceph-disk prepare --dmcrypt --fs-type xfs --zap-disk --filestore /dev/vdb Creating new GPT entries. Setting name! partNum is 4 REALLY setting name! The operation has completed successfully. mke2fs 1.42.13 (17-May-2015) /dev/vdb5 contains a ext4 file system last mounted on /var/lib/ceph/osd-lockbox/94bad3e2-b9a3-45d8-aa9f-7bae30a36167 on Fri Oct 20 16:07:46 2017 Proceed anyway? (y,n) y Creating filesystem with 10240 1k blocks and 2560 inodes Filesystem UUID: 27c3b987-eca8-41a3-9e23-c7aaf499eda1 Superblock backups stored on blocks: 8193 Allocating group tables: done Writing inode tables: done Creating journal (1024 blocks): done Writing superblocks and filesystem accounting information: done creating /var/lib/ceph/osd-lockbox/beef2e39-df51-498e-b979-3ad7d2a61967/keyring added entity client.osd-lockbox.beef2e39-df51-498e-b979-3ad7d2a61967 auth auth(auid = 18446744073709551615 key=AQAkJepZgV8hNhAAZaZ8vl1ljbVY82689K5qwg== with 0 caps) creating /var/lib/ceph/osd-lockbox/beef2e39-df51-498e-b979-3ad7d2a61967/osd_keyring added entity osd.3 auth auth(auid = 18446744073709551615 key=AQAkJepZmogKMRAAlcJ9V+JIJKyIOIQY8SD/9Q== with 0 caps) Warning: The kernel is still using the old partition table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8) The operation has completed successfully. wipefs: error: /dev/vdb5: probing initialization failed: Device or resource busy ceph-disk: Error: Command '['/sbin/wipefs', '--all', '/dev/vdb5']' returned non-zero exit status 1 #
Grokking the code, the lockbox is created on /dev/vdb5 and then mounted; the subsequent prepare call then tries to re-zap the entire disk, which fails as the lockbox is mounted.
#3 Updated by James Page about 3 years ago
Link back to Ubuntu distro bug: https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/1604501