Project

General

Profile

Bug #20556

ceph-disk fails to prepare an osd device with --dmcrypt (bluestore)

Added by shangzhong zhu 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Urgent
Category:
-
Target version:
-
Start date:
07/10/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-disk
Release:
master
Needs Doc:
No

Description

[root@ceph117 ~]# ceph-disk -v prepare --bluestore --block.db /dev/sdk --block.wal /dev/sdk /dev/sdk --dmcrypt
command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_type: Will colocate block with data on /dev/sdk
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_db_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_wal_size
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_or_create_partition: Creating osd partition on /dev/sdk
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = lockbox
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
create_partition: Creating lockbox partition num 3 size 10 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=3:0:+10M --change-name=3:ceph lockbox --partition-guid=3:None --typecode=3:fb3aabf9-d25f-47cc-bf5e-721d181642be --mbrtogpt -- /dev/sdk
The operation has completed successfully.
update_partition: Calling partprobe on created device /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
command: Running command: /usr/bin/flock -s /dev/sdk /usr/sbin/partprobe /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk3 uuid path is /sys/dev/block/8:163/dm/uuid
populate: Creating lockbox fs on %s: mkfs -t ext4 /dev/sdk3
command_check_call: Running command: /usr/sbin/mkfs -t ext4 /dev/sdk3
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
2560 inodes, 10240 blocks
512 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=10485760
2 block groups
8192 blocks per group, 8192 fragments per group
1280 inodes per group
Superblock backups stored on blocks: 
        8193

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

populate: Mounting lockbox mount -t ext4 /dev/sdk3 /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command_check_call: Running command: /usr/bin/mount -t ext4 /dev/sdk3 /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/osd-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/osd-uuid.5634.tmp
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/ceph_fsid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/ceph_fsid.5634.tmp
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command_check_call: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring config-key put dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks LboRTY8ECOEMZXy9GrDdb6K3RMOkkrNw/dXHsmO0LYxvi2ccRcxrwUemq1NVDIbrdywpGjusHUZaiIsFD75NWMQORJdJCm0fGeVYWfIH24Ko1oBbpS4QkqbHsdjnlF6k6mfwgtH/B2wUq8ePoF7OpxtPh/3V7oyBpyR53FF0YrA=
set dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks
command: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring auth get-or-create client.osd-lockbox.2673183f-b1a7-445c-be22-62ac78054b77 mon allow command "config-key get" with key="dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks" 
create_key: stderr 
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/key-management-mode.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/key-management-mode.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/e936fed2-53dd-4d20-a450-82afdd4d385b -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block-uuid.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/35e71028-e32a-4900-bac3-aa38406f02fc -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.db-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.db-uuid.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/21b7d3ff-41bd-4d0a-aa20-ef39c8283193 -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.wal-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.wal-uuid.5634.tmp
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/magic.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/magic.5634.tmp
command_check_call: Running command: /usr/sbin/sgdisk --typecode=3:fb3aabf9-d25f-47cc-bf5e-721d1816496b -- /dev/sdk
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_data_partition: Creating osd partition on /dev/sdk
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = data
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
create_partition: Creating data partition num 1 size 100 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=1:0:+100M --change-name=1:ceph data --partition-guid=1:2673183f-b1a7-445c-be22-62ac78054b77 --typecode=1:89c57f98-2fe5-4dc0-89c1-5ec00ceff2be --mbrtogpt -- /dev/sdk
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
update_partition: Calling partprobe on created device /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
command: Running command: /usr/bin/flock -s /dev/sdk /usr/sbin/partprobe /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk1 uuid path is /sys/dev/block/8:161/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = block.db
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
create_partition: Creating block.db partition num 3 size 20480 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=3:0:+20480M --change-name=3:ceph block.db --partition-guid=3:35e71028-e32a-4900-bac3-aa38406f02fc --typecode=3:7521c784-4626-4260-bc8d-ba77a0f5f2be --mbrtogpt -- /dev/sdk
Could not create partition 3 from 227328 to 42170367
Error encountered; not saving changes.
'/usr/sbin/sgdisk --new=3:0:+20480M --change-name=3:ceph block.db --partition-guid=3:35e71028-e32a-4900-bac3-aa38406f02fc --typecode=3:7521c784-4626-4260-bc8d-ba77a0f5f2be --mbrtogpt -- /dev/sdk' failed with status code 4

History

#1 Updated by shangzhong zhu 3 months ago

[root@ceph117 ~]# ceph-disk -v prepare --bluestore --block.db /dev/sdk --block.wal /dev/sdk /dev/sdk --dmcrypt

If either --block.db and --block.wal are specified to be the same whole device, ceph-disk will fail.

The detailed info as follow:

command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_type: Will colocate block with data on /dev/sdk
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_db_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup bluestore_block_wal_size
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_or_create_partition: Creating osd partition on /dev/sdk
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = lockbox
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
create_partition: Creating lockbox partition num 3 size 10 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=3:0:+10M --change-name=3:ceph lockbox --partition-guid=3:None --typecode=3:fb3aabf9-d25f-47cc-bf5e-721d181642be --mbrtogpt -- /dev/sdk
The operation has completed successfully.
update_partition: Calling partprobe on created device /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
command: Running command: /usr/bin/flock -s /dev/sdk /usr/sbin/partprobe /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk3 uuid path is /sys/dev/block/8:163/dm/uuid
populate: Creating lockbox fs on %s: mkfs -t ext4 /dev/sdk3
command_check_call: Running command: /usr/sbin/mkfs -t ext4 /dev/sdk3
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
2560 inodes, 10240 blocks
512 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=10485760
2 block groups
8192 blocks per group, 8192 fragments per group
1280 inodes per group
Superblock backups stored on blocks: 
        8193

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (1024 blocks): done
Writing superblocks and filesystem accounting information: done

populate: Mounting lockbox mount -t ext4 /dev/sdk3 /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command_check_call: Running command: /usr/bin/mount -t ext4 /dev/sdk3 /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/osd-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/osd-uuid.5634.tmp
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/ceph_fsid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/ceph_fsid.5634.tmp
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command_check_call: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring config-key put dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks LboRTY8ECOEMZXy9GrDdb6K3RMOkkrNw/dXHsmO0LYxvi2ccRcxrwUemq1NVDIbrdywpGjusHUZaiIsFD75NWMQORJdJCm0fGeVYWfIH24Ko1oBbpS4QkqbHsdjnlF6k6mfwgtH/B2wUq8ePoF7OpxtPh/3V7oyBpyR53FF0YrA=
set dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks
command: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring auth get-or-create client.osd-lockbox.2673183f-b1a7-445c-be22-62ac78054b77 mon allow command "config-key get" with key="dm-crypt/osd/2673183f-b1a7-445c-be22-62ac78054b77/luks" 
create_key: stderr 
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/key-management-mode.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/key-management-mode.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/e936fed2-53dd-4d20-a450-82afdd4d385b -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block-uuid.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/35e71028-e32a-4900-bac3-aa38406f02fc -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.db-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.db-uuid.5634.tmp
adjust_symlink: Creating symlink /var/lib/ceph/osd-lockbox/21b7d3ff-41bd-4d0a-aa20-ef39c8283193 -> /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.wal-uuid.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/block.wal-uuid.5634.tmp
command: Running command: /usr/sbin/restorecon -R /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/magic.5634.tmp
command: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd-lockbox/2673183f-b1a7-445c-be22-62ac78054b77/magic.5634.tmp
command_check_call: Running command: /usr/sbin/sgdisk --typecode=3:fb3aabf9-d25f-47cc-bf5e-721d1816496b -- /dev/sdk
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
set_data_partition: Creating osd partition on /dev/sdk
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = data
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
create_partition: Creating data partition num 1 size 100 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=1:0:+100M --change-name=1:ceph data --partition-guid=1:2673183f-b1a7-445c-be22-62ac78054b77 --typecode=1:89c57f98-2fe5-4dc0-89c1-5ec00ceff2be --mbrtogpt -- /dev/sdk
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot.
The operation has completed successfully.
update_partition: Calling partprobe on created device /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
command: Running command: /usr/bin/flock -s /dev/sdk /usr/sbin/partprobe /dev/sdk
command_check_call: Running command: /usr/bin/udevadm settle --timeout=600
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk1 uuid path is /sys/dev/block/8:161/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
ptype_tobe_for_name: name = block.db
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
get_dm_uuid: get_dm_uuid /dev/sdk uuid path is /sys/dev/block/8:160/dm/uuid
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
create_partition: Creating block.db partition num 3 size 20480 on /dev/sdk
command_check_call: Running command: /usr/sbin/sgdisk --new=3:0:+20480M --change-name=3:ceph block.db --partition-guid=3:35e71028-e32a-4900-bac3-aa38406f02fc --typecode=3:7521c784-4626-4260-bc8d-ba77a0f5f2be --mbrtogpt -- /dev/sdk
Could not create partition 3 from 227328 to 42170367
Error encountered; not saving changes.
'/usr/sbin/sgdisk --new=3:0:+20480M --change-name=3:ceph block.db --partition-guid=3:35e71028-e32a-4900-bac3-aa38406f02fc --typecode=3:7521c784-4626-4260-bc8d-ba77a0f5f2be --mbrtogpt -- /dev/sdk' failed with status code 4

#2 Updated by shangzhong zhu 3 months ago

Root cause:
The partition number 3 has been used by lockbox. The ceph-disk uses the same partition number to create block.db partition.

#3 Updated by Nathan Cutler 3 months ago

  • Description updated (diff)

#5 Updated by Sage Weil 3 months ago

  • Subject changed from ceph-disk fails to prepare an osd device with --dmcrypt to ceph-disk fails to prepare an osd device with --dmcrypt (bluestore)
  • Status changed from New to Verified
  • Priority changed from Normal to Urgent

#6 Updated by Kefu Chai 3 months ago

  • Status changed from Verified to Need Review

#7 Updated by Josh Durgin 3 months ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF