Project

General

Profile

Bug #14080 ยป kvsrv01.log

zap+prepare loop log. - Matthew Taylor, 12/17/2015 12:24 AM

 
[ceph@ceph-admin01 openstack_eqsy3]$ cat prepare_2.sh
#!/bin/bash

target_host=kvsrv01

for disk_name in $(cat disks2.txt); do
ceph-deploy disk zap ${target_host}:${disk_name}
ceph-deploy --overwrite-conf osd prepare ${target_host}:${disk_name}
done
[ceph@ceph-admin01 openstack_eqsy3]$ cat disks2.txt
/dev/sdx
/dev/sdw
/dev/sdt
/dev/sds
/dev/sdr
/dev/sdq
/dev/sdn
/dev/sdj
/dev/sdg

[ceph@ceph-admin01 openstack_eqsy3]$ ./prepare_2.sh
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdx
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1eb87a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x1ea9c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdx', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdx on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdx
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdx
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdx
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdx
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdx', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x16e8758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x16d7b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdx:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdx journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdx
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:117456e2-c0fc-4ab7-8604-1ea0d65c95b8 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdx
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdx
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdx failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/117456e2-c0fc-4ab7-8604-1ea0d65c95b8
[kvsrv01][WARNING] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/117456e2-c0fc-4ab7-8604-1ea0d65c95b8
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating osd partition on /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:9dba6dcc-2a7b-412d-8a02-dad2d99ff777 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdx
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on created device /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdx
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdx failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating xfs fs on /dev/sdx1
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdx1
[kvsrv01][DEBUG ] meta-data=/dev/sdx1 isize=2048 agcount=4, agsize=24091717 blks
[kvsrv01][DEBUG ] = sectsz=4096 attr=2, projid32bit=1
[kvsrv01][DEBUG ] = crc=0 finobt=0
[kvsrv01][DEBUG ] data = bsize=4096 blocks=96366865, imaxpct=25
[kvsrv01][DEBUG ] = sunit=0 swidth=0 blks
[kvsrv01][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 ftype=0
[kvsrv01][DEBUG ] log =internal log bsize=4096 blocks=47054, version=2
[kvsrv01][DEBUG ] = sectsz=4096 sunit=1 blks, lazy-count=1
[kvsrv01][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0
[kvsrv01][WARNING] DEBUG:ceph-disk:Mounting /dev/sdx1 on /var/lib/ceph/tmp/mnt.CE_9NZ with options noatime,inode64
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdx1 /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.CE_9NZ/journal -> /dev/disk/by-partuuid/117456e2-c0fc-4ab7-8604-1ea0d65c95b8
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon -R /var/lib/ceph/tmp/mnt.CE_9NZ/ceph_fsid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.CE_9NZ/ceph_fsid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon -R /var/lib/ceph/tmp/mnt.CE_9NZ/fsid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.CE_9NZ/fsid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon -R /var/lib/ceph/tmp/mnt.CE_9NZ/journal_uuid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.CE_9NZ/journal_uuid.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon -R /var/lib/ceph/tmp/mnt.CE_9NZ/magic.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.CE_9NZ/magic.76367.tmp
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/restorecon -R /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.CE_9NZ
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdx uuid path is /sys/dev/block/65:112/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/sdx
[kvsrv01][DEBUG ] Warning: The kernel is still using the old partition table.
[kvsrv01][DEBUG ] The new table will be used at the next reboot.
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdx
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm trigger --action=add --sysname-match sdx1
[kvsrv01][INFO ] checking OSD status...
[kvsrv01][INFO ] Running command: sudo ceph --cluster=ceph osd stat --format=json
[ceph_deploy.osd][DEBUG ] Host kvsrv01 is now ready for osd use.
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdw
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x28c27a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x28b3c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdw', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdw on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdw
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdw
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdw
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdw
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdw', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1804758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x17f3b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdw:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdw journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdw
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdw
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdw uuid path is /sys/dev/block/65:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdw
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:86bcaa75-6fd7-40cc-a1d3-a1dd9fc74c69 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdw
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdw
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdw
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdw failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdw
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdw failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdw
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdw failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdw failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdw
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdt
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x226a7a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x225bc08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdt', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdt on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdt
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdt
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdt
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdt
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdt', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1f14758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x1f03b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdt:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdt journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdt
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdt
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdt uuid path is /sys/dev/block/65:48/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdt
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:8a4d5a0c-21bb-4bf2-a7c9-8291579695e2 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdt
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdt
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdt
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdt failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdt
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdt failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdt
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdt failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdt failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdt
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sds
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x10c17a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x10b2c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sds', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sds on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sds
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sds
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sds
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sds
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sds', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0xf22758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0xf11b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sds:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sds journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sds
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sds
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sds uuid path is /sys/dev/block/65:32/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sds
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:0fc3d75e-0a43-48c0-869c-5c28351fff93 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sds
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sds
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sds
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sds failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sds
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sds failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sds
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sds failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sds failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sds
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdr
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x20367a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x2027c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdr', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdr on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdr
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdr
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdr
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdr
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdr', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1dab758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x1d9ab90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdr:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdr journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdr
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdr
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdr uuid path is /sys/dev/block/65:16/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdr
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:b339516c-b837-418c-ad4e-73e066c9955f --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdr
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdr
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdr
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdr failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdr
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdr failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdr
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdr failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdr failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdr
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdq
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x16b97a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x16aac08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdq', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdq on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdq
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdq
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdq
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdq
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdq', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0xd55758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0xd44b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdq:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdq journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:cddb96da-6ab0-49fc-9bdc-16c40fe59e03 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdq
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdq failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdq failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/cddb96da-6ab0-49fc-9bdc-16c40fe59e03
[kvsrv01][WARNING] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/cddb96da-6ab0-49fc-9bdc-16c40fe59e03
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdq uuid path is /sys/dev/block/65:0/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating osd partition on /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:8261ff6e-545b-42ee-99b1-92ac86bc0af3 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdq
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on created device /dev/sdq
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdq failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdq failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdq
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdq failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1935, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1634, in prepare_dev
[kvsrv01][WARNING] update_partition(data, 'created')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdq failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdq
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdn
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x16627a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x1653c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdn', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdn on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdn
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdn
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdn
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdn
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdn', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x165a758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x1649b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdn:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdn journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdn
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdn
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdn uuid path is /sys/dev/block/8:208/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdn
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:f676f221-9b1d-422f-bfc8-4274a5a6b0a7 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdn
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdn
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdn
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdn failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdn
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdn failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdn
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdn failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdn failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdn
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdj
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x11557a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x1146c08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdj', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdj on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdj
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdj
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdj
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdj
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdj', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x2615758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x2604b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdj:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdj journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdj
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdj
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdj uuid path is /sys/dev/block/8:144/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdj
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:29c11d72-15b7-4275-8887-bb7050308680 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdj
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdj
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdj
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdj failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdj
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdj failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdj
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdj failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdj failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdj
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs

[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy disk zap kvsrv01:/dev/sdg
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : False
[ceph_deploy.cli][INFO ] subcommand : zap
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x11ae7a0>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] func : <function disk at 0x119fc08>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdg', None)]
[ceph_deploy.osd][DEBUG ] zapping /dev/sdg on kvsrv01
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[kvsrv01][DEBUG ] zeroing last few blocks of device
[kvsrv01][DEBUG ] find the location of an executable
[kvsrv01][INFO ] Running command: sudo /usr/sbin/ceph-disk zap /dev/sdg
[kvsrv01][WARNING] Caution: invalid backup GPT header, but valid main header; regenerating
[kvsrv01][WARNING] backup header from main header.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
[kvsrv01][WARNING] on the recovery & transformation menu to examine the two tables.
[kvsrv01][WARNING]
[kvsrv01][WARNING] Warning! One or more CRCs don't match. You should repair the disk!
[kvsrv01][WARNING]
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
[kvsrv01][DEBUG ] verification and recovery are STRONGLY recommended.
[kvsrv01][DEBUG ] ****************************************************************************
[kvsrv01][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or
[kvsrv01][DEBUG ] other utilities.
[kvsrv01][DEBUG ] Creating new GPT entries.
[kvsrv01][DEBUG ] The operation has completed successfully.
[ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdg
[ceph_deploy.osd][INFO ] re-reading known partitions will display errors
[kvsrv01][INFO ] Running command: sudo partx -a /dev/sdg
[ceph_deploy.conf][DEBUG ] found configuration file at: /home/ceph/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.30): /usr/bin/ceph-deploy --overwrite-conf osd prepare kvsrv01:/dev/sdg
[ceph_deploy.cli][INFO ] ceph-deploy options:
[ceph_deploy.cli][INFO ] username : None
[ceph_deploy.cli][INFO ] disk : [('kvsrv01', '/dev/sdg', None)]
[ceph_deploy.cli][INFO ] dmcrypt : False
[ceph_deploy.cli][INFO ] verbose : False
[ceph_deploy.cli][INFO ] overwrite_conf : True
[ceph_deploy.cli][INFO ] subcommand : prepare
[ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys
[ceph_deploy.cli][INFO ] quiet : False
[ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x2142758>
[ceph_deploy.cli][INFO ] cluster : ceph
[ceph_deploy.cli][INFO ] fs_type : xfs
[ceph_deploy.cli][INFO ] func : <function osd at 0x2131b90>
[ceph_deploy.cli][INFO ] ceph_conf : None
[ceph_deploy.cli][INFO ] default_release : False
[ceph_deploy.cli][INFO ] zap_disk : False
[ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks kvsrv01:/dev/sdg:
[kvsrv01][DEBUG ] connection detected need for sudo
[kvsrv01][DEBUG ] connected to host: kvsrv01
[kvsrv01][DEBUG ] detect platform information from remote host
[kvsrv01][DEBUG ] detect machine type
[kvsrv01][DEBUG ] find the location of an executable
[ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[ceph_deploy.osd][DEBUG ] Deploying osd to kvsrv01
[kvsrv01][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[ceph_deploy.osd][DEBUG ] Preparing host kvsrv01 disk /dev/sdg journal None activate False
[kvsrv01][INFO ] Running command: sudo ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdg
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --cluster ceph
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --cluster ceph
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] INFO:ceph-disk:Will colocate journal with data on /dev/sdg
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:get_dm_uuid /dev/sdg uuid path is /sys/dev/block/8:96/dm/uuid
[kvsrv01][WARNING] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdg
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:a212c544-9085-4f07-9ab4-285a6206a407 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdg
[kvsrv01][DEBUG ] The operation has completed successfully.
[kvsrv01][WARNING] DEBUG:ceph-disk:Calling partprobe on prepared device /dev/sdg
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdg
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdg failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdg
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdg failed : unknown error (ignored)
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /usr/bin/udevadm settle --timeout=600
[kvsrv01][WARNING] INFO:ceph-disk:Running command: /sbin/partprobe /dev/sdg
[kvsrv01][WARNING] DEBUG:ceph-disk:partprobe /dev/sdg failed : unknown error (ignored)
[kvsrv01][WARNING] Traceback (most recent call last):
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4055, in <module>
[kvsrv01][WARNING] main(sys.argv[1:])
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 4009, in main
[kvsrv01][WARNING] args.func(args)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1905, in main_prepare
[kvsrv01][WARNING] luks=luks
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1507, in prepare_journal
[kvsrv01][WARNING] return prepare_journal_dev(data, journal, journal_size, journal_uuid, journal_dm_keypath, cryptsetup_parameters, luks)
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1422, in prepare_journal_dev
[kvsrv01][WARNING] update_partition(journal, 'prepared')
[kvsrv01][WARNING] File "/sbin/ceph-disk", line 1262, in update_partition
[kvsrv01][WARNING] raise Error('partprobe %s failed : %s' % (dev, error))
[kvsrv01][WARNING] __main__.Error: Error: partprobe /dev/sdg failed : unknown error
[kvsrv01][ERROR ] RuntimeError: command returned non-zero exit status: 1
[ceph_deploy.osd][ERROR ] Failed to execute command: ceph-disk -v prepare --cluster ceph --fs-type xfs -- /dev/sdg
[ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs
    (1-1/1)