Bug #5107
closedceph-deploy: on centos 6.3, osd create command should be cleaned up
0%
Description
on centos 6.3, [burnupi05, burnupi21]
while osd create command when used with zapdisk option, does create osds successfully [disks mounted and osd daemon running], it reports "failed to create osds" at the end of the command."zapdisk" option doesn't seem to zap the entire disk.so results in error sometimes.
[ubuntu@burnupi05 ceph-deploy]$ ./ceph-deploy osd create burnupi05:sdb burnupi21:sdc --zap-disk
ceph-disk-prepare --zap-disk -- /dev/sdb returned 1
****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
The operation has completed successfully.
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
Information: Moved requested sector from 2097153 to 2099200 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
meta-data=/dev/sdb1 isize=2048 agcount=4, agsize=60948415 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=243793659, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=119039, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The operation has completed successfully.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.
Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.
Warning! One or more CRCs don't match. You should repair the disk!
INFO:ceph-disk:Will colocate journal with data on /dev/sdb
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sdb (Device or resource busy). As a result, it may not reflect all of your changes until after reboot.
ceph-disk: Error: Command '['partprobe', '/dev/sdb']' returned non-zero exit status 1
ceph-disk-prepare --zap-disk -- /dev/sdc returned 1
****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
The operation has completed successfully.
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
Information: Moved requested sector from 2097153 to 2099200 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
meta-data=/dev/sdc1 isize=2048 agcount=4, agsize=60948415 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=243793659, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=119039, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The operation has completed successfully.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.
Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.
Warning! One or more CRCs don't match. You should repair the disk!
INFO:ceph-disk:Will colocate journal with data on /dev/sdc
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sdc (Device or resource busy). As a result, it may not reflect all of your changes until after reboot.
ceph-disk: Error: Command '['partprobe', '/dev/sdc']' returned non-zero exit status 1
ceph-deploy: Failed to create 2 OSDs
[ubuntu@burnupi05 ceph-deploy]$ ./ceph-deploy disk list burnupi21
/dev/sda :
/dev/sda1 other, ext4, mounted on /
/dev/sdb :
/dev/sdb1 other, ext4
/dev/sdc :
/dev/sdc1 ceph data, active, cluster ceph, osd.1, journal /dev/sdc2
/dev/sdc2 ceph journal, for /dev/sdc1
/dev/sdd :
/dev/sdd1 ceph data, prepared, unknown cluster c653a3d8-4b27-422b-b7a2-60c62a2afcad, osd.0, journal /dev/sdd2
/dev/sdd2 ceph journal, for /dev/sdd1
/dev/sde :
/dev/sde1 ceph data, prepared, unknown cluster 98cf9617-b5c8-4399-a0ec-7f5a638390f6, osd.0, journal /dev/sde2
/dev/sde2 ceph journal, for /dev/sde1
/dev/sdf :
/dev/sdf1 other, xfs
/dev/sdg :
/dev/sdg1 other, ext4
/dev/sdh :
/dev/sdh1 other, ext4
/dev/sdi :
/dev/sdi1 other
/dev/sdi2 other
/dev/sdi3 other
/dev/sdi4 other
[ubuntu@burnupi05 ceph-deploy]$ ./ceph-deploy disk list burnupi05
/dev/sda :
/dev/sda1 other, ext4, mounted on /
/dev/sdb :
/dev/sdb1 ceph data, active, cluster ceph, osd.0, journal /dev/sdb2
/dev/sdb2 ceph journal, for /dev/sdb1
/dev/sdc :
/dev/sdc1 other, xfs
/dev/sdc2 other
/dev/sdd other, unknown
/dev/sde :
/dev/sde1 ceph data, unprepared
/dev/sde2 ceph journal
/dev/sdf other, ext4
/dev/sdg :
/dev/sdg1 ceph data, unprepared
/dev/sdg2 ceph journal
/dev/sdh other, ext4
/dev/sdi other, ext4
on burnupi05:
[ubuntu@burnupi05 ceph-deploy]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 917G 3.4G 913G 1% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
/dev/sdb1 930G 36M 930G 1% /var/lib/ceph/osd/ceph-0
[ubuntu@burnupi05 ceph-deploy]$ ps -ef | grep ceph
root 39145 1 0 16:46 pts/0 00:00:01 /usr/bin/ceph-mon -i burnupi05 --pid-file /var/run/ceph/mon.burnupi05.pid -c /etc/ceph/ceph.conf
root 40535 1 1 16:47 ? 00:00:02 /usr/bin/ceph-osd -i 0 --pid-file /var/run/ceph/osd.0.pid -c /etc/ceph/ceph.conf
ubuntu 40890 64481 0 16:51 pts/0 00:00:00 grep ceph
on burnupi21:
[ubuntu@burnupi21 .ssh]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 917G 3.3G 914G 1% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
/dev/sdc1 930G 35M 930G 1% /var/lib/ceph/osd/ceph-1
[ubuntu@burnupi21 .ssh]$ ps -ef | grep ceph
root 37445 1 0 16:46 ? 00:00:00 /usr/bin/ceph-mon -i burnupi21 --pid-file /var/run/ceph/mon.burnupi21.pid -c /etc/ceph/ceph.conf
root 38985 1 0 16:48 ? 00:00:00 /usr/bin/ceph-osd -i 1 --pid-file /var/run/ceph/osd.1.pid -c /etc/ceph/ceph.conf
ubuntu 39412 56074 0 16:51 pts/0 00:00:00 grep ceph
Updated by Tamilarasi muthamizhan almost 11 years ago
- Subject changed from ceph-deploy: on centos 6.3, zapdisk needs cleanup to ceph-deploy: on centos 6.3, osd create command should be cleaned up
[ubuntu@burnupi05 ceph-deploy]$ ./ceph-deploy osd create burnupi05:sdc
ceph-disk-prepare -- /dev/sdc returned 1
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
Information: Moved requested sector from 2097153 to 2099200 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
meta-data=/dev/sdc1 isize=2048 agcount=4, agsize=60948415 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=243793659, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=119039, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The operation has completed successfully.
INFO:ceph-disk:Will colocate journal with data on /dev/sdc
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sdc (Device or resource busy). As a result, it may not reflect all of your changes until after reboot.
ceph-disk: Error: Command '['partprobe', '/dev/sdc']' returned non-zero exit status 1
ceph-deploy: Failed to create 1 OSDs
looks like the osd create command has not cleared the disk completely when trying to create a osd and so the error even though the command is succesful.
It would be better to cleanup the osd create command to be better so as to avoid confusion to the users.