Project

General

Profile

Bug #4984

ceph_deploy: osd create succeeds with an error message (partprobe returns error)

Added by Tamilarasi muthamizhan almost 11 years ago. Updated almost 11 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
ceph-deploy
Target version:
-
% Done:

100%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

this is not consistent though, recently hit this on rhel 6.3,

tamil@ubuntu:~/ceph-deploy-latest/rhel1/ceph-deploy$ ./ceph-deploy osd create burnupi28:sde --zap-disk ********************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended. ********************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
The operation has completed successfully.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Traceback (most recent call last):
File "./ceph-deploy", line 9, in <module>
load_entry_point('ceph-deploy==0.1', 'console_scripts', 'ceph-deploy')()
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/ceph_deploy/cli.py", line 112, in main
return args.func(args)
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/ceph_deploy/osd.py", line 293, in osd
prepare(args, cfg, activate_prepared_disk=True)
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/ceph_deploy/osd.py", line 177, in prepare
dmcrypt_dir=args.dmcrypt_key_dir,
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/proxy.py", line 255, in <lambda>
(conn.operator(type_, self, args, kwargs))
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/connection.py", line 66, in operator
return self.send_request(type_, (object, args, kwargs))
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/baseconnection.py", line 323, in send_request
return self.__handle(m)
File "/home/tamil/ceph-deploy-latest/rhel1/ceph-deploy/virtualenv/local/lib/python2.7/site-packages/pushy-0.5.1-py2.7.egg/pushy/protocol/baseconnection.py", line 639, in __handle
raise e
pushy.protocol.proxy.ExceptionProxyInformation: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
Information: Moved requested sector from 2097153 to 2099200 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
meta-data=/dev/sde1 isize=2048 agcount=4, agsize=60948415 blks = sectsz=512 attr=2
data = bsize=4096 blocks=243793659, imaxpct=25 = sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=119039, version=2 = sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The operation has completed successfully.
Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!

INFO:ceph-disk:Will colocate journal with data on /dev/sde
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sde (Device or resource busy). As a result, it may not reflect all of your changes until after reboot.
ceph-disk: Error: Command '['partprobe', '/dev/sde']' returned non-zero exit status 1
: Command '['ceph-disk-prepare', '--zap-disk', '--', '/dev/sde']' returned non-zero exit status 1

[ubuntu@burnupi28 ceph-noarch]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 917G 5.3G 912G 1% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
/dev/sde1 930G 35M 930G 1% /var/lib/ceph/osd/ceph-0


Subtasks

Bug #5259: osd create command fails inconsistently on ubuntuDuplicate


Related issues

Duplicated by devops - Bug #5107: ceph-deploy: on centos 6.3, osd create command should be cleaned up Duplicate 05/17/2013

History

#1 Updated by Anonymous almost 11 years ago

  • Priority changed from Normal to High

#2 Updated by Sage Weil almost 11 years ago

  • Subject changed from ceph_deploy: osd create succeeds with an error message to ceph_deploy: osd create succeeds with an error message (partprobe returns error)

#3 Updated by Anonymous almost 11 years ago

  • Priority changed from High to Urgent

#4 Updated by Ian Colle almost 11 years ago

  • Status changed from New to 7
  • Assignee set to Tamilarasi muthamizhan

Please confirm if this is still happening or not.

#5 Updated by Tamilarasi muthamizhan almost 11 years ago

yes, the problem still exists.

[ubuntu@burnupi21 ceph-deploy]$ ./ceph-deploy osd create burnupi21:sdc burnupi21:sdd --zap-disk
ceph-disk-prepare --zap-disk -- /dev/sdd returned 1
****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
GPT data structures destroyed! You may now partition the disk using fdisk or
other utilities.
The operation has completed successfully.
Information: Moved requested sector from 34 to 2048 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
Information: Moved requested sector from 2097153 to 2099200 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
meta-data=/dev/sdd1              isize=2048   agcount=4, agsize=60948415 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=243793659, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=119039, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
The operation has completed successfully.

Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! Main and backup partition tables differ! Use the 'c' and 'e' options
on the recovery & transformation menu to examine the two tables.

Warning! One or more CRCs don't match. You should repair the disk!

INFO:ceph-disk:Will colocate journal with data on /dev/sdd
Warning: WARNING: the kernel failed to re-read the partition table on /dev/sdd (Device or resource busy).  As a result, it may not reflect all of your changes until after reboot.
ceph-disk: Error: Command '['partprobe', '/dev/sdd']' returned non-zero exit status 1

ceph-deploy: Failed to create 1 OSDs

[ubuntu@burnupi21 ceph-deploy]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             917G  3.3G  914G   1% /
tmpfs                 7.8G     0  7.8G   0% /dev/shm
/dev/sdc1             930G   35M  930G   1% /var/lib/ceph/osd/ceph-0
/dev/sdd1             930G   33M  930G   1% /var/lib/ceph/osd/ceph-1

#6 Updated by Tamilarasi muthamizhan almost 11 years ago

  • Status changed from 7 to New
  • Assignee deleted (Tamilarasi muthamizhan)

#7 Updated by Sage Weil almost 11 years ago

  • Assignee set to Anonymous

#8 Updated by Sage Weil almost 11 years ago

  • Assignee changed from Anonymous to Sage Weil

#9 Updated by Sage Weil almost 11 years ago

  • Status changed from New to 7
  • Assignee changed from Sage Weil to Tamilarasi muthamizhan

pushed wip-4948. works ok on centos/rhel, but we should verify it also behaves on ubuntu and debian.

#10 Updated by Tamilarasi muthamizhan almost 11 years ago

it should have been wip-4984 :)

#11 Updated by Sage Weil almost 11 years ago

  • Status changed from 7 to In Progress
  • Assignee changed from Tamilarasi muthamizhan to Sage Weil

#12 Updated by Sage Weil almost 11 years ago

  • Status changed from In Progress to Resolved

woot! tested and backported to cuttlefish!

still issues on reboot with wheezy... #5194

Also available in: Atom PDF