Actions
Bug #24796
closedceph-volume fails with KeyError: 'ceph.cluster_name'
Added by Sébastien Han almost 6 years ago. Updated over 5 years ago.
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
I'm trying to purge /dev/sdb
[root@ceph-osd0 /]# ceph-volume lvm zap /dev/sdb --destroy --> Zapping: /dev/sdb --> KeyError: 'ceph.cluster_name'
Updated by Alfredo Deza almost 6 years ago
can you please add the relevant log output for this call (/var/log/ceph/ceph-volume.log) or rerun with:
CEPH_VOLUME_DEBUG=1 ceph-volume lvm zap /dev/sdb --destroy
And paste that output
Updated by Sébastien Han almost 6 years ago
[2018-07-06 14:36:21,056][ceph_volume.main][INFO ] Running command: ceph-volume lvm zap /dev/sdb
[2018-07-06 14:36:21,056][ceph_volume.process][INFO ] Running command: /usr/sbin/lvs --noheadings --readonly --separator=";" -o lv_tags,lv_path,lv_name,vg_name,lv_uuid
[2018-07-06 14:36:21,285][ceph_volume.process][INFO ] stdout ";"/dev/VolGroup00/LogVol00";"LogVol00";"VolGroup00";"gqFct1-q2eW-5I7a-IQl2-g2Vm-Nsiq-B2CqHo
[2018-07-06 14:36:21,285][ceph_volume.process][INFO ] stdout ";"/dev/VolGroup00/LogVol01";"LogVol01";"VolGroup00";"T8lHub-uZnx-CX9A-354Z-Jd3P-fTPq-dve374
[2018-07-06 14:36:21,285][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2/osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0,ceph.block_uuid=BTOKvd-zxmP-58uL-CAy6-coEt-Ir8e-VDdV16,ceph.cephx_lockbox_secret=AQCbbD9btcZ7NxAA1X+fydCuBVkyZZ6yZfymUw==,ceph.cluster_fsid=0a9374ec-2998-47df-9a7f-d030e5b9c261,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.encrypted=1,ceph.osd_fsid=ea618678-b6d7-489a-9180-a7b67c3b69a0,ceph.osd_id=0,ceph.type=block,ceph.vdo=0";"/dev/ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2/osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0";"osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0";"ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2";"BTOKvd-zxmP-58uL-CAy6-coEt-Ir8e-VDdV16
[2018-07-06 14:36:21,285][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7/osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d,ceph.type=block";"/dev/ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7/osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d";"osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d";"ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7";"iQwkRD-8xsr-Gwzi-YcUe-z93X-48Vy-w923Gn[2018-07-06 14:36:21,285][ceph_volume.devices.lvm.zap][INFO ] Zapping: /dev/sdb
[2018-07-06 14:36:21,286][ceph_volume.process][INFO ] Running command: /usr/sbin/pvs --no-heading --readonly --separator=";" -o pv_name,pv_tags,pv_uuid,vg_name,lv_uuid
[2018-07-06 14:36:21,514][ceph_volume.process][INFO ] stdout /dev/sda";"";"ijcyzY-sgjK-cCfq-aeoB-FstK-Z2Ri-tyzNBr";"ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2";"BTOKvd-zxmP-58uL-CAy6-coEt-Ir8e-VDdV16
[2018-07-06 14:36:21,514][ceph_volume.process][INFO ] stdout /dev/sdb";"";"9zlKVn-J1A8-zwAV-XLKz-gpb6-Wave-IUpQuF";"ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7";"iQwkRD-8xsr-Gwzi-YcUe-z93X-48Vy-w923Gn[2018-07-06 14:36:21,514][ceph_volume.process][INFO ] stdout /dev/vda3";"";"qGTgpA-Amb9-J2GQ-bP3A-2X6n-g5Ud-zPWx6a";"VolGroup00";"gqFct1-q2eW-5I7a-IQl2-g2Vm-Nsiq-B2CqHo
[2018-07-06 14:36:21,514][ceph_volume.process][INFO ] stdout /dev/vda3";"";"qGTgpA-Amb9-J2GQ-bP3A-2X6n-g5Ud-zPWx6a";"VolGroup00";"T8lHub-uZnx-CX9A-354Z-Jd3P-fTPq-dve374[2018-07-06 14:36:21,514][ceph_volume.process][INFO ] Running command: /usr/sbin/lvs --noheadings --readonly --separator=";" -o lv_tags,lv_path,lv_name,vg_name,lv_uuid
[2018-07-06 14:36:21,741][ceph_volume.process][INFO ] stdout ";"/dev/VolGroup00/LogVol00";"LogVol00";"VolGroup00";"gqFct1-q2eW-5I7a-IQl2-g2Vm-Nsiq-B2CqHo
[2018-07-06 14:36:21,741][ceph_volume.process][INFO ] stdout ";"/dev/VolGroup00/LogVol01";"LogVol01";"VolGroup00";"T8lHub-uZnx-CX9A-354Z-Jd3P-fTPq-dve374
[2018-07-06 14:36:21,741][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2/osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0,ceph.block_uuid=BTOKvd-zxmP-58uL-CAy6-coEt-Ir8e-VDdV16,ceph.cephx_lockbox_secret=AQCbbD9btcZ7NxAA1X+fydCuBVkyZZ6yZfymUw==,ceph.cluster_fsid=0a9374ec-2998-47df-9a7f-d030e5b9c261,ceph.cluster_name=ceph,ceph.crush_device_class=None,ceph.encrypted=1,ceph.osd_fsid=ea618678-b6d7-489a-9180-a7b67c3b69a0,ceph.osd_id=0,ceph.type=block,ceph.vdo=0";"/dev/ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2/osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0";"osd-block-ea618678-b6d7-489a-9180-a7b67c3b69a0";"ceph-4d5af038-df98-437a-b038-a2f3f45d0cd2";"BTOKvd-zxmP-58uL-CAy6-coEt-Ir8e-VDdV16
[2018-07-06 14:36:21,741][ceph_volume.process][INFO ] stdout ceph.block_device=/dev/ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7/osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d,ceph.type=block";"/dev/ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7/osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d";"osd-block-5fafadcf-662a-47cf-9bb4-f1912fa8e33d";"ceph-fca417c9-7960-4627-82c9-5954d7d8e7b7";"iQwkRD-8xsr-Gwzi-YcUe-z93X-48Vy-w923Gn[2018-07-06 14:36:21,741][ceph_volume][ERROR ] exception caught by decoratorTraceback (most recent call last): File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw) File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 153, in main terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py", line 38, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/zap.py", line 169, in main
self.zap(args)
File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/zap.py", line 73, in zap
osd_path = "/var/lib/ceph/osd/{}-{}".format(lv.tags['ceph.cluster_name'], lv.tags['ceph.osd_id'])
KeyError: 'ceph.cluster_name'
Updated by Alfredo Deza almost 6 years ago
- Status changed from New to 12
Yeah, I can hit the same in some situations, thanks for reporting this!
Adding CLI output:
(tmp) root@node9:/home/vagrant# CEPH_VOLUME_DEBUG=1 ceph-volume lvm zap --destroy /dev/sdf
--> Zapping: /dev/sdf
Traceback (most recent call last):
File "/home/vagrant/tmp/bin/ceph-volume", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/home/vagrant/ceph-volume/bin/ceph-volume", line 6, in <module>
main.Volume()
File "/home/vagrant/ceph-volume/ceph_volume/main.py", line 40, in __init__
self.main(self.argv)
File "/home/vagrant/ceph-volume/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/home/vagrant/ceph-volume/ceph_volume/main.py", line 156, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/home/vagrant/ceph-volume/ceph_volume/terminal.py", line 182, in dispatch
instance.main()
File "/home/vagrant/ceph-volume/ceph_volume/devices/lvm/main.py", line 40, in main
terminal.dispatch(self.mapper, self.argv)
File "/home/vagrant/ceph-volume/ceph_volume/terminal.py", line 182, in dispatch
instance.main()
File "/home/vagrant/ceph-volume/ceph_volume/devices/lvm/zap.py", line 177, in main
self.zap(args)
File "/home/vagrant/ceph-volume/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/home/vagrant/ceph-volume/ceph_volume/devices/lvm/zap.py", line 78, in zap
osd_path = "/var/lib/ceph/osd/{}-{}".format(lv.tags['ceph.cluster_name'], lv.tags['ceph.osd_id'])
KeyError: 'ceph.cluster_name'
Updated by Alfredo Deza over 5 years ago
- Status changed from 12 to Resolved
- Assignee set to Andrew Schoen
Master PR https://github.com/ceph/ceph/pull/23117
Mimic PR https://github.com/ceph/ceph/pull/23127
Luminous PR https://github.com/ceph/ceph/pull/23128
Updated by Sebastian Wagner over 2 years ago
- Related to Bug #51765: executing create_from_spec_one failed / KeyError: 'ceph.cluster_fsid' added
Actions