Bug #58137
OSD prepare job fails with KeyError: 'KNAME'
% Done:
0%
Source:
Tags:
backport_processed
Backport:
quincy,pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Description
[2022-11-13 18:27:04,778][ceph_volume.devices.raw.prepare][ERROR ] raw prepare was unable to complete
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 91, in safe_prepare
self.prepare()
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 134, in prepare
tmpfs,
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 51, in prepare_bluestore
block = prepare_dmcrypt(key, block, 'block', fsid)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 23, in prepare_dmcrypt
kname = disk.lsblk(device)['KNAME']
KeyError: 'KNAME'
[2022-11-13 18:27:04,780][ceph_volume.devices.raw.prepare][INFO ] will rollback OSD ID creation
[2022-11-13 18:27:04,781][ceph_volume.process][INFO ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.4 --yes-i-really-mean-it
[2022-11-13 18:27:05,553][ceph_volume.process][INFO ] stderr purged osd.4
[2022-11-13 18:27:05,571][ceph_volume.process][INFO ] Running command: /usr/bin/systemctl is-active ceph-osd@4
[2022-11-13 18:27:05,584][ceph_volume.process][INFO ] stderr System has not been booted with systemd as init system (PID 1). Can't operate.
[2022-11-13 18:27:05,585][ceph_volume.process][INFO ] stderr Failed to connect to bus: Host is down
[2022-11-13 18:27:05,589][ceph_volume.util.system][WARNING] Executable lvs not found on the host, will return lvs as-is
[2022-11-13 18:27:05,590][ceph_volume.process][INFO ] Running command: lvs --noheadings --readonly --separator=";" -a --units=b --nosuffix -S tags={ceph.osd_id=4} -o lv_tags,lv_path,lv_name,vg_name,lv_uuid,lv_size
[2022-11-13 18:27:05,969][ceph_volume][ERROR ] exception caught by decorator
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 91, in safe_prepare
self.prepare()
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 134, in prepare
tmpfs,
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 51, in prepare_bluestore
block = prepare_dmcrypt(key, block, 'block', fsid)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 23, in prepare_dmcrypt
kname = disk.lsblk(device)['KNAME']
KeyError: 'KNAME'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 153, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/main.py", line 32, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 169, in main
self.safe_prepare(self.args)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/prepare.py", line 95, in safe_prepare
rollback_osd(self.args, self.osd_id)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/common.py", line 35, in rollback_osd
Zap(['--destroy', '--osd-id', osd_id]).main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/zap.py", line 404, in main
self.zap_osd()
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/zap.py", line 301, in zap_osd
devices = find_associated_devices(self.args.osd_id, self.args.osd_fsid)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/lvm/zap.py", line 88, in find_associated_devices
'%s' % osd_id or osd_fsid)
RuntimeError: Unable to find any LV for zapping OSD: 4
see https://github.com/rook/rook/issues/11304 for details
Related issues
History
#1 Updated by Guillaume Abrioux 2 months ago
- Status changed from In Progress to Fix Under Review
- Pull request ID set to 49171
#2 Updated by Guillaume Abrioux 13 days ago
- Status changed from Fix Under Review to Pending Backport
#3 Updated by Backport Bot 13 days ago
- Copied to Backport #58574: quincy: OSD prepare job fails with KeyError: 'KNAME' added
#4 Updated by Backport Bot 13 days ago
- Copied to Backport #58575: pacific: OSD prepare job fails with KeyError: 'KNAME' added
#5 Updated by Backport Bot 13 days ago
- Tags set to backport_processed