Actions
Bug #43856
openceph-volume inventory --format json fails with 'KeyError: 'ceph.cluster_name''
% Done:
0%
Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Admittedly it's a messy node:
> admin:~ # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT vda 254:0 0 42G 0 disk ├─vda1 254:1 0 2M 0 part ├─vda2 254:2 0 20M 0 part /boot/efi └─vda3 254:3 0 42G 0 part / vdb 254:16 0 25G 0 disk └─ceph--foo-osd--data--foo 253:1 0 24G 0 lvm vdc 254:32 0 25G 0 disk vdd 254:48 0 25G 0 disk vde 254:64 0 25G 0 disk vdf 254:80 0 25G 0 disk vdg 254:96 0 25G 0 disk └─ceph--0380f4e0--0815--4ae9--9127--0bfcb47421d2-osd--data--f29088ea--63c4--4c30--a4d1--86e6edbcdab1 253:0 0 24G 0 lvm vdh 254:112 0 25G 0 disk vdi 254:128 0 25G 0 disk vdj 254:144 0 25G 0 disk vdk 254:160 0 25G 0 disk └─vdk1 254:161 0 25G 0 part ├─foo-bar 253:5 0 20G 0 lvm └─foo-lvstuff 253:6 0 3G 0 lvm vdl 254:176 0 25G 0 disk vdm 254:192 0 25G 0 disk vdn 254:208 0 25G 0 disk vdo 254:224 0 25G 0 disk vdp 254:240 0 25G 0 disk vdq 254:256 0 25G 0 disk └─ceph--0e6fa561--a021--4815--b90e--5793e09bacae-osd--block--902a942b--7ab6--4278--9078--61360f413294 253:4 0 24G 0 lvm vdr 254:272 0 25G 0 disk vds 254:288 0 25G 0 disk └─ceph--ac9c3398--9e94--4be5--98c5--5a26306f39c1-osd--block--3315307a--08d9--4d64--809a--53aaa5cf2943 253:3 0 24G 0 lvm vdt 254:304 0 25G 0 disk └─ceph--0ed54b3e--589c--472b--b864--0c42d514b63c-osd--block--4383d060--d500--46db--bc78--9d4930c6930a 253:2 0 24G 0 lvm vdu 254:320 0 25G 0 disk └─ceph--f6380ab4--c69c--4a33--876f--eb6d5a6b49bd-osd--block--a14be65f--9478--4ed9--946a--bc5f4e31ff9b 253:7 0 24G 0 lvm
but this case should be handled though:
find the commands used to trigger this issue here:
https://gist.github.com/jschmid1/4bf2970e8748bcf02bd7120b3c550f46
admin:~ # ceph-volume inventory
stderr: unable to read label for /dev/vda2: (2) No such file or directory
stderr: unable to read label for /dev/vda3: (2) No such file or directory
stderr: unable to read label for /dev/vda1: (2) No such file or directory
stderr: unable to read label for /dev/vda: (2) No such file or directory
stderr: unable to read label for /dev/vdb: (2) No such file or directory
stderr: unable to read label for /dev/vdc: (2) No such file or directory
stderr: unable to read label for /dev/vdd: (2) No such file or directory
stderr: unable to read label for /dev/vde: (2) No such file or directory
stderr: unable to read label for /dev/vdf: (2) No such file or directory
stderr: unable to read label for /dev/vdg: (2) No such file or directory
stderr: unable to read label for /dev/vdh: (2) No such file or directory
stderr: unable to read label for /dev/vdi: (2) No such file or directory
stderr: unable to read label for /dev/vdj: (2) No such file or directory
stderr: unable to read label for /dev/vdk1: (2) No such file or directory
stderr: unable to read label for /dev/vdk: (2) No such file or directory
stderr: unable to read label for /dev/vdl: (2) No such file or directory
stderr: unable to read label for /dev/vdm: (2) No such file or directory
stderr: unable to read label for /dev/vdn: (2) No such file or directory
stderr: unable to read label for /dev/vdo: (2) No such file or directory
stderr: unable to read label for /dev/vdp: (2) No such file or directory
stderr: unable to read label for /dev/vdq: (2) No such file or directory
stderr: unable to read label for /dev/vdr: (2) No such file or directory
stderr: unable to read label for /dev/vds: (2) No such file or directory
stderr: unable to read label for /dev/vdt: (2) No such file or directory
stderr: unable to read label for /dev/vdu: (2) No such file or directory
Device Path Size rotates available Model name
/dev/vdc 25.00 GB True True
/dev/vdd 25.00 GB True True
/dev/vde 25.00 GB True True
/dev/vdf 25.00 GB True True
/dev/vdh 25.00 GB True True
/dev/vdi 25.00 GB True True
/dev/vdj 25.00 GB True True
/dev/vdl 25.00 GB True True
/dev/vdm 25.00 GB True True
/dev/vdn 25.00 GB True True
/dev/vdo 25.00 GB True True
/dev/vdp 25.00 GB True True
/dev/vdr 25.00 GB True True
/dev/vda 42.00 GB True False
/dev/vdb 25.00 GB True False
/dev/vdg 25.00 GB True False
/dev/vdk 25.00 GB True False
/dev/vdq 25.00 GB True False
/dev/vds 25.00 GB True False
/dev/vdt 25.00 GB True False
/dev/vdu 25.00 GB True False
admin:~ # ceph-volume inventory --format=json
stderr: unable to read label for /dev/vda2: (2) No such file or directory
stderr: unable to read label for /dev/vda3: (2) No such file or directory
stderr: unable to read label for /dev/vda1: (2) No such file or directory
stderr: unable to read label for /dev/vda: (2) No such file or directory
stderr: unable to read label for /dev/vdb: (2) No such file or directory
stderr: unable to read label for /dev/vdc: (2) No such file or directory
stderr: unable to read label for /dev/vdd: (2) No such file or directory
stderr: unable to read label for /dev/vde: (2) No such file or directory
stderr: unable to read label for /dev/vdf: (2) No such file or directory
stderr: unable to read label for /dev/vdg: (2) No such file or directory
stderr: unable to read label for /dev/vdh: (2) No such file or directory
stderr: unable to read label for /dev/vdi: (2) No such file or directory
stderr: unable to read label for /dev/vdj: (2) No such file or directory
stderr: unable to read label for /dev/vdk1: (2) No such file or directory
stderr: unable to read label for /dev/vdk: (2) No such file or directory
stderr: unable to read label for /dev/vdl: (2) No such file or directory
stderr: unable to read label for /dev/vdm: (2) No such file or directory
stderr: unable to read label for /dev/vdn: (2) No such file or directory
stderr: unable to read label for /dev/vdo: (2) No such file or directory
stderr: unable to read label for /dev/vdp: (2) No such file or directory
stderr: unable to read label for /dev/vdq: (2) No such file or directory
stderr: unable to read label for /dev/vdr: (2) No such file or directory
stderr: unable to read label for /dev/vds: (2) No such file or directory
stderr: unable to read label for /dev/vdt: (2) No such file or directory
stderr: unable to read label for /dev/vdu: (2) No such file or directory
--> KeyError: 'ceph.cluster_name'
admin:~ #
Traceback (most recent call last):
File "/usr/sbin/ceph-volume", line 11, in <module>
load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()
File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 39, in __init__
self.main(self.argv)
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 150, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/inventory/main.py", line 38, in main
self.format_report(Devices())
File "/usr/lib/python3.6/site-packages/ceph_volume/inventory/main.py", line 42, in format_report
print(json.dumps(inventory.json_report()))
File "/usr/lib/python3.6/site-packages/ceph_volume/util/device.py", line 51, in json_report
output.append(device.json_report())
File "/usr/lib/python3.6/site-packages/ceph_volume/util/device.py", line 196, in json_report
output['lvs'] = [lv.report() for lv in self.lvs]
File "/usr/lib/python3.6/site-packages/ceph_volume/util/device.py", line 196, in <listcomp>
output['lvs'] = [lv.report() for lv in self.lvs]
File "/usr/lib/python3.6/site-packages/ceph_volume/api/lvm.py", line 945, in report
'cluster_name': self.tags['ceph.cluster_name'],
KeyError: 'ceph.cluster_name'
jxs@zulu ~/projects/ceph/build ±drive_group_ssh⚡ » ceph orchestrator osd create ssh-dev1:foo/bar
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-01-28T12:33:13.301+0100 7f2eecc50700 -1 WARNING: all dangerous and experimental features are enabled.
2020-01-28T12:33:13.333+0100 7f2eecc50700 -1 WARNING: all dangerous and experimental features are enabled.
Error EINVAL: Traceback (most recent call last):
File "/home/jxs/projects/ceph/src/pybind/mgr/mgr_module.py", line 1064, in _handle_command
return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
File "/home/jxs/projects/ceph/src/pybind/mgr/mgr_module.py", line 304, in call
return self.func(mgr, **kwargs)
File "/home/jxs/projects/ceph/src/pybind/mgr/orchestrator.py", line 140, in wrapper
return func(*args, **kwargs)
File "/home/jxs/projects/ceph/src/pybind/mgr/orchestrator_cli/module.py", line 366, in _create_osd
orchestrator.raise_if_exception(completion)
File "/home/jxs/projects/ceph/src/pybind/mgr/orchestrator.py", line 655, in raise_if_exception
raise e
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/home/jxs/projects/ceph/src/pybind/mgr/cephadm/module.py", line 132, in do_work
res = self._on_complete_(*args, **kwargs)
File "/home/jxs/projects/ceph/src/pybind/mgr/cephadm/module.py", line 189, in <lambda>
return cls(on_complete=lambda x: f(*x), value=value, name=name, **c_kwargs)
File "/home/jxs/projects/ceph/src/pybind/mgr/cephadm/module.py", line 922, in _get_inventory
['--', 'inventory', '--format=json'])
File "/home/jxs/projects/ceph/src/pybind/mgr/cephadm/module.py", line 681, in _run_cephadm
code, '\n'.join(err)))
RuntimeError: cephadm exited with an error code: 1, stderr:INFO:cephadm:/usr/bin/podman:stderr --> KeyError: 'ceph.cluster_name'
Traceback (most recent call last):
File "<stdin>", line 2705, in <module>
File "<stdin>", line 545, in _infer_fsid
File "<stdin>", line 1981, in command_ceph_volume
File "<stdin>", line 474, in call_throws
RuntimeError: Failed command: /usr/bin/podman run --rm --net=host --privileged -e CONTAINER_IMAGE=ceph/daemon-base:latest-master-devel -e NODE_NAME=admin -v /var/log/ceph/307fa37f-5447-4436-8266-3366ed055a60:/var/log/ceph:z -v /var/lib/ceph/307fa37f-5447-4436-8266-3366ed055a60/crash:/var/lib/ceph/crash:z -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm --entrypoint /usr/sbin/ceph-volume ceph/daemon-base:latest-master-devel inventory --format=json
Actions