Ceph » ceph-volume

Running vgdisplay showed that ceph-volume tried to create a disk on failed disk. (We didn't know we had a bad did so this is information that was new to us) and when the command failed it left three bad volume groups. Since you cannot rename them you need to use the following command to delete them.

vgdisplay to find the bad volume groups

vgremove --select vg_uuid=your uuid -f # -f forces it to be removed

[root@ceph-storage3 ~]# ceph-volume lvm create --bluestore --data /dev/sdu 
Running command: ceph-authtool --gen-print-key
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 664894a8-530a-4557-b2f4-1af5b391f2b7
--> Was unable to complete a new OSD, will rollback changes
--> OSD will be fully purged from the cluster, because the ID was generated
Running command: ceph osd purge osd.140 --yes-i-really-mean-it
 stderr: purged osd.140
Traceback (most recent call last):
  File "/sbin/ceph-volume", line 6, in <module>
    main.Volume()
  File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 37, in __init__
    self.main(self.argv)
  File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59, in newfunc
    return f(*a, **kw)
  File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 153, in main
    terminal.dispatch(self.mapper, subcommand_args)
  File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch
    instance.main()
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py", line 38, in main
    terminal.dispatch(self.mapper, self.argv)
  File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in dispatch
    instance.main()
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/create.py", line 74, in main
    self.create(args)
  File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root
    return func(*a, **kw)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/create.py", line 26, in create
    prepare_step.safe_prepare(args)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/prepare.py", line 217, in safe_prepare
    self.prepare(args)
  File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root
    return func(*a, **kw)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/prepare.py", line 283, in prepare
    block_lv = self.prepare_device(args.data, 'block', cluster_fsid, osd_fsid)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/prepare.py", line 193, in prepare_device
    if api.get_vg(vg_name=vg_name):
  File "/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py", line 334, in get_vg
    return vgs.get(vg_name=vg_name, vg_tags=vg_tags)
  File "/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py", line 429, in get
    raise MultipleVGsError(vg_name)
ceph_volume.exceptions.MultipleVGsError: Got more than 1 result looking for volume group: ceph-6a2e8f21-bca2-492b-8869-eecc995216cc