Bug #23498
ceph-volume: unable to use existing partition as filestore journal
0%
Description
Followup to issue 23497.
Running Luminous 12.2.4++ on Ubuntu Trusty 14.04.5.
This is trying to convert ceph-disk & even older setups to ceph-volume.
I patched ceph-volume to not pass --yes to lvcreate per issue 23497, but still ended up bailing out because a partition, esp one that is zeroed out, won't have a PARTUUID, or even reliably show up in blkid.
ceph-volume should accept an existing partition as the journal. There's lots of other OSDs with their journals on the NVME device already.
root@cephstore12145:/# ceph-volume lvm create --filestore --journal /dev/nvme0n1p65 --data /dev/sde --dmcrypt Running command: ceph-authtool --gen-print-key Running command: ceph-authtool --gen-print-key Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 8c599b90-a39d-4eb1-9165-87fb73015674 stderr: lsblk: unknown column: PKNAME,PARTLABEL stderr: lsblk: unknown column: PKNAME,PARTLABEL Running command: vgcreate --force --yes ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef /dev/sde stderr: No physical volume label read from /dev/sde stdout: Physical volume "/dev/sde" successfully created stdout: Volume group "ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef" successfully created Running command: lvcreate -l 100%FREE -n osd-data-8c599b90-a39d-4eb1-9165-87fb73015674 ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef stdout: Logical volume "osd-data-8c599b90-a39d-4eb1-9165-87fb73015674" created --> blkid could not detect a PARTUUID for device: /dev/nvme0n1p65 --> Was unable to complete a new OSD, will rollback changes --> OSD will be fully purged from the cluster, because the ID was generated Running command: ceph osd purge osd.110 --yes-i-really-mean-it stderr: purged osd.110 --> RuntimeError: unable to use device # parted /dev/nvme0n1 'unit MiB' print Model: Unknown (unknown) Disk /dev/nvme0n1: 457863MiB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 65 1.00MiB 1025MiB 1024MiB ceph journal 66 1025MiB 2049MiB 1024MiB ceph journal 67 2049MiB 3073MiB 1024MiB ceph journal 68 3073MiB 4097MiB 1024MiB ceph journal 69 4097MiB 5121MiB 1024MiB ceph journal 70 5121MiB 6145MiB 1024MiB ceph journal 71 6145MiB 7169MiB 1024MiB ceph journal 72 7169MiB 8193MiB 1024MiB ceph journal 73 8193MiB 9217MiB 1024MiB ceph journal 74 9217MiB 10241MiB 1024MiB ceph journal 76 11265MiB 12289MiB 1024MiB ceph journal 77 12289MiB 13313MiB 1024MiB ceph journal 78 13313MiB 14337MiB 1024MiB ceph journal 79 14337MiB 15361MiB 1024MiB ceph journal 80 15361MiB 16385MiB 1024MiB ceph journal 81 16385MiB 17409MiB 1024MiB ceph journal 83 18433MiB 19457MiB 1024MiB ceph journal 84 19457MiB 20481MiB 1024MiB ceph journal 85 20481MiB 21505MiB 1024MiB ceph journal 86 21505MiB 22529MiB 1024MiB ceph journal 87 22529MiB 23553MiB 1024MiB ceph journal 88 23553MiB 24577MiB 1024MiB ceph journal 89 24577MiB 25601MiB 1024MiB ceph journal 90 25601MiB 26625MiB 1024MiB ceph journal 91 26625MiB 27649MiB 1024MiB ceph journal 92 27649MiB 28673MiB 1024MiB ceph journal 93 28673MiB 29697MiB 1024MiB ceph journal 94 29697MiB 30721MiB 1024MiB ceph journal 95 30721MiB 31745MiB 1024MiB ceph journal 96 31745MiB 32769MiB 1024MiB ceph journal 97 32769MiB 33793MiB 1024MiB ceph journal 98 33793MiB 34817MiB 1024MiB ceph journal 99 34817MiB 35841MiB 1024MiB ceph journal 100 35841MiB 36865MiB 1024MiB ceph journal 101 36865MiB 37889MiB 1024MiB ceph journal 1 37889MiB 47684MiB 9795MiB 102 102 47684MiB 48708MiB 1024MiB ceph journal 103 48708MiB 49732MiB 1024MiB ceph journal 104 49732MiB 50756MiB 1024MiB ceph journal 64 334982MiB 457863MiB 122881MiB xfs ceph data # blkid -p /dev/nvme0n1* |sed -r 's,UUID="[^"]+",UUID="redacted",g' /dev/nvme0n1: PTTYPE="gpt" /dev/nvme0n1p100: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="ceph journal" PART_ENTRY_UUID="redacted" PART_ENTRY_TYPE="45b0969e-9b03-4f30-b4c6-35865ceff106" PART_ENTRY_NUMBER="100" PART_ENTRY_OFFSET="73402368" PART_ENTRY_SIZE="2097152" PART_ENTRY_DISK="259:0" /dev/nvme0n1p101: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p102: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p103: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p104: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p64: UUID="redacted" TYPE="xfs" USAGE="filesystem" /dev/nvme0n1p66: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p67: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p68: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p69: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p70: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p71: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p72: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p73: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p74: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p76: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p77: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p78: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p79: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p80: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p81: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p83: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p84: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p85: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p86: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p87: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p88: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p89: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p90: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p91: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p92: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p93: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p94: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p95: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p96: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p97: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p98: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" /dev/nvme0n1p99: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto"
History
#1 Updated by Alfredo Deza almost 6 years ago
- Status changed from New to Closed
We rely on blkid to recognize these disks, so if blkid is unable to provide the information needed then nothing will work.
As noted on the other issues you opened, ceph-volume does not support Trusty
Same as issue #23497, we somehow missed the window to add a warning on this (for Trusty users)