Project

General

Profile

Bug #23498

ceph-volume: unable to use existing partition as filestore journal

Added by Robin Johnson about 6 years ago. Updated almost 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Followup to issue 23497.

Running Luminous 12.2.4++ on Ubuntu Trusty 14.04.5.

This is trying to convert ceph-disk & even older setups to ceph-volume.

I patched ceph-volume to not pass --yes to lvcreate per issue 23497, but still ended up bailing out because a partition, esp one that is zeroed out, won't have a PARTUUID, or even reliably show up in blkid.

ceph-volume should accept an existing partition as the journal. There's lots of other OSDs with their journals on the NVME device already.

root@cephstore12145:/# ceph-volume lvm create --filestore --journal /dev/nvme0n1p65 --data /dev/sde --dmcrypt 
Running command: ceph-authtool --gen-print-key
Running command: ceph-authtool --gen-print-key
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 8c599b90-a39d-4eb1-9165-87fb73015674
 stderr: lsblk: unknown column: PKNAME,PARTLABEL
 stderr: lsblk: unknown column: PKNAME,PARTLABEL
Running command: vgcreate --force --yes ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef /dev/sde
 stderr: No physical volume label read from /dev/sde
 stdout: Physical volume "/dev/sde" successfully created
 stdout: Volume group "ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef" successfully created
Running command: lvcreate -l 100%FREE -n osd-data-8c599b90-a39d-4eb1-9165-87fb73015674 ceph-7edc3c91-1c72-4e87-8a47-af786c2c78ef
 stdout: Logical volume "osd-data-8c599b90-a39d-4eb1-9165-87fb73015674" created
--> blkid could not detect a PARTUUID for device: /dev/nvme0n1p65
--> Was unable to complete a new OSD, will rollback changes
--> OSD will be fully purged from the cluster, because the ID was generated
Running command: ceph osd purge osd.110 --yes-i-really-mean-it
 stderr: purged osd.110
-->  RuntimeError: unable to use device

# parted /dev/nvme0n1 'unit MiB' print
Model: Unknown (unknown)
Disk /dev/nvme0n1: 457863MiB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start      End        Size       File system  Name          Flags
65      1.00MiB    1025MiB    1024MiB                 ceph journal
66      1025MiB    2049MiB    1024MiB                 ceph journal
67      2049MiB    3073MiB    1024MiB                 ceph journal
68      3073MiB    4097MiB    1024MiB                 ceph journal
69      4097MiB    5121MiB    1024MiB                 ceph journal
70      5121MiB    6145MiB    1024MiB                 ceph journal
71      6145MiB    7169MiB    1024MiB                 ceph journal
72      7169MiB    8193MiB    1024MiB                 ceph journal
73      8193MiB    9217MiB    1024MiB                 ceph journal
74      9217MiB    10241MiB   1024MiB                 ceph journal
76      11265MiB   12289MiB   1024MiB                 ceph journal
77      12289MiB   13313MiB   1024MiB                 ceph journal
78      13313MiB   14337MiB   1024MiB                 ceph journal
79      14337MiB   15361MiB   1024MiB                 ceph journal
80      15361MiB   16385MiB   1024MiB                 ceph journal
81      16385MiB   17409MiB   1024MiB                 ceph journal
83      18433MiB   19457MiB   1024MiB                 ceph journal
84      19457MiB   20481MiB   1024MiB                 ceph journal
85      20481MiB   21505MiB   1024MiB                 ceph journal
86      21505MiB   22529MiB   1024MiB                 ceph journal
87      22529MiB   23553MiB   1024MiB                 ceph journal
88      23553MiB   24577MiB   1024MiB                 ceph journal
89      24577MiB   25601MiB   1024MiB                 ceph journal
90      25601MiB   26625MiB   1024MiB                 ceph journal
91      26625MiB   27649MiB   1024MiB                 ceph journal
92      27649MiB   28673MiB   1024MiB                 ceph journal
93      28673MiB   29697MiB   1024MiB                 ceph journal
94      29697MiB   30721MiB   1024MiB                 ceph journal
95      30721MiB   31745MiB   1024MiB                 ceph journal
96      31745MiB   32769MiB   1024MiB                 ceph journal
97      32769MiB   33793MiB   1024MiB                 ceph journal
98      33793MiB   34817MiB   1024MiB                 ceph journal
99      34817MiB   35841MiB   1024MiB                 ceph journal
100     35841MiB   36865MiB   1024MiB                 ceph journal
101     36865MiB   37889MiB   1024MiB                 ceph journal
 1      37889MiB   47684MiB   9795MiB                 102
102     47684MiB   48708MiB   1024MiB                 ceph journal
103     48708MiB   49732MiB   1024MiB                 ceph journal
104     49732MiB   50756MiB   1024MiB                 ceph journal
64      334982MiB  457863MiB  122881MiB  xfs          ceph data

# blkid -p /dev/nvme0n1* |sed -r 's,UUID="[^"]+",UUID="redacted",g'
/dev/nvme0n1: PTTYPE="gpt" 
/dev/nvme0n1p100: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" PART_ENTRY_SCHEME="gpt" PART_ENTRY_NAME="ceph journal" PART_ENTRY_UUID="redacted" PART_ENTRY_TYPE="45b0969e-9b03-4f30-b4c6-35865ceff106" PART_ENTRY_NUMBER="100" PART_ENTRY_OFFSET="73402368" PART_ENTRY_SIZE="2097152" PART_ENTRY_DISK="259:0" 
/dev/nvme0n1p101: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p102: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p103: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p104: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p64: UUID="redacted" TYPE="xfs" USAGE="filesystem" 
/dev/nvme0n1p66: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p67: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p68: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p69: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p70: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p71: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p72: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p73: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p74: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p76: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p77: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p78: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p79: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p80: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p81: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p83: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p84: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p85: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p86: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p87: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p88: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p89: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p90: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p91: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p92: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p93: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p94: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p95: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p96: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p97: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p98: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 
/dev/nvme0n1p99: UUID="redacted" VERSION="1" TYPE="crypto_LUKS" USAGE="crypto" 

History

#1 Updated by Alfredo Deza almost 6 years ago

  • Status changed from New to Closed

We rely on blkid to recognize these disks, so if blkid is unable to provide the information needed then nothing will work.

As noted on the other issues you opened, ceph-volume does not support Trusty

Same as issue #23497, we somehow missed the window to add a warning on this (for Trusty users)

Also available in: Atom PDF