Project

General

Profile

Bug #63391

OSDs fail to be created on PVs or LVs in v17.2.7 due to failure in ceph-volume raw list

Added by Travis Nielsen 4 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Urgent
Target version:
% Done:

0%

Source:
Community (user)
Tags:
backport_processed
Backport:
reef,quincy
Regression:
Yes
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

ceph-volume is failing to list OSDs or devices that are on PVs or LVs in Rook clusters with v17.2.7.
See Rook issue https://github.com/rook/rook/issues/13136

This prevents new OSDs of this type from being created, and also prevents these types of upgraded OSDs from starting after the upgrade.

The raw list results in a list index out of range:

[2023-10-31 17:29:08,090][ceph_volume.devices.raw.list][DEBUG ] inspecting devices: ['/mnt/set1-data-0bv5xb']
[2023-10-31 17:29:08,090][ceph_volume][ERROR ] exception caught by decorator
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 59, in newfunc
return f(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/main.py", line 153, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/main.py", line 32, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python3.6/site-packages/ceph_volume/terminal.py", line 194, in dispatch
instance.main()
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/list.py", line 166, in main
self.list(args)
File "/usr/lib/python3.6/site-packages/ceph_volume/decorators.py", line 16, in is_root
return func(*a, **kw)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/list.py", line 122, in list
report = self.generate(args.device)
File "/usr/lib/python3.6/site-packages/ceph_volume/devices/raw/list.py", line 91, in generate
info_device = [info for info in info_devices if info['NAME'] == dev][0]
IndexError: list index out of range


Related issues

Copied to ceph-volume - Backport #63490: quincy: OSDs fail to be created on PVs or LVs in v17.2.7 due to failure in ceph-volume raw list Resolved
Copied to ceph-volume - Backport #63491: reef: OSDs fail to be created on PVs or LVs in v17.2.7 due to failure in ceph-volume raw list Resolved

History

#1 Updated by Guillaume Abrioux 4 months ago

  • Status changed from New to In Progress
  • Assignee set to Guillaume Abrioux
  • Priority changed from Normal to Urgent

#2 Updated by Guillaume Abrioux 4 months ago

  • Backport set to reef,quincy
  • Pull request ID set to 54392

#3 Updated by Guillaume Abrioux 4 months ago

  • Status changed from In Progress to Fix Under Review

#4 Updated by Matthew Booth 4 months ago

As noted in the GH issue and PR, I don't think the linked PR fixes the reported crash. I have pushed https://github.com/ceph/ceph/pull/54400 as an alternative, which I believe does fix the reported crash.

see https://github.com/ceph/ceph/pull/54392#discussion_r1386867434

#5 Updated by Guillaume Abrioux 4 months ago

  • Status changed from Fix Under Review to Pending Backport

#6 Updated by Guillaume Abrioux 4 months ago

  • Copied to Backport #63490: quincy: OSDs fail to be created on PVs or LVs in v17.2.7 due to failure in ceph-volume raw list added

#7 Updated by Guillaume Abrioux 4 months ago

  • Copied to Backport #63491: reef: OSDs fail to be created on PVs or LVs in v17.2.7 due to failure in ceph-volume raw list added

#8 Updated by Guillaume Abrioux 4 months ago

  • Tags set to backport_processed

#9 Updated by Guillaume Abrioux 3 months ago

  • Status changed from Pending Backport to Resolved

#10 Updated by Christian Rohmann 3 months ago

May I ask if there will be a bugfix to the 17.2.7 point release with this fix?

Also available in: Atom PDF