Project

General

Profile

Actions

Bug #44149

closed

zapping filestore journals broken

Added by Jan Fajerski about 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus, mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

--> Zapping: /dev/nvme0n1
--> Zapping lvm member /dev/nvme0n1. lv_path is /dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672
Running command: /bin/dd if=/dev/zero of=/dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672 bs=1M count=10 conv=fsync
--> More than 1 LV left in VG, will proceed to destroy LV only
--> Removing LV because --destroy was given: /dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672
Running command: /sbin/lvremove -v -f /dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672
 stdout: Logical volume "osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672" successfully removed
 stderr: WARNING: Device mismatch detected for ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672 which is accessing /dev/loop0 instead of /dev/nvme0n1.
 stderr: WARNING: Device mismatch detected for ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb which is accessing /dev/loop0 instead of /dev/nvme0n1.
 stderr: Removing ceph--journals--0136ed34--bdaa--4722--953c--b0beee5686e3-osd--journal--c7677f43--15dd--4d1c--ace2--fb0f93e27672 (252:2)
 stderr: Archiving volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" metadata (seqno 45).
 stderr: Releasing logical volume "osd-journal-c7677f43-15dd-4d1c-ace2-fb0f93e27672" 
 stderr: Creating volume group backup "/etc/lvm/backup/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" (seqno 46).
--> Zapping lvm member /dev/nvme0n1. lv_path is /dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb
Running command: /bin/dd if=/dev/zero of=/dev/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb bs=1M count=10 conv=fsync
--> Only 1 LV left in VG, will proceed to destroy volume group ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
Running command: /sbin/vgremove -v -f ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
 stderr: WARNING: Device mismatch detected for ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3/osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb which is accessing /dev/loop0 instead of /dev/nvme0n1.
 stderr: Removing ceph--journals--0136ed34--bdaa--4722--953c--b0beee5686e3-osd--journal--c8e85fca--a057--43a3--a246--0248cf8392eb (252:3)
 stderr: Archiving volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" metadata (seqno 46).
    Releasing logical volume "osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb" 
 stderr: Creating volume group backup "/etc/lvm/backup/ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" (seqno 47).
 stdout: Logical volume "osd-journal-c8e85fca-a057-43a3-a246-0248cf8392eb" successfully removed
 stderr: Removing physical volume "/dev/nvme0n1" from volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" 
 stderr: Removing physical volume "/dev/nvme1n1" from volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" 
 stdout: Volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" successfully removed
--> Found empty VG ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3, removing
Running command: /sbin/vgremove -v -f ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
 stderr: VG name on command line not found in list of VGs: ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
 stderr: Volume group "ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3" not found
  Cannot process volume group ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
--> Unable to remove vg ceph-journals-0136ed34-bdaa-4722-953c-b0beee5686e3
Traceback (most recent call last):
  File "/sbin/ceph-volume", line 9, in <module>
    load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()
  File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 39, in __init__
    self.main(self.argv)
  File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59, in newfunc
    return f(*a, **kw)
  File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 150, in main
    terminal.dispatch(self.mapper, subcommand_args)
  File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py", line 42, in main
    terminal.dispatch(self.mapper, self.argv)
  File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 194, in dispatch
    instance.main()
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/zap.py", line 390, in main
    self.zap()
  File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16, in is_root
    return func(*a, **kw)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/zap.py", line 267, in zap
    self.zap_lvm_member(device)
  File "/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/zap.py", line 231, in zap_lvm_member
    api.remove_vg(lv.vg_name)
  File "/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py", line 843, in remove_vg
    fail_msg=fail_msg,
  File "/usr/lib/python2.7/site-packages/ceph_volume/process.py", line 153, in run
    raise RuntimeError(msg)
RuntimeError: command returned non-zero exit status: 5

Related issues 2 (0 open2 closed)

Copied to ceph-volume - Backport #44153: nautilus: zapping filestore journals brokenResolvedJan FajerskiActions
Copied to ceph-volume - Backport #44154: mimic: zapping filestore journals brokenResolvedJan FajerskiActions
Actions #1

Updated by Jan Fajerski about 4 years ago

  • Status changed from New to Pending Backport
  • Backport set to nautilus, mimic
  • Pull request ID set to 33332
Actions #2

Updated by Jan Fajerski about 4 years ago

  • Copied to Backport #44153: nautilus: zapping filestore journals broken added
Actions #3

Updated by Jan Fajerski about 4 years ago

  • Copied to Backport #44154: mimic: zapping filestore journals broken added
Actions #4

Updated by Nathan Cutler about 4 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF