Bug #12787: ceph-disk does not activate with systemd on some CentOS 7 - Ceph - Ceph

Custom queries

Backports: mimic
Backports: missing release
Backports: nautilus
Bluestore
Bug queue
Bug queue - no subprojects
Bug triage
Ceph backlog
Crash queue
Crash triage
Feature Requests
Feedback
My issues
Need Review
Pending backports
Priority queue
Product Backlog Scrub
Project Triage
Test Failures

Actions

Copy link

Bug #12787

closed

ceph-disk does not activate with systemd on some CentOS 7

Added by Loïc Dachary over 8 years ago. Updated over 8 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Loïc Dachary

Category:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Steps to reproduce on CentOS 7

ceph-disk prepare /dev/vdb

The ceph-osd is not running and mounting /dev/vdb1 shows it has not been activated

ceph-disk activate /dev/vdb1

partprobe /dev/vdb

Successfully activates the osd

Related issues 2 (0 open — 2 closed)

Related to Ceph - Bug #13000: ceph-disk failure to activate with dmcrypt

Resolved

Loïc Dachary

09/09/2015

Actions

Has duplicate Ceph - Bug #13155: ceph-disk: permission denied if udev races with mkfs

Duplicate

Loïc Dachary

09/18/2015

Actions

Issue # Delay: days Cancel

History
Notes
Property changes
Associated revisions

Actions

Copy link

Updated by Loïc Dachary over 8 years ago

Status changed from New to 12

It worked once, I'm not sure why but that suggests a race condition somewhere ?

Actions

Copy link

Updated by Sage Weil over 8 years ago

Status changed from 12 to Resolved

Actions

Copy link

Updated by Loïc Dachary over 8 years ago

Subject changed from ceph-disk does not activate with systemd to ceph-disk does not activate with systemd on some CentOS 7
Status changed from Resolved to 12
Assignee set to Loïc Dachary

The problem can be reproduced reliably with http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1508.qcow2 but not with the vm povisioned on the sepia lab which is based on http://ceph.com/cloudinit/centos-7.0-20140723-cloudimg-amd64.raw

Actions

Copy link

Updated by Loïc Dachary over 8 years ago

Description updated (diff)

Actions

Copy link

Updated by Loïc Dachary over 8 years ago

There is a race : udev add /dev/vdc1 runs ceph-disk trigger and if that happens bfore /dev/vdc1 is ready (i.e. before the mount etc.), it will do nothing and there will not be another udev add event after that, hence no activation.

ceph-disk prepare /dev/vdc output, intermixed with udevadm monitor output

/sbin/sgdisk --new=2:0:100M --change-name=2:'ceph journal' --partition-guid=2:249432d7-9c67-4e19-b446-9384efcd5065 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/vdc
KERNEL[3370.628725] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
UDEV  [3371.074948] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
KERNEL[3371.945837] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
KERNEL[3371.951824] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
UDEV  [3372.593555] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
UDEV  [3377.879706] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
KERNEL[3370.628725] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)

/sbin/partprobe /dev/vdc
KERNEL[3431.743172] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
KERNEL[3432.118319] remove   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
KERNEL[3432.125467] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
UDEV  [3432.959776] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
UDEV  [3432.972899] remove   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
UDEV  [3438.232337] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)

/sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:71c8e6d0-1f70-40d8-8772-b0c45b502c08 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/vdc
KERNEL[3469.583940] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
UDEV  [3470.027603] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
KERNEL[3470.879338] remove   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
UDEV  [3470.890560] remove   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
KERNEL[3470.909782] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
KERNEL[3470.916695] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc1 (block)
KERNEL[3470.917270] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
UDEV  [3471.552154] change   /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc (block)
UDEV  [3481.900785] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc1 (block)
UDEV  [3482.005960] add      /devices/pci0000:00/0000:00:07.0/virtio4/block/vdc/vdc2 (block)
/usr/bin/mount -t xfs -o noatime,inode64 -- /dev/vdc1 /var/lib/ceph/tmp/mnt.1V8zzZ

Actions

Copy link