Project

General

Profile

Actions
Copy link

Bug #13155

closed

ceph-disk: permission denied if udev races with mkfs

Added by Loïc Dachary over 8 years ago. Updated over 8 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
Loïc Dachary
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

If the udev add event for the data partition is received before the udev add event for the journal partition, activate will fail because it does not (yet) have permission on the journal device

Sep 18 10:03:12 target230089 kernel: XFS (vdb1): Mounting V4 Filesystem
Sep 18 10:03:13 target230089 kernel: XFS (vdb1): Ending clean mount
Sep 18 10:03:14 target230089 ceph-disk: got monmap epoch 1
Sep 18 10:03:14 target230089 ceph-disk: 2015-09-18 10:03:14.638902 7fa06f5b6900 -1 filestore(/var/lib/ceph/tmp/mnt.XRpboz) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.XRpboz/journal: (13) Permissio\
n denied
Sep 18 10:03:14 target230089 ceph-disk: 2015-09-18 10:03:14.638960 7fa06f5b6900 -1 OSD::mkfs: ObjectStore::mkfs failed with error -13
Sep 18 10:03:14 target230089 ceph-disk: 2015-09-18 10:03:14.639095 7fa06f5b6900 -1  ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.XRpboz: (13) Permission denied
Sep 18 10:03:14 target230089 ceph-disk: ERROR:ceph-disk:Failed to activate
Sep 18 10:03:14 target230089 python: detected unhandled Python exception in '/usr/sbin/ceph-disk'

Related issues 1 (0 open — 1 closed)

Is duplicate of Ceph - Bug #12787: ceph-disk does not activate with systemd on some CentOS 7ResolvedLoïc Dachary08/26/2015

Actions
Actions
Copy link
 #1

Updated by Loïc Dachary over 8 years ago

  • Subject changed from ceph-disk: permission denied if data activates before journal to ceph-disk: permission denied if udev races with mkfs
  • udev event add on /dev/vdb1 data partition received
  • udev chown root /dev/vdbd1
  • ceph udev rule chown /dev/vdb1 to ceph
  • udev event add on /dev/vdb2 journal partition
  • udev chown root /dev/vdb2
  • ceph-disk activate tries to mkfs the data partition and fail with permission denied
  • ceph udev rule chown /dev/vdb2 to ceph
Actions
Copy link
 #2

Updated by Loïc Dachary over 8 years ago

There is no sure way to protect against this race. Maybe it should just retry a few times. A workaround is to add a udevadm settle before the mkfs: it makes it less likely that the race occur by waiting for the udev list of events to drain. The race can still happen though.

Actions
Copy link
 #3

Updated by Loïc Dachary over 8 years ago

  • Status changed from In Progress to Duplicate
Actions
Copy link

Also available in: Atom PDF