Bug #16351
closedjewel : 60-ceph-partuuid-workaround-rules still needed by debian jessie (udev 215-17)
Added by alexandre derumier almost 8 years ago. Updated over 7 years ago.
0%
Description
Hi,
since this commit
https://github.com/ceph/ceph/commit/9f77244b8e0782921663e52005b725cca58a8753
osd disk are not mounted anymore on debian jessie ((udev 215-17) at boot
I have looked at udev log, at the 95-ceph-osd.rules is never triggered
if I add the old 60-ceph-partuuid-workaround-rules, it's working fine
Files
| ceph-disk.strace.sde1.4787.gz (8.77 KB) ceph-disk.strace.sde1.4787.gz | /bin/systemctl --no-block restart ceph-disk@dev/sde1.service | Denis Pudov, 08/05/2016 03:12 PM | |
| ceph-disk.strace.sde1.4545.gz (142 KB) ceph-disk.strace.sde1.4545.gz | /usr/sbin/ceph-disk --log-stdout -v trigger /dev/sde1 | Denis Pudov, 08/05/2016 03:13 PM |
Updated by Loïc Dachary almost 8 years ago
- Assignee set to Loïc Dachary
- Priority changed from Normal to Urgent
Updated by Loïc Dachary almost 8 years ago
- Description updated (diff)
- Status changed from New to In Progress
Updated by Daniel Swarbrick almost 8 years ago
I ran into this bug also, and can confirm that re-adding the old 60-ceph-partuuid-workaround-rules fixes the problem.
Updated by Loïc Dachary almost 8 years ago
On Jessie 8.5, the rules work as expected when removing/adding a disk
udevadm monitor ... KERNEL[399.124705] add /devices/pci0000:00/0000:00:07.0 (pci) KERNEL[399.128475] add /devices/pci0000:00/0000:00:07.0/virtio3 (virtio) UDEV [399.131617] add /devices/pci0000:00/0000:00:07.0 (pci) KERNEL[399.132247] add /devices/virtual/bdi/254:16 (bdi) UDEV [399.132499] add /devices/pci0000:00/0000:00:07.0/virtio3 (virtio) UDEV [399.133016] add /devices/virtual/bdi/254:16 (bdi) KERNEL[399.142008] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb (block) KERNEL[399.142033] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb/vdb1 (block) KERNEL[399.142045] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb/vdb2 (block) UDEV [399.526162] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb (block) UDEV [403.304562] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb/vdb2 (block) UDEV [403.395816] add /devices/pci0000:00/0000:00:07.0/virtio3/block/vdb/vdb1 (block)
and
ceph 938 6.4 0.4 831352 32052 ? Ssl 13:50 0:00 /usr/bin/ceph-osd -f --cluster ceph --id 0 --setuser ceph --setgroup ceph
But it does not work when rebooting. Trying to figure out why. The fact that adding 60-ceph-partuuid-workaround-rules fixes the issue is helpful.
Updated by Loïc Dachary almost 8 years ago
For the record, it works on CentOS 7.2 but a user reported it failed in the same way on CentOS 7 (maybe an older version) ?
Updated by Loïc Dachary almost 8 years ago
Maybe the problem is that ADD udev events are not fired at boot time
Updated by Loïc Dachary almost 8 years ago
The CentOS failure happens when the root file system is on LVM. When the root file system is on a regular partition, the OSD shows up as expected.
Updated by alexandre derumier almost 8 years ago
Loic Dachary wrote:
The CentOS failure happens when the root file system is on LVM. When the root file system is on a regular partition, the OSD shows up as expected.
on my debian jessie setup, it is without lvm
/etc/fstab
UUID=a9309e01-0f80-475b-95e1-33a9ddb4bdff / xfs defaults 0 1
Updated by Loïc Dachary almost 8 years ago
- Status changed from In Progress to Fix Under Review
Updated by Loïc Dachary almost 8 years ago
- Status changed from Fix Under Review to Pending Backport
Updated by Loïc Dachary almost 8 years ago
- Copied to Backport #16577: jewel : 60-ceph-partuuid-workaround-rules still needed by debian jessie (udev 215-17) added
Updated by Denis Pudov almost 8 years ago
- File ceph-disk.strace.sde1.4787.gz ceph-disk.strace.sde1.4787.gz added
- File ceph-disk.strace.sde1.4545.gz ceph-disk.strace.sde1.4545.gz added
alexandre derumier wrote:
Hi,
since this commit
https://github.com/ceph/ceph/commit/9f77244b8e0782921663e52005b725cca58a8753osd disk are not mounted anymore on debian jessie ((udev 215-17) at boot
I have looked at udev log, at the 95-ceph-osd.rules is never triggered
On my Jessie (8.5) with 4.6.0 backports kernel its triggered.
I have added strace to ceph-disk at the 95-ceph-osd.rules:
RUN+="/usr/bin/strace -tt -s 1024 -o /var/tmp/ceph-disk.strace.$name -ff /usr/sbin/ceph-disk --log-stdout -v trigger /dev/$name"
and got four strace output per device.
At the stage of calling trigger (/bin/systemctl --no-block restart ceph-disk@dev/sde1.service) get an error from systemd (full strace at attach):
org.freedesktop.DBus.Error.InvalidArgs.*sysinit.target is queued, ignoring restart request for unit ceph-disk@dev-sde1.service
if I add the old 60-ceph-partuuid-workaround-rules, it's working fine
For me it's not fine... systemd tries many times to run ceph-osd@.service and eventually get:
авг 05 18:00:18 storage-ceph-01y systemd[1]: ceph-osd@58.service: main process exited, code=exited, status=1/FAILURE авг 05 18:00:18 storage-ceph-01y systemd[1]: Unit ceph-osd@58.service entered failed state. авг 05 18:00:18 storage-ceph-01y systemd[1]: ceph-osd@58.service start request repeated too quickly, refusing to start. авг 05 18:00:18 storage-ceph-01y systemd[1]: Failed to start Ceph object storage daemon.
Updated by alexandre derumier almost 8 years ago
do you have tried the fix from loic ?
https://github.com/ceph/ceph/pull/9885/commits/35004a628b2969d8b2f1c02155bb235165a1d809
Updated by Denis Pudov almost 8 years ago
Yes, i have added 60-ceph-partuuid-workaround-rules to /lib/udev/rules.d/ and got mounts but have problems with start ceph-osd@.service.
Updated by alexandre derumier almost 8 years ago
The fix is
60-ceph-by-parttypeuuid.rules
https://github.com/dachary/ceph/blob/35004a628b2969d8b2f1c02155bb235165a1d809/udev/60-ceph-by-parttypeuuid.rules
not old
60-ceph-partuuid-workaround-rules
Updated by Denis Pudov almost 8 years ago
Alexandre, thank you!
60-ceph-by-parttypeuuid.rules works!
Updated by Loïc Dachary over 7 years ago
Additional pull request, for the record: https://github.com/ceph/ceph/pull/10115
Updated by Loïc Dachary over 7 years ago
- Status changed from Pending Backport to Resolved
Updated by joke lee over 7 years ago
it's still not mount osd after reboot when the osd disk is raid0