Ceph » Linux kernel client

ISSUE

If you have used mount with the --bind option to overlay one cephfs location on another, then creating any file in the parent directory that holds the mountpoint with lead to a "Device or resource busy" error on any operation on that mountpoint, including any attempt to unmount it. Recoverable only by rebooting.

Syslog will report something like this whenever you try to do anything on that previously-accessible mountpoint:
ceph: splice_dentry error -16 ffff88003d2a2d98 inode ffff88003d29f988 ino 100000b0022.fffffffffffffffe

SCOPE

This behaviour exhibits itself in the cephfs kernel client, but not with the FUSE client.

HOW TO REPRODUCE

The following shell sequence assumes two nodes, having both mounted the same cephfs file-system on /data, via the kernel driver. It assumes you are root:

[ON SERVER 1]

mkdir -p /data/source
 echo ok > /data/source/file_in_source

mkdir -p /data/target/mountpoint/

mount --bind /data/source /data/target/mountpoint

cat /data/target/mountpoint/file_in_source

1. you should have received the output:
2. ok

[ON SERVER 2, WITH THE SAME /data CEPHFS:]

mount --bind /data/source /data/target/mountpoint
 echo ok > /data/target/file_in_parent

[BACK ON SERVER 1]
cat /data/target/mountpoint/file_in_source

1. This time, you will receive an error:
2. cat: /data/target/mountpoint/file_in_source: Device or resource busy

1. /data/target/mountpoint is now unusable and, indeed, unmountable