Project

General

Profile

Bug #57641

Ceph FS fscrypt clones missing fscrypt metadata

Added by marcel lauhoff 2 months ago. Updated about 2 months ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
-
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Tags:
Backport:
pacific,quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, libcephfs, mgr/volumes
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Summary

When cloning a Ceph FS volume containing fscrypt-enabled subtrees,
the clone misses fscrypt metadata. The fscrypted data becomes inaccessible.

Steps to reproduce

Ceph commands:

ceph fs subvolume create subvol a
ceph fs subvolume snapshot create a subvol snap
ceph fs subvolume snapshot clone a subvol snap subvolclone

On a host running kernel git://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux.git branch ceph-fscrypt commitish 46d15ce3eb30

# subvolume
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# mkdir encrypted
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# fscryptctl add_key encrypted
asdfasdfasdfasdfasdfasdfasdfasdf
35601b42a0fb06b9ed6145adf29021a0
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# fscryptctl set_policy 35601b42a0fb06b9ed6145adf29021a0 encrypted/
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# fscryptctl get_policy encrypted
Encryption policy for encrypted:
        Policy version: 2
        Master key identifier: 35601b42a0fb06b9ed6145adf29021a0
        Contents encryption mode: AES-256-XTS
        Filenames encryption mode: AES-256-CTS
        Flags: PAD_32

# snapshot
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/.snap/snap/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# ls encrypted/
bar
rapido1:/mnt/cephfs/volumes/_nogroup/subvol/.snap/snap/b2ffc43d-c28a-48a8-bb92-e759ae84cd07# fscryptctl get_policy encrypted/
Encryption policy for encrypted/:
        Policy version: 2
        Master key identifier: 35601b42a0fb06b9ed6145adf29021a0
        Contents encryption mode: AES-256-XTS
        Filenames encryption mode: AES-256-CTS
        Flags: PAD_32

# clone
rapido1:/mnt/cephfs/volumes/_nogroup/subvolclone/c0053865-5f33-413f-9116-49a9ef6ee641# ls encrypted/
GZNGZhZa7K9FXj+ShkjhU77lk3CkwjAw93itRdjB,oc
rapido1:/mnt/cephfs/volumes/_nogroup/subvolclone/c0053865-5f33-413f-9116-49a9ef6ee641# fscryptctl get_policy encrypted/
error: getting policy for encrypted/: file or directory not encrypted

# xattrs
rapido1:/mnt/cephfs/volumes/_nogroup# getfattr -n ceph.fscrypt.auth subvol/*/encrypted
# file: subvol/b2ffc43d-c28a-48a8-bb92-e759ae84cd07/encrypted
ceph.fscrypt.auth=0sAQAAACgAAAACAQQDAAAAADVgG0Kg+wa57WFFrfKQIaCzVphLRaRLAru8cny30lOh

rapido1:/mnt/cephfs/volumes/_nogroup# getfattr -n ceph.fscrypt.auth subvolclone/*/encrypted
subvolclone/c0053865-5f33-413f-9116-49a9ef6ee641/encrypted: ceph.fscrypt.auth: No such attribute

Possible fix

Copying the ceph.fscrypt.auth xattr in the mgr/volumes async clone seems to work:
https://github.com/irq0/ceph/commit/001f5e88c3486ea370abc82546f3a8d98725090d

History

#1 Updated by Venky Shankar 2 months ago

Hi Marcel,

Copying the ceph.fscrypt.auth xattr in the mgr/volumes async clone seems to work:
https://github.com/irq0/ceph/commit/001f5e88c3486ea370abc82546f3a8d98725090d

Mind submitting this change as a PR?

#2 Updated by Venky Shankar 2 months ago

  • Target version set to v18.0.0
  • Backport set to pacific,quincy

#3 Updated by Venky Shankar about 2 months ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 48410

Hi Marcel,

I pushed a PR with your commit. We should probably take this opportunity to copy user xattrs during clone.

#4 Updated by marcel lauhoff about 2 months ago

Thanks Venky, I did not submit this right away as I'm unsure the xattr copy is the right way to do this.

I wonder how much of a performance hit the getxattr(source_path, "ceph.fscrypt.auth") is. This probably costs another round trip to the MDS, not?

Also available in: Atom PDF