Bug #63939
openfscrypt on CephFS forgets that directory is encrypted after unmount
0%
Description
Summary: After unmounting and remounting CephFS, encrypted directories are not recognized as encrypted.
Severity: This could lead to data loss
Description: This issue is present in Ubuntu 23.10 with Kernel 6.6.7 from mainline and Ceph 18.2.0 - (6.6+ required for fscrypt support on CephFS). fscrypt is installed via apt. The Ceph cluster is v17.2.7 right now.
How to reproduce:
Create CephFS named mycephfs on the cluster
$ sudo mount -t ceph user@.mycephfs=/ /mnt
$ cd /mnt
$ /fscrypt setup
$ mkdir cryptdir
$ fscrypt encrypt cryptdir
The result here is correct, I get "cryptdir is now encrypted, unlocked and ready to use."
At this point I can fscrypt lock and unlock the directory
When the directory is locked, the files inside are encrypted, with encrypted filenames. When it is unlocked. the files are decrypted.
Now unmount and remount:
$ sudo umount /mnt
$ sudo mount -t ceph user@.mycephfs=/ /mnt
$ cd /mnt
ls shows cryptdir.
$ fscrypt unlock cryptdir
[ERROR] fscrypt unlock: file or directory cryptdir is not encrypted
The files and directories inside the directory are encrypted, but fscrypt does not recognize that they are.
I tried locking the directory before umount and leaving it unlocked before umount. Neither works.
I tried to find a solution in the bug tracker and online, but came up empty.
I also reported this on the fscrypt project, but they said it works with other FS and advised to open a bug here.
Updated by Christian Huebner 4 months ago
Interestingly the issue with 'is not encrypted' does not seem to be limited to umount and remount:
root@ubuntu:/mnt# mkdir test4
root@ubuntu:/mnt# fscrypt encrypt test4
Should we create a new protector? [y/N]
Enter custom passphrase for protector "thecrypta":
"test4" is now encrypted, unlocked, and ready for use.
root@ubuntu:/mnt# vi test4/testfile
root@ubuntu:/mnt# ls test4
testfile
root@ubuntu:/mnt# fscrypt lock test4
"test4" is now locked.
root@ubuntu:/mnt# ls test4
I1eTGrt5j2K08BlIdpUs++w4tFnJtes5JuY7n1,gja4
root@ubuntu:/mnt# fscrypt unlock test4
[ERROR] fscrypt unlock: file or directory "test4" is not encrypted
root@ubuntu:/mnt#
For some reason the metadata read does not seem to work right. the error 'is not encrypted' is in fscrypt/metadata/policy.go in the function GetPolicy.
It is triggered whem getPolicyIoctl gets the error unix.ENODATA
So I infer the policy is either not written or not read properly on CephFS
It works correctly on ext4fs.
Updated by Christian Huebner 4 months ago
Update: the problem appears to be writing into the encrypted (unlocked) CephFS directory.
Locking and unlocking of an encrypted empty directory works
Locking of an encrypted directory with a file written in it does work (or at least does not produce an error)
Unlocking of an encrypted directory with a file written in it does not.
I get the error message "[ERROR] fscrypt unlock: file or directory "crypt" is not encrypted"
root@ubuntu:~# mount -t ceph admin@.crhfs=/ /mnt root@ubuntu:~# cd /mnt root@ubuntu:/mnt# ls root@ubuntu:/mnt# mkdir crypt root@ubuntu:/mnt# fscrypt encrypt crypt Should we create a new protector? [y/N] n Enter custom passphrase for protector "thecrypta": "crypt" is now encrypted, unlocked, and ready for use. root@ubuntu:/mnt# fscrypt lock crypt "crypt" is now locked. root@ubuntu:/mnt# fscrypt unlock crypt Enter custom passphrase for protector "thecrypta": "crypt" is now unlocked and ready for use. root@ubuntu:/mnt# touch crypt/sdfdsfsdf root@ubuntu:/mnt# fscrypt lock crypt "crypt" is now locked. root@ubuntu:/mnt# fscrypt unlock crypt [ERROR] fscrypt unlock: file or directory "crypt" is not encrypted
Updated by Christian Huebner 4 months ago
I upgraded the cluster to 18.2.1 (Reef). The issue stays the same. Client is still the same at 18.2.0