Ceph » CephFS

When a snapshot is taken of the subvolume, the .snap directory shows the snapshot when viewed from the NFS mount and cephfs mount.

However, when the snapshot is deleted the cephfs mount correctly shows an empty .snap directory - but the NFS mount still shows the snapshot entry and the directories and files listed. You can even copy a file from the snapshot to the main filesystem!

e.g

root@nfs-client-02 ~]# ls /mnt/cephfs/.snap/_snapit_1099511628282/
172.18.199.3.workload.0.0  smalldfile  smallfile  specstorage  testfile
[root@nfs-client-02 ~]# ls /mnt/nfs/.snap/_snapit_1099511628282/
172.18.199.3.workload.0.0  smalldfile  smallfile  specstorage  testfile
[root@nfs-client-02 ~]# # snapshot has been deleted
[root@nfs-client-02 ~]# ls /mnt/cephfs/.snap
[root@nfs-client-02 ~]# ls /mnt/nfs/.snap/_snapit_1099511628282/
172.18.199.3.workload.0.0  smalldfile  smallfile  specstorage  testfile                                   <--- still shows a snapshot!
[root@nfs-client-02 ~]# cp /mnt/nfs/.snap/_snapit_1099511628282/testfile ~/.                              <- copying a file from a snapshot that has been deleted worked!
[root@nfs-client-02 ~]# cat testfile
hello from the test file

df on the client shows the following

172.16.36.61:/volumes/_nogroup/nfs-client-02/c53381bb-c6e8-4ac2-89bf-c07b042ab56b  100G  5.0G   95G   5% /mnt/cephfs
172.18.200.10:/nfs-client-02                                                       100G  5.0G   95G   5% /mnt/nfs
172.18.200.10:/nfs-client-02/.snap                                                 100G  5.0G   95G   5% /mnt/nfs/.snap
172.18.200.10:/nfs-client-02/.snap/_snapit_1099511628282                           100G  5.0G   95G   5% /mnt/nfs/.snap/_snapit_1099511628282

There appears to be a sync issue when there are multiple snapshots too.

When I create several snapshots on cephfs I see this

[root@nfs-client-02 ~]# ls /mnt/cephfs/.snap
_snapit-again_1099511628282  _snapit-again-2_1099511628282

But when I view the nfs mount which is the same subvolume

[root@nfs-client-02 ~]# ls /mnt/nfs/.snap                                <-- _snapit-again-2 is missing!
_snapit-again_1099511628282

I'm not sure where the issue lies - Ganesha or cephfs, so raising against cephfs