Project

General

Profile

Bug #46278

mds: Subvolume snapshot directory does not save attribute "ceph.quota.max_bytes" of snapshot source directory tree

Added by Shyamsundar Ranganathan 4 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
octopus,nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
snapshots
Pull request ID:
Crash signature:

Description

NOTE: This is applicable even without using subvolume snapshots

A snapshot of a directory with quota set on it, via the xattr "ceph.quota.max_bytes", does not contain the same xattr preserved on the snapshot directory.

This prevents a clone of a subvolume snapshot, to read and set the same quota on the cloned subvolume.

Tests/experiments done:
1. Quota enforcement, based on a simple test, with hierarchical quotas, enforces the first minimum quota that is breached (as expected)
- NOTE: (q:NUM) is the quota on the directory is NUM bytes
- if quota looked like so: /dir(q:1024)/subdir(q:2048) a dd of > 1024 bytes on /dir/subdir/dd.bin fails
- if quota looked like so: /dir(q:2048)/subdir(q:1024) a dd of > 1024 bytes on /dir/subdir/dd.bin fails

2. Snapshot COW blocks do not seem to be accounted for in the space consumption of the snapshot source directory
- if quota looked like so: /dir(q:1024), the following operations never fail and snapshots hold onto older content, which would total to greater than the directory quota
- (repeat for i:1..10)
- dd if=/dev/urandom of=/mnt/dev/cephfs/dir/dd.bin count=1 bs=1000
- mkdir /mnt/dev/cephfs/dir/.snap/snap[i]
- rm /mnt/dev/cephfs/dir/dd.bin
- I am unsure if this is the right behavior

3. Also tested if a subdirectory of a directory being snapshotted, that has an explicit quota set on it is preserved in the snapshot, it is not
- Test goes something like so,
- getfattr -n ceph.quota.max_bytes /mnt/dev/cephfs/dir/sdir # file: mnt/dev/cephfs/dir/sdir
ceph.quota.max_bytes="1024"
- mkdir /mnt/dev/cephfs/dir/.snap/snap5
- getfattr -n ceph.quota.max_bytes /mnt/dev/cephfs/dir/.snap/snap5/sdir
/mnt/dev/cephfs/dir/.snap/snap5/sdir: ceph.quota.max_bytes: No such attribute

Based on 1 and 2 above, we could safely preserve the quota xattr on the snapshot directory, as the parent of the snapshot directory would enforce any quota breach.

I am not sure about (2) though, as in, if that is the designed/expected behavior.


Related issues

Copied to fs - Backport #46795: octopus: mds: Subvolume snapshot directory does not save attribute "ceph.quota.max_bytes" of snapshot source directory tree Resolved
Copied to fs - Backport #46796: nautilus: mds: Subvolume snapshot directory does not save attribute "ceph.quota.max_bytes" of snapshot source directory tree Resolved

History

#1 Updated by Patrick Donnelly 4 months ago

  • Status changed from New to Triaged
  • Assignee set to Zheng Yan
  • Target version set to v16.0.0
  • Backport set to octopus,nautilus
  • Labels (FS) snapshots added

#2 Updated by Zheng Yan 4 months ago

try following patch

diff --git a/src/client/Client.cc b/src/client/Client.cc
index 847cfb7b65..d1944499c6 100644
--- a/src/client/Client.cc
+++ b/src/client/Client.cc
@@ -11854,7 +11854,8 @@ int Client::ll_removexattr(Inode *in, const char *name, const UserPerm& perms)
 bool Client::_vxattrcb_quota_exists(Inode *in)
 {
   return in->quota.is_enable() &&
-        in->snaprealm && in->snaprealm->ino == in->ino;
+        (in->snapid != CEPH_NOSNAP ||
+         (in->snaprealm && in->snaprealm->ino == in->ino));
 }
 size_t Client::_vxattrcb_quota(Inode *in, char *val, size_t size)
 {

#3 Updated by Shyamsundar Ranganathan 4 months ago

Zheng Yan wrote:

try following patch
[...]

Thanks, tested with ceph-fuse and setting the quota as before and inspecting the snapshots. The snapshot directories now return the required xattr information as required with the patch. Thanks!

(jFYI) We would need the fix for the kernel clients as well as ceph-csi prefers to use the kernel client.

#4 Updated by Shyamsundar Ranganathan 3 months ago

The fix may not be required from the kernel clients in addition to libcephfs, as volume manager is the entity that would read and apply this to a clone, which uses libcephfs.

#5 Updated by Patrick Donnelly 3 months ago

  • Status changed from Triaged to Fix Under Review
  • Assignee changed from Zheng Yan to Shyamsundar Ranganathan
  • Pull request ID set to 36249

#6 Updated by Patrick Donnelly 3 months ago

  • Status changed from Fix Under Review to Pending Backport

#7 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #46795: octopus: mds: Subvolume snapshot directory does not save attribute "ceph.quota.max_bytes" of snapshot source directory tree added

#8 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #46796: nautilus: mds: Subvolume snapshot directory does not save attribute "ceph.quota.max_bytes" of snapshot source directory tree added

#9 Updated by Nathan Cutler about 2 months ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Also available in: Atom PDF