Ceph » Linux kernel client

This can be reproduced by the qa teuthology always, locally never.

For the listxattr it will return one extended attribute user.snrub="fish2\012" as expected and the size is 16, but the MDS side seems still having at least 2 instead, which the total size is 60:

2023-02-16T02:04:07.646+0000 7f7b2b949700 10 mds.0.server xattr kv pairs size too big. cur_xattrs_size 60, inc 65520

I added one debug log in mds side to print what exactly the existing extented distributes are there in MDS side:

http://qa-proxy.ceph.com/teuthology/xiubli-2023-02-16_12:36:18-fs:fscrypt-wip-fscrypt-20230216-1802-distro-default-smithi/7176159/teuthology.log

Xiubo Li wrote:

This can be reproduced by the qa teuthology always, locally never.

For the listxattr it will return one extended attribute user.snrub="fish2\012" as expected and the size is 16, but the MDS side seems still having at least 2 instead, which the total size is 60:

[...]

I added one debug log in mds side to print what exactly the existing extented distributes are there in MDS side:

http://qa-proxy.ceph.com/teuthology/xiubli-2023-02-16_12:36:18-fs:fscrypt-wip-fscrypt-20230216-1802-distro-default-smithi/7176159/teuthology.log

There was one extra security.selinux added in MDS side:

2023-02-16T13:17:37.170+0000 7fbb1580e700 20 mds.0.server handler=default xattr handler
2023-02-16T13:17:37.170+0000 7fbb1580e700 10 mds.0.server xattr kv pairs size too big. cur_xattrs_size 60, inc 65520
2023-02-16T13:17:37.170+0000 7fbb1580e700 10 mds.0.server  xattr name security.selinux value buffer::ptr(0~28 0x55ce669f9f00 in raw 0x55ce669f9f00 len 28 nref 210)
2023-02-16T13:17:37.170+0000 7fbb1580e700 10 mds.0.server  xattr name user.snrub value buffer::ptr(0~6 0x55ce66de26c0 in raw 0x55ce66de26c0 len 6 nref 211)
2023-02-16T13:17:37.170+0000 7fbb1580e700  7 mds.0.server reply_client_request -28 ((28) No space left on device) client_request(client.5196:13697 setxattr #0x10000009869 user.long_attr 2023-02-16T13:17:37.170489+0000 caller_uid=0, caller_gid=0{0,}) v6
2023-02-16T13:17:37.170+0000 7fbb1580e700 10 mds.0.server apply_allocated_inos 0x0 / [] / 0x0

If the CONFIG_CEPH_FS_SECURITY_LABEL is enabled the kernel ceph itself will set the security.selinux extended attribute to MDS. And it will also eat some space of the total size.

diff --git a/tests/generic/020 b/tests/generic/020
index be5cecad..594535b5 100755
--- a/tests/generic/020
+++ b/tests/generic/020
@@ -150,9 +150,11 @@ _attr_get_maxval_size()
                # it imposes a maximum size for the full set of xattrs
                # names+values, which by default is 64K.  Compute the maximum
                # taking into account the already existing attributes
-               max_attrval_size=$(getfattr --dump -e hex $filename 2>/dev/null | \
+               size=$(getfattr --dump -e hex $filename 2>/dev/null | \
                        awk -F "=0x" '/^user/ {len += length($1) + length($2) / 2} END {print len}')
-               max_attrval_size=$((65536 - $max_attrval_size - $max_attrval_namelen))
+               selinux_size=$(getfattr -n 'security.selinux' --dump -e hex $filename 2>/dev/null | \
+                       awk -F "=0x" '/^security/ {len += length($1) + length($2) / 2} END {print len}')
+               max_attrval_size=$((65536 - $size - $selinux_size - $max_attrval_namelen))
                ;;
        *)
                # Assume max ~1 block of attrs

With the above patch the test passed:

# ./check generic/020
FSTYP         -- ceph
PLATFORM      -- Linux/x86_64 lxbceph1 6.2.0-rc6+ #208 SMP PREEMPT_DYNAMIC Mon Feb 13 09:30:30 CST 2023
MKFS_OPTIONS  -- 10.72.47.117:40735:/testB
MOUNT_OPTIONS -- -o test_dummy_encryption,name=admin,nowsync,copyfrom,rasize=4096,context=system_u:object_r:root_t:s0 -o context=system_u:object_r:root_t:s0 10.72.47.117:40735:/testB /mnt/kcephfs.B

generic/020 135s ... 135s
Ran: generic/020
Passed all 1 tests

Sent the patch out to xfstests-dev to fix it.