Ceph » CephFS

Just sleep for 10 seconds the test passed:

[root@lxbceph1 xfstests-dev]# git diff tests/
diff --git a/tests/generic/444 b/tests/generic/444
index 8f6b7c2c..f7715e9c 100755
--- a/tests/generic/444
+++ b/tests/generic/444
@@ -31,6 +31,7 @@ setfacl -d -m u:101:rwx $TDIR
 chown 100:100 $TDIR
 chmod 2755 $TDIR

+sleep 10
 _runas -u 100 -g 100 -- mkdir $TDIR/testsub1
 stat -c %A $TDIR/testsub1
 _runas -u 100 -g 101 -- mkdir $TDIR/testsub2

When making the testsub1 and testsub2 in MDS side while preparing new Cinode, it will use parent diri's non-projected inode, please see https://github.com/ceph/ceph/blob/main/src/mds/Server.cc#L3361-L3372.

Just wait for 10s, which is larger than 5s, the mdlog has been flushed.

I am not sure why with 5.18-rc5 it worked, while with new kernel code it won't.

Anyway we should fix it in MDS side first.

It was introduce by recent fixing from [1].

When setting the system.posix_acl_default it will always do sync request, which in MDS side it will revoke the Xx cap from kclient:

2080 <7>[  425.580581] ceph:  d_revalidate 0000000073151ee9 'testdir.444' inode 0000000015190edc offset 0x0
2081 <7>[  425.580588] ceph:  dentry_lease_is_valid - dentry 0000000073151ee9 = 0
2082 <7>[  425.580594] ceph:  __ceph_caps_issued_mask ino 0x10000000000 cap 00000000f795da95 issued pAsLsXsFsxcr (mask Fs)
2083 <7>[  425.580600] ceph:  __touch_cap 000000003068a869 cap 00000000f795da95 mds0
2084 <7>[  425.580606] ceph:  dentry_dir_lease_touch 000000005f9dcb27 0000000073151ee9 'testdir.444' (offset 0x0)
2085 <7>[  425.580613] ceph:  dir_lease_is_valid dir 000000003068a869 v3 dentry 0000000073151ee9 = 1
2086 <7>[  425.580619] ceph:  d_revalidate 0000000073151ee9 valid
2087 <7>[  425.581070] ceph:  __ceph_caps_issued 0000000015190edc cap 00000000dcba0f3b issued pAsxLsXsxFsx
2088 <7>[  425.581084] ceph:  __get_required_blob_size c=0 names.size=0 vals.size=0
2089 <7>[  425.581095] ceph:  __ceph_setxattr do sync setxattr: version: 1 size: 80 max: 0                                                             
2090 <7>[  425.581177] ceph:  setxattr value size: 44
2091 <7>[  425.581323] ceph:  xattr.ver (before): 1
2092 <7>[  425.581332] ceph:  do_request on 00000000429e7c02
2093 <7>[  425.581341] ceph:  submit_request on 00000000429e7c02 for inode 0000000000000000
2094 <7>[  425.581354] ceph:  reserve caps ctx=00000000ae8fc6af need=1
2095 <7>[  425.581360] ceph:  reserve caps ctx=00000000ae8fc6af 7 = 5 used + 1 resv + 1 avail
2096 <7>[  425.581367] ceph:  __register_request 00000000429e7c02 tid 11
2097 <7>[  425.581374] ceph:  __choose_mds 0000000015190edc is_hash=0 (0x0) mode 2
2098 <7>[  425.581382] ceph:  __choose_mds 0000000015190edc 100000005df.fffffffffffffffe mds0 (auth cap 00000000dcba0f3b)
2099 <7>[  425.581393] ceph:  do_request mds0 session 00000000146e08e4 state open
2100 <7>[  425.581401] ceph:  __prepare_send_request 00000000429e7c02 tid 11 setxattr (attempt 1)
2101 <7>[  425.581409] ceph:   inode 0000000015190edc 100000005df.fffffffffffffffe
2102 <7>[  425.581416] ceph:   path system.posix_acl_default

Then later in MDS side in [2] when making new testsub1 directory and preparing the new CInode it will always get the non-projected inode, but the mdlog hasn't flushed yet from previous setxattr request as mentioned in https://tracker.ceph.com/issues/56010?issue_count=86&issue_position=1&next_issue_id=55857#note-1.

I will fix it in MDS side instead.

[1] https://github.com/ceph/ceph-client/commit/dea94541077519a9c62dba9f980b1509e5a52872
[2] https://github.com/ceph/ceph/blob/main/src/mds/Server.cc#L3361-L3372