Ceph » CephFS

2020-06-24T23:17:26.500 INFO:tasks.workunit.client.0.smithi065.stderr:++ /home/ubuntu/cephtest/mnt.0/client.0/tmp/tmp/../pjd-fstest-20090130-RC/tests/xacl/../../fstest -u 65531 -g 65532 chown fstest_558e69cc508cd3cec9561f3d93555013/fstest_ce1a878fb21d47b6e5c0c0d4dc17d42f 65531 65532
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:++ tail -1
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:+ r=EIO
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:+ echo EIO
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:+ egrep '^EPERM$'
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:+ '[' 1 -eq 0 ']'
2020-06-24T23:17:26.501 INFO:tasks.workunit.client.0.smithi065.stderr:+ echo 'not ok 77'

From: /ceph/teuthology-archive/pdonnell-2020-06-24_22:47:27-fs-wip-pdonnell-testing-20200624.212840-distro-basic-smithi/5176636/teuthology.log

and many other tests in that run. Also other runs affected. So far I've only seen this on CentOS 8 and RHEL 8.

client log reports:

2020-06-24T23:17:25.234+0000 7fefa9ffb700  8 client.4716 _ll_setattrx 0x10000000584.head mask 6
2020-06-24T23:17:25.234+0000 7fefa9ffb700 20 client.4716 may_setattr 0x10000000584.head(faked_ino=0 ref=7 ll_ref=14 cap_refs={} open={1=0} mode=100454 size=0/0 nlink=1 btime=2020-06-24T23:17:25.169887+0000 mtime=2020-06-24T23:17:25.169887+0000 ctime=2020-06-24T23:17:25.216351+0000 caps=pAsxLsXsFscr(0=pAsxLsXsFscr) dirty_caps=Ax flushing_caps=Ax objectset[0x10000000584 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000583.head["fstest_ce1a878fb21d47b6e5c0c0d4dc17d42f"] 0x7fefac05ccb0); UserPerm(uid: 0, gid: 0)
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 client.4716 _getattr mask AsXs issued=1
2020-06-24T23:17:25.234+0000 7fefa9ffb700  3 client.4716 may_setattr 0x7fefac05ccb0 = 0
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 client.4716 _do_setattr mask 6 issued pAsxLsXsFscr
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 mark_caps_dirty 0x10000000584.head(faked_ino=0 ref=7 ll_ref=14 cap_refs={} open={1=0} mode=100454 size=0/0 nlink=1 btime=2020-06-24T23:17:25.169887+0000 mtime=2020-06-24T23:17:25.169887+0000 ctime=2020-06-24T23:17:25.235611+0000 caps=pAsxLsXsFscr(0=pAsxLsXsFscr) dirty_caps=Ax flushing_caps=Ax objectset[0x10000000584 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000583.head["fstest_ce1a878fb21d47b6e5c0c0d4dc17d42f"] 0x7fefac05ccb0) Ax -> Ax
HERE
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 client.4716 changing uid to 65531
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 mark_caps_dirty 0x10000000584.head(faked_ino=0 ref=7 ll_ref=14 cap_refs={} open={1=0} mode=100454 size=0/0 nlink=1 btime=2020-06-24T23:17:25.169887+0000 mtime=2020-06-24T23:17:25.169887+0000 ctime=2020-06-24T23:17:25.235617+0000 caps=pAsxLsXsFscr(0=pAsxLsXsFscr) dirty_caps=Ax flushing_caps=Ax objectset[0x10000000584 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000583.head["fstest_ce1a878fb21d47b6e5c0c0d4dc17d42f"] 0x7fefac05ccb0) Ax -> Ax
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 client.4716 changing gid to 65532
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 mark_caps_dirty 0x10000000584.head(faked_ino=0 ref=7 ll_ref=14 cap_refs={} open={1=0} mode=100454 size=0/0 nlink=1 btime=2020-06-24T23:17:25.169887+0000 mtime=2020-06-24T23:17:25.169887+0000 ctime=2020-06-24T23:17:25.235617+0000 caps=pAsxLsXsFscr(0=pAsxLsXsFscr) dirty_caps=Ax flushing_caps=Ax objectset[0x10000000584 ts 0/0 objects 0 dirty_or_tx 0] parents=0x10000000583.head["fstest_ce1a878fb21d47b6e5c0c0d4dc17d42f"] 0x7fefac05ccb0) Ax -> Ax
2020-06-24T23:17:25.234+0000 7fefa9ffb700 10 client.4716 fill_stat on 0x10000000584 snap/devhead mode 0100454 mtime 2020-06-24T23:17:25.169887+0000 ctime 2020-06-24T23:17:25.235617+0000
2020-06-24T23:17:25.234+0000 7fefa9ffb700  3 client.4716 ll_setattr 0x10000000584.head = 0

From: /ceph/teuthology-archive/pdonnell-2020-06-24_22:47:27-fs-wip-pdonnell-testing-20200624.212840-distro-basic-smithi/5176636/remote/smithi065/log/ceph-client.0.37654.log.gz

Which would indicate the operation succeeds from ceph-fuse perspective. So this looks like some issue in libfuse?