Project

General

Profile

Actions
Copy link

Bug #65246

open

qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)

Added by Rishabh Dave about 1 month ago. Updated 21 days ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
Rishabh Dave
Category:
Testing
Target version:
Ceph - v20.0.0
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
qa, qa-failure
Pull request ID:
56846
Crash signature (v1):
Crash signature (v2):

Description

The failure was found here - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/7625771. Following is the traceback from this log -

2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:======================================================================
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:ERROR: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:Test root_squash with multi fs
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephfs/test_admin.py", line 1427, in test_multifs_single_path_rootsquash
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:    self.captester = CapTester(self.mount_a, '/')
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephfs/caps_helper.py", line 318, in __init__
2024-03-27T10:50:14.697 INFO:tasks.cephfs_test_runner:    self._create_test_files(mount, path)
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephfs/caps_helper.py", line 208, in _create_test_files
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    self.path = self._gen_test_file_path(path)
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephfs/caps_helper.py", line 232, in _gen_test_file_path
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    self.mount.run_shell(f'mkdir {dirpath}')
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_155268c4e432a12433aa833f174f9fe3b1016ae0/qa/tasks/cephfs/mount.py", line 786, in run_shell
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    return self.client_remote.run(args=args, **kwargs)
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    r.wait()
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:    raise CommandFailedError(
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/testdir)'
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:
2024-03-27T10:50:14.698 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2024-03-27T10:50:14.699 INFO:tasks.cephfs_test_runner:Ran 28 tests in 1361.269s

This test was merged in this PR - https://github.com/ceph/ceph/pull/55833 and hasn't been modified since. The issue wasn't caught so far because the PR wasn't merged on the testing branch during QA.

Here's the GH comment that reports QA results - https://github.com/ceph/ceph/pull/55833#pullrequestreview-1936365177
Here's the QA result on wiki - https://tracker.ceph.com/projects/cephfs/wiki/Main#14th-March-2024
Here's the QA run - https://pulpito.ceph.com/?branch=wip-vshankar-testing-20240307.013758
Here'the testing branch - https://github.com/ceph/ceph-ci/commits/wip-vshankar-testing-20240307.013758/

Searching for 55833 in commits of this testing branch yields nothing.

Actions
Copy link
 #1

Updated by Rishabh Dave about 1 month ago

I was successful in reproducing this test failure with a recent version main branch - https://pulpito.ceph.com/rishabh-2024-04-01_10:16:38-fs:functional-rishabh-main-2024mar26-1827-testing-default-smithi/

Actions
Copy link
 #2

Updated by Rishabh Dave about 1 month ago

The issue can't be reproduced with kclient - https://pulpito.ceph.com/rishabh-2024-04-01_06:24:56-fs:functional-rishabh-main-2024mar26-1827-testing-default-smithi/

2024-04-01T07:11:55.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-04-01T07:11:55.278+0000 7f42acff9640  1 -- 172.21.15.16:0/621206058 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_admin.TestFsAuthorize.test_multifs_single_path_rootsquash"]}]=0  v1230) ==== 154+0+0 (secure 0 0 0) 0x7f42a0059d80 con 0x7f42b0154f70
...
2024-04-01T07:11:55.292 INFO:tasks.cephfs_test_runner:Test root_squash with multi fs ... skipped 'only FUSE client has CEPHFS_FEATURE_MDS_AUTH_CAPS needed to enforce root_squash MDS caps'

    def test_multifs_single_path_rootsquash(self):
        """ 
        Test root_squash with multi fs
        """

Here's another instance of the same - https://pulpito.ceph.com/rishabh-2024-03-29_19:05:44-fs:functional-rishabh-main-2024mar26-1827-testing-default-smithi/

Actions
Copy link
 #3

Updated by Venky Shankar about 1 month ago

  • Category set to Testing
  • Status changed from New to Triaged
  • Target version set to v20.0.0
Actions
Copy link
 #4

Updated by Rishabh Dave 21 days ago

  • Status changed from Triaged to Fix Under Review
  • Pull request ID set to 56846
Actions
Copy link

Also available in: Atom PDF