Bug #63141: qa/cephfs: test_idem_unaffected_root_squash fails - CephFS - Ceph

Actions

Copy link

Bug #63141

open

qa/cephfs: test_idem_unaffected_root_squash fails

Added by Rishabh Dave 7 months ago. Updated 26 days ago.

Status:

Pending Backport

Priority:

Normal

Assignee:

Xiubo Li

Category:

Correctness/Safety

Target version:

Ceph - v19.0.0

% Done:

Source:

Tags:

backport_processed

Backport:

quincy, reef, squid

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

qa-suite

Labels (FS):

qa, qa-failure

Pull request ID:

54778

Crash signature (v1):

Crash signature (v2):

Description

test_admin.TestFsAuthorizeUpdate.test_idem_unaffected_root_squash fails with following traceback -

2023-10-06T23:37:32.389 INFO:tasks.cephfs_test_runner:======================================================================
2023-10-06T23:37:32.389 INFO:tasks.cephfs_test_runner:ERROR: test_idem_unaffected_root_squash (tasks.cephfs.test_admin.TestFsAuthorizeUpdate)
2023-10-06T23:37:32.389 INFO:tasks.cephfs_test_runner:Test that "root_squash" is not deleted from MDS caps when user runs
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/test_admin.py", line 1556, in test_idem_unaffected_root_squash
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:    self._remount(self.mount_a, self.fs.name, keyring, PATH)
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/test_admin.py", line 1648, in _remount
2023-10-06T23:37:32.390 INFO:tasks.cephfs_test_runner:    mount_x.remount(client_id=self.client_id,
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/mount.py", line 594, in remount
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:    retval = self.mount(mntopts=mntopts, check_status=check_status)
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/kernel_mount.py", line 59, in mount
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:    retval = self._run_mount_cmd(mntopts, check_status)
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_ceph-c_14d8d032507701c76ca224e769ee65a96b660a8c/qa/tasks/cephfs/kernel_mount.py", line 83, in _run_mount_cmd
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:    self.client_remote.run(args=mount_cmd, timeout=300,
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:    r.wait()
2023-10-06T23:37:32.391 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait
2023-10-06T23:37:32.392 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2023-10-06T23:37:32.392 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status
2023-10-06T23:37:32.392 INFO:tasks.cephfs_test_runner:    raise CommandFailedError(
2023-10-06T23:37:32.392 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi082 with status 32: 'sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/dir1 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,secret=AQAumiBl36BaBhAAGMba3BmR0NFayKOV7dwdNA==,conf=/etc/ceph/ceph.conf,norbytes,name=testuser,mds_namespace=cephfs'

Initially the failure was seen while testing PR 50212 (which did not have related changes) -
https://pulpito.ceph.com/rishabh-2023-10-09_07:55:14-fs-rishabh-fs-swap-subcmd-testing-default-smithi
https://pulpito.ceph.com/rishabh-2023-10-05_14:34:02-fs-rishabh-fs-swap-subcmd-testing-default-smithi
https://pulpito.ceph.com/rishabh-2023-10-05_12:39:10-fs-rishabh-fs-swap-subcmd-testing-default-smithi

And now then it was seen while running integration tests for CephFS PRs -
https://pulpito.ceph.com/rishabh-2023-10-06_11:56:52-fs-rishabh-cephfs-mon-testing-default-smithi/7414755

After disabling running of this paricular test, test_admin.py runs successfully (for PR 50212; it adds more tests to test_admin.py) -
https://pulpito.ceph.com/rishabh-2023-10-09_08:10:08-fs-rishabh-fs-swap-subcmd-testing-default-smithi/

Related issues 3 (1 open — 2 closed)

Actions

Copy link

Updated by Rishabh Dave 7 months ago

With vstart_runner.py, the test is running fine -

2023-10-09 16:49:27,459.459 DEBUG:__main__:> ./bin/ceph log 'Ended test tasks.cephfs.test_admin.TestFsAuthorizeUpdate.test_idem_unaffected_root_squash'                            
2023-10-09 16:49:28,279.279 INFO:__main__:Stopped test: test_idem_unaffected_root_squash (tasks.cephfs.test_admin.TestFsAuthorizeUpdate.test_idem_unaffected_root_squash)          
Test that "root_squash" is not deleted from MDS caps when user runs in 80.101708s                                                                                                  
2023-10-09 16:49:28,279.279 INFO:__main__:test_idem_unaffected_root_squash (tasks.cephfs.test_admin.TestFsAuthorizeUpdate.test_idem_unaffected_root_squash)                        
2023-10-09 16:49:28,279.279 INFO:__main__:Test that "root_squash" is not deleted from MDS caps when user runs ... ok                                                               
2023-10-09 16:49:28,279.279 INFO:__main__:                                                                                                                                         
2023-10-09 16:49:28,279.279 INFO:__main__:----------------------------------------------------------------------                                                                   
2023-10-09 16:49:28,280.280 INFO:__main__:Ran 1 test in 80.102s                                                                                                                    
2023-10-09 16:49:28,280.280 INFO:__main__:

Actions

Copy link