Project

General

Profile

Bug #57087

qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure

Added by Kotresh Hiremath Ravishankar 4 months ago. Updated 3 months ago.

Status:
Triaged
Priority:
Normal
Category:
Administration/Usability
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
tools
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Seen in https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/6959187

Looks like the data scan has not recovered the file

2022-08-05T01:56:30.914 INFO:teuthology.orchestra.run:Running command with timeout 900
2022-08-05T01:56:30.915 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo bash -c 'cat subdir/21')
2022-08-05T01:56:30.941 DEBUG:teuthology.orchestra.run:got remote process result: 1
2022-08-05T01:56:30.943 INFO:teuthology.orchestra.run.smithi087.stderr:cat: subdir/21: No such file or directory
2022-08-05T01:56:30.944 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules...

Traceback

2022-08-05T01:56:40.301 INFO:tasks.cephfs_test_runner:test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) ... ERROR
2022-08-05T01:56:40.302 INFO:tasks.cephfs_test_runner:
2022-08-05T01:56:40.302 INFO:tasks.cephfs_test_runner:======================================================================
2022-08-05T01:56:40.303 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)
2022-08-05T01:56:40.303 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2022-08-05T01:56:40.304 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2022-08-05T01:56:40.304 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_cee46c3e5b9015d27983e08f8ebddfb22d21d78e/qa/tasks/cephfs/test_data_scan.py", line 495, in test_fragmented_injection
2022-08-05T01:56:40.306 INFO:tasks.cephfs_test_runner:    out = self.mount_a.run_shell_payload(f"cat subdir/{victim_dentry}", sudo=True).stdout.getvalue().strip()
2022-08-05T01:56:40.306 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_cee46c3e5b9015d27983e08f8ebddfb22d21d78e/qa/tasks/cephfs/mount.py", line 675, in run_shell_payload
2022-08-05T01:56:40.307 INFO:tasks.cephfs_test_runner:    return self.run_shell(["bash", "-c", Raw(f"'{payload}'")], **kwargs)
2022-08-05T01:56:40.308 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_cee46c3e5b9015d27983e08f8ebddfb22d21d78e/qa/tasks/cephfs/mount.py", line 672, in run_shell
2022-08-05T01:56:40.309 INFO:tasks.cephfs_test_runner:    return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, stdout=stdout, stderr=stderr, **kwargs)
2022-08-05T01:56:40.309 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9e4bf63f00ab3f82a26635c2779f4c3f1b73fb53/teuthology/orchestra/remote.py", line 510, in run
2022-08-05T01:56:40.310 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2022-08-05T01:56:40.310 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9e4bf63f00ab3f82a26635c2779f4c3f1b73fb53/teuthology/orchestra/run.py", line 455, in run
2022-08-05T01:56:40.311 INFO:tasks.cephfs_test_runner:    r.wait()
2022-08-05T01:56:40.311 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9e4bf63f00ab3f82a26635c2779f4c3f1b73fb53/teuthology/orchestra/run.py", line 161, in wait
2022-08-05T01:56:40.312 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2022-08-05T01:56:40.312 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_9e4bf63f00ab3f82a26635c2779f4c3f1b73fb53/teuthology/orchestra/run.py", line 183, in _raise_for_status
2022-08-05T01:56:40.313 INFO:tasks.cephfs_test_runner:    node=self.hostname, label=self.label
2022-08-05T01:56:40.313 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec sudo bash -c 'cat subdir/21')" 

History

#1 Updated by Kotresh Hiremath Ravishankar 4 months ago

  • Description updated (diff)

#2 Updated by Kotresh Hiremath Ravishankar 4 months ago

Note that the test successfully passed on the re-run
https://pulpito.ceph.com/yuriw-2022-08-10_20:34:29-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi/6966101

The PRs included in this build for testing are as follows:

1. https://github.com/ceph/ceph/pull/46901 - pacific: qa/cephfs: fallback to older way of get_op_read_count
2. https://github.com/ceph/ceph/pull/47282 - pacific: mds: standby-replay daemon always removed in MDSMonitor::prepare_beacon
3. https://github.com/ceph/ceph/pull/47307 - pacific: mgr/telemetry: reset health warning after re-opting-in
4. https://github.com/ceph/ceph/pull/47369 - pacific: mgr/volumes: Fix subvolume creation in FIPS enabled system.

#3 Updated by Venky Shankar 3 months ago

  • Category set to Administration/Usability
  • Status changed from New to Triaged
  • Assignee set to Milind Changire
  • Target version set to v18.0.0

Also available in: Atom PDF