Bug #57218: qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails - CephFS - Ceph

Actions

Copy link

Bug #57218

open

qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails

Added by Kotresh Hiremath Ravishankar over 1 year ago. Updated 7 months ago.

Status:

Triaged

Priority:

Normal

Assignee:

Kotresh Hiremath Ravishankar

Category:

Correctness/Safety

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

Labels (FS):

qa, qa-failure

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Seen in https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/6979587

2022-08-19T07:33:22.464 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/teuthology/run_tasks.py", line 106, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_5cd353a5faa5cc051c0002bf5cca2445d5285a85/qa/tasks/mds_thrash.py", line 415, in task
    assert manager.is_clean()
AssertionError
2022-08-19T07:33:22.682 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2e500c24ddd0482daabe6d656e48e7db
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/teuthology/run_tasks.py", line 106, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_5cd353a5faa5cc051c0002bf5cca2445d5285a85/qa/tasks/mds_thrash.py", line 415, in task
    assert manager.is_clean()
AssertionError

The mds trasher didn't find all the pgs clean.

  def is_clean(self):
        """ 
        True if all pgs are clean
        """ 
        pgs = self.get_pg_stats()
        if self._get_num_active_clean(pgs) == len(pgs):
            return True
        else:
            self.dump_pgs_not_active_clean()
            return False

NOTE: The issue is seen only once and the subsequent run [1] has passed. I am approving the PRs as it has passed the second time.
The PRs included in the batch are as below.

https://github.com/ceph/ceph/pull/46949 - pacific: ceph-fuse: add dedicated snap stag map for each directory
https://github.com/ceph/ceph/pull/47056 - pacific: mds: notify the xattr_version to replica MDSes
https://github.com/ceph/ceph/pull/47059 - pacific: mds: switch to use projected inode instead
https://github.com/ceph/ceph/pull/47180 - pacific: mds: fix crash when exporting unlinked dir
https://github.com/ceph/ceph/pull/47433 - pacific: mds: skip fetching the dirfrags if not a directory

[1] https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/6981274