Project

General

Profile

Bug #57218

qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails

Added by Kotresh Hiremath Ravishankar 3 months ago. Updated 3 months ago.

Status:
Triaged
Priority:
Normal
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Seen in https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/6979587

2022-08-19T07:33:22.464 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/teuthology/run_tasks.py", line 106, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_5cd353a5faa5cc051c0002bf5cca2445d5285a85/qa/tasks/mds_thrash.py", line 415, in task
    assert manager.is_clean()
AssertionError
2022-08-19T07:33:22.682 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2e500c24ddd0482daabe6d656e48e7db
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_b1d387f12b117399cb87c86aaa341398fa0c0919/teuthology/run_tasks.py", line 106, in run_tasks
    manager.__enter__()
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_5cd353a5faa5cc051c0002bf5cca2445d5285a85/qa/tasks/mds_thrash.py", line 415, in task
    assert manager.is_clean()
AssertionError

The mds trasher didn't find all the pgs clean.

  def is_clean(self):
        """ 
        True if all pgs are clean
        """ 
        pgs = self.get_pg_stats()
        if self._get_num_active_clean(pgs) == len(pgs):
            return True
        else:
            self.dump_pgs_not_active_clean()
            return False

NOTE: The issue is seen only once and the subsequent run [1] has passed. I am approving the PRs as it has passed the second time.
The PRs included in the batch are as below.

https://github.com/ceph/ceph/pull/46949 - pacific: ceph-fuse: add dedicated snap stag map for each directory
https://github.com/ceph/ceph/pull/47056 - pacific: mds: notify the xattr_version to replica MDSes
https://github.com/ceph/ceph/pull/47059 - pacific: mds: switch to use projected inode instead
https://github.com/ceph/ceph/pull/47180 - pacific: mds: fix crash when exporting unlinked dir
https://github.com/ceph/ceph/pull/47433 - pacific: mds: skip fetching the dirfrags if not a directory

[1] https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smithi/6981274

History

#1 Updated by Venky Shankar 3 months ago

  • Category set to Correctness/Safety
  • Assignee set to Kotresh Hiremath Ravishankar

#2 Updated by Venky Shankar 3 months ago

  • Status changed from New to Triaged
  • Target version set to v18.0.0

Also available in: Atom PDF