Ceph » CephFS

h1. MAIN

h3. NEW ENTRY BELOW

h3. 5 Sep 2023

https://pulpito.ceph.com/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/

orch:cephadm suite run: http://pulpito.front.sepia.ceph.com/rishabh-2023-09-05_12:16:09-orch:cephadm-wip-rishabh-2023aug3-b5-testing-default-smithi/

  this run has failures but acc to Adam King these are not relevant and should be ignored

* https://tracker.ceph.com/issues/61892

  test_snapshot_remove (test_strays.TestStrays) failed

* https://tracker.ceph.com/issues/59348

  test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota

* https://tracker.ceph.com/issues/54462

  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128

* https://tracker.ceph.com/issues/62067

  ffsb.sh failure "Resource temporarily unavailable"

* https://tracker.ceph.com/issues/57656 

  dbench: write failed on handle 10010 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/57655

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/62187

  iozone.sh: line 5: iozone: command not found

* https://tracker.ceph.com/issues/61399

  ior build failure

* https://tracker.ceph.com/issues/57676

  qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/55805

  error scrub thrashing reached max tries in 900 secs

h3. 31 Aug 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230824.045828

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/62187

    iozone: command not found

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/59531

    quincy: "OSD bench result of 228617.361065 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio)"

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59413

    cephfs: qa snaptest-git-ceph.sh failed with "got remote process result: 128"

* https://tracker.ceph.com/issues/62653

    qa: unimplemented fcntl command: 1036 with fsstress

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/62658

    error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds

* https://tracker.ceph.com/issues/62188

    AttributeError: 'RemoteProcess' object has no attribute 'read' (happens only with multis-auth test)

h3. 25 Aug 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230822.064807

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61243

    test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) failed

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/62484

    qa: ffsb.sh test failure

* https://tracker.ceph.com/issues/59531

    quincy: "OSD bench result of 228617.361065 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.7. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio)"

* https://tracker.ceph.com/issues/62510

    snaptest-git-ceph.sh failure with fs/thrash

h3. 24 Aug 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230822.060131

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/62510

    snaptest-git-ceph.sh failure with fs/thrash

* https://tracker.ceph.com/issues/62484

    qa: ffsb.sh test failure

* https://tracker.ceph.com/issues/57087

    qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/62187

    iozone: command not found

* https://tracker.ceph.com/issues/62188

    AttributeError: 'RemoteProcess' object has no attribute 'read' (happens only with multis-auth test)

* https://tracker.ceph.com/issues/62567

    postgres workunit times out - MDS_SLOW_REQUEST in logs

h3. 22 Aug 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230809.035933

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61243

    test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) failed

* https://tracker.ceph.com/issues/62188

    AttributeError: 'RemoteProcess' object has no attribute 'read' (happens only with multis-auth test)

* https://tracker.ceph.com/issues/62510

    snaptest-git-ceph.sh failure with fs/thrash

* https://tracker.ceph.com/issues/62511

    src/mds/MDLog.cc: 299: FAILED ceph_assert(!mds_is_shutting_down)

h3. 14 Aug 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230808.093601

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59531

    cluster [WRN] OSD bench result of 137706.272521 IOPS exceeded the threshold

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/59684 [kclient bug]

    Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)

* https://tracker.ceph.com/issues/61243 (NEW)

    test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) failed

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

h3. 28 JULY 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230725.053049

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/61399

    ior build failure

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59531

    cluster [WRN] OSD bench result of 137706.272521 IOPS exceeded the threshold

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://github.com/ceph/ceph/pull/52556

    task/test_nfs: AttributeError: 'TestNFS' object has no attribute 'run_ceph_cmd' (see note #4)

* https://tracker.ceph.com/issues/62187

    iozone: command not found

* https://tracker.ceph.com/issues/61399

    qa: build failure for ior (the failed instance is when compiling `mdtest`)

* https://tracker.ceph.com/issues/62188

    AttributeError: 'RemoteProcess' object has no attribute 'read' (happens only with multis-auth test)

h3. 24 Jul 2023

https://pulpito.ceph.com/rishabh-2023-07-13_21:35:13-fs-wip-rishabh-2023Jul13-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-07-14_10:26:42-fs-wip-rishabh-2023Jul13-testing-default-smithi/

There were few failure from one of the PRs under testing. Following run confirms that removing this PR fixes these failures -

https://pulpito.ceph.com/rishabh-2023-07-18_02:11:50-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/

One more extra run to check if blogbench.sh fail every time:

https://pulpito.ceph.com/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/

blogbench.sh failure were seen on above runs for first time, following run with main branch that confirms that "blogbench.sh" was not related to any of the PRs that are under testing -

https://pulpito.ceph.com/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/

* https://tracker.ceph.com/issues/61892

  test_snapshot_remove (test_strays.TestStrays) failed

* https://tracker.ceph.com/issues/53859

  test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/61982

  test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)

* https://tracker.ceph.com/issues/52438

  qa: ffsb timeout

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57655

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/48773

  reached max tries: scrub does not complete

* https://tracker.ceph.com/issues/58340

  mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/61400

  valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/57206

  libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10010 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/61399

  ior build failure

* https://tracker.ceph.com/issues/57676

  error during scrub thrashing: backtrace

* https://tracker.ceph.com/issues/38452

  'sudo -u postgres -- pgbench -s 500 -i' failed

* https://tracker.ceph.com/issues/62126

  blogbench.sh failure

h3. 18 July 2023

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/61182

    cephfs-mirror-ha-workunit: reached maximum tries (50) after waiting for 300 seconds

* https://tracker.ceph.com/issues/61957

    test_client_limits.TestClientLimits.test_client_release_bug

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61892

    test_strays.TestStrays.test_snapshot_remove failed

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/44565

    src/mds/SimpleLock.h: 528: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock())

* https://tracker.ceph.com/issues/62067

    ffsb.sh failure "Resource temporarily unavailable"

h3. 17 July 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230704.040136

* https://tracker.ceph.com/issues/61982

    Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/61182

    cephfs-mirror-ha-workunit: reached maximum tries (50) after waiting for 300 seconds

* https://tracker.ceph.com/issues/61957

    test_client_limits.TestClientLimits.test_client_release_bug

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon: segmentation fault in rocksdb+tcmalloc

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61892

    test_strays.TestStrays.test_snapshot_remove failed

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/62036

    src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())

* https://tracker.ceph.com/issues/61737

    coredump from '/bin/podman pull quay.ceph.io/ceph-ci/ceph:pacific'

* https://tracker.ceph.com/issues/44565

    src/mds/SimpleLock.h: 528: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock())

h3. 13 July 2023 Run 2

https://pulpito.ceph.com/rishabh-2023-07-08_23:33:40-fs-wip-rishabh-2023Jul9-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-07-09_20:19:09-fs-wip-rishabh-2023Jul9-testing-default-smithi/

* https://tracker.ceph.com/issues/61957

  test_client_limits.TestClientLimits.test_client_release_bug

* https://tracker.ceph.com/issues/61982

  Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57655

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61400

  valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/61399

  ior build failure

h3. 13 July 2023

https://pulpito.ceph.com/vshankar-2023-07-04_11:45:30-fs-wip-vshankar-testing-20230704.040242-testing-default-smithi/

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon: segmentation fault in rocksdb+tcmalloc

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61945

    LibCephFS.DelegTimeout failure

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59346

    fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/61982

    Test failure: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)

h3. 13 Jul 2023

https://pulpito.ceph.com/rishabh-2023-07-05_22:21:20-fs-wip-rishabh-2023Jul5-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-07-06_19:33:28-fs-wip-rishabh-2023Jul5-testing-default-smithi/

* https://tracker.ceph.com/issues/61957

  test_client_limits.TestClientLimits.test_client_release_bug

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write" 

* https://tracker.ceph.com/issues/48773

  scrub does not complete: reached max tries

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument" 

* https://tracker.ceph.com/issues/52438

  qa: ffsb timeout

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/58742

  xfstests-dev: kcephfs: generic

* https://tracker.ceph.com/issues/61399

  libmpich: undefined references to fi_strerror

h3. 12 July 2023

https://pulpito.ceph.com/rishabh-2023-07-05_18:32:52-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/

* https://tracker.ceph.com/issues/61892

  test_strays.TestStrays.test_snapshot_remove failed

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/53859

  qa: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

* https://tracker.ceph.com/issues/58742

  xfstests-dev: kcephfs: generic

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/52438

  qa: ffsb timeout

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57655

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61182

  cephfs-mirror-ha-workunit: reached maximum tries (50) after waiting for 300 seconds

* https://tracker.ceph.com/issues/61400

  valgrind+ceph-mon issues: sudo ceph --cluster ceph osd crush tunables default

* https://tracker.ceph.com/issues/48773

  reached max tries: scrub does not complete

h3. 05 July 2023

https://pulpito.ceph.com/pdonnell-2023-07-05_03:38:33-fs:libcephfs-wip-pdonnell-testing-20230705.003205-distro-default-smithi/

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

h3. 27 Jun 2023

https://pulpito.ceph.com/rishabh-2023-06-21_23:38:17-fs-wip-rishabh-improvements-authmon-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/61399

  libmpich: undefined references to fi_strerror

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/61831

  Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring)

h3. 22 June 2023

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon: segmentation fault in rocksdb+tcmalloc

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61394

    qa/quincy: cluster [WRN] evicting unresponsive client smithi152 (4298), after 303.726 seconds" in cluster log

* https://tracker.ceph.com/issues/61762

    qa: wait_for_clean: failed before timeout expired

* https://tracker.ceph.com/issues/61775

    cephfs-mirror: mirror daemon does not shutdown (in mirror ha tests)

* https://tracker.ceph.com/issues/44565

    src/mds/SimpleLock.h: 528: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock())

* https://tracker.ceph.com/issues/61790

    cephfs client to mds comms remain silent after reconnect

* https://tracker.ceph.com/issues/61791

    snaptest-git-ceph.sh test timed out (job dead)

h3. 20 June 2023

https://pulpito.ceph.com/vshankar-2023-06-15_04:58:28-fs-wip-vshankar-testing-20230614.124123-testing-default-smithi/

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/54462

  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128

* https://tracker.ceph.com/issues/58340

  mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon: segmentation fault in rocksdb+tcmalloc

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/44565

    src/mds/SimpleLock.h: 528: FAILED ceph_assert(state == LOCK_XLOCK || state == LOCK_XLOCKDONE || state == LOCK_XLOCKSNAP || state == LOCK_LOCK_XLOCK || state == LOCK_LOCK || is_locallock())

* https://tracker.ceph.com/issues/61737

    coredump from '/bin/podman pull quay.ceph.io/ceph-ci/ceph:pacific'

h3. 16 June 2023

https://pulpito.ceph.com/rishabh-2023-05-16_10:39:13-fs-wip-rishabh-2023May15-1524-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-05-17_11:09:48-fs-wip-rishabh-2023May15-1524-testing-default-smithi/

https://pulpito.ceph.com/rishabh-2023-05-18_10:01:53-fs-wip-rishabh-2023May15-1524-testing-default-smithi/

(bins were rebuilt with a subset of orig PRs) https://pulpito.ceph.com/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/

* https://tracker.ceph.com/issues/59344

  qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59346

  fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/54462

  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128

* https://tracker.ceph.com/issues/61399

  libmpich: undefined references to fi_strerror

* https://tracker.ceph.com/issues/58945

  xfstests-dev: ceph-fuse: generic 

* https://tracker.ceph.com/issues/58742

  xfstests-dev: kcephfs: generic

h3. 24 May 2023

https://pulpito.ceph.com/pdonnell-2023-05-23_18:20:18-fs-wip-pdonnell-testing-20230523.134409-distro-default-smithi/

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/59683

    Error: Unable to find a match: userspace-rcu-devel libedit-devel device-mapper-devel with fscrypt tests

* https://tracker.ceph.com/issues/61399

    qa: "[Makefile:299: ior] Error 1"

* https://tracker.ceph.com/issues/61265

    qa: tasks.cephfs.fuse_mount:process failed to terminate after unmount

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/59346

    qa/workunits/fs/test_python.sh failed with "AssertionError: DiskQuotaExceeded not raised by write"

* https://tracker.ceph.com/issues/61400

    valgrind+ceph-mon: segmentation fault in rocksdb+tcmalloc

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59344

    qa: workunit test fs/quota/quota.sh failed with "setfattr: .: Invalid argument"

* https://tracker.ceph.com/issues/61407

    mds: abort on CInode::verify_dirfrags

* https://tracker.ceph.com/issues/48773

    qa: scrub does not complete

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/61409

    qa: _test_stale_caps does not wait for file flush before stat

h3. 15 May 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230509.090020

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230509.090020-6

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/59684 [kclient bug]

    Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/61148

    dbench test results in call trace in dmesg [kclient bug]

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

h3. 11 May 2023

https://pulpito.ceph.com/yuriw-2023-05-10_18:21:40-fs-wip-yuri7-testing-2023-05-10-0742-distro-default-smithi/

* https://tracker.ceph.com/issues/59684 [kclient bug]

  Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)

* https://tracker.ceph.com/issues/59348

  qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

* https://tracker.ceph.com/issues/57655

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/57676

  qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/55805

  error during scrub thrashing reached max tries in 900 secs

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57656

  [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/58220

  Command failed (workunit test fs/quota/quota.sh) on smithi081 with status 1:

* https://tracker.ceph.com/issues/58220#note-9

  workunit fs/test_python.sh: test_disk_quota_exceeeded_error failure

* https://tracker.ceph.com/issues/59342

  qa/workunits/kernel_untar_build.sh failed when compiling the Linux source

* https://tracker.ceph.com/issues/58949

    test_cephfs.test_disk_quota_exceeeded_error - AssertionError: DiskQuotaExceeded not raised by write

* https://tracker.ceph.com/issues/61243 (NEW)

  test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) failed

h3. 11 May 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230509.054005

(no fsstress job failure [https://tracker.ceph.com/issues/58340] since https://github.com/ceph/ceph/pull/49553

 was included in the branch, however, the PR got updated and needs retest).

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/59683

    Error: Unable to find a match: userspace-rcu-devel libedit-devel device-mapper-devel with fscrypt tests

* https://tracker.ceph.com/issues/59684 [kclient bug]

    Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

h3. 09 May 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230506.143554

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/59350

    qa: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ERROR

* https://tracker.ceph.com/issues/59683

    Error: Unable to find a match: userspace-rcu-devel libedit-devel device-mapper-devel with fscrypt tests

* https://tracker.ceph.com/issues/59684 [kclient bug]

    Test failure: test_fscrypt_dummy_encryption_with_quick_group (tasks.cephfs.test_fscrypt.TestFscrypt)

* https://tracker.ceph.com/issues/59348

    qa: ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)

h3. 10 Apr 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230330.105356

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

h3. 31 Mar 2023

run: http://pulpito.front.sepia.ceph.com/rishabh-2023-03-03_21:39:49-fs-wip-rishabh-2023Mar03-2316-testing-default-smithi/

re-run (some PRs removed from batch): http://pulpito.front.sepia.ceph.com/rishabh-2023-03-11_05:54:03-fs-wip-rishabh-2023Mar10-1727-testing-default-smithi/

re-run (some PRs removed from batch): http://pulpito.front.sepia.ceph.com/rishabh-2023-03-23_08:27:28-fs-wip-rishabh-2023Mar20-2250-testing-default-smithi/

There were many more re-runs for "failed+dead" jobs as well as for individual jobs. half of the PRs from the batch were removed (gradually over subsequent re-runs).

* https://tracker.ceph.com/issues/57676

  qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58220

  Command failed (workunit test fs/quota/quota.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58220#note-9

  workunit fs/test_python.sh: test_disk_quota_exceeeded_error failure

* https://tracker.ceph.com/issues/56695

  Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/58564 

  workuit dbench failed with error code 1

* https://tracker.ceph.com/issues/57206

  libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/57580

  Test failure: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps)

* https://tracker.ceph.com/issues/58940

  ceph osd hit ceph_abort

* https://tracker.ceph.com/issues/55805

  error scrub thrashing reached max tries in 900 secs

h3. 30 March 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230315.085747

* https://tracker.ceph.com/issues/58938

    qa: xfstests-dev's generic test suite has 7 failures with kclient

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

h3. 29 March 2023

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20230317.095222

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/57087

    qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/59230

    Test failure: test_object_deletion (tasks.cephfs.test_damage.TestDamage)

* https://tracker.ceph.com/issues/58938

    qa: xfstests-dev's generic test suite has 7 failures with kclient

h3. 13 Mar 2023

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

h3. 09 Mar 2023

https://pulpito.ceph.com/vshankar-2023-03-03_04:39:14-fs-wip-vshankar-testing-20230303.023823-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2023-03-08_15:12:36-fs-wip-vshankar-testing-20230308.112059-testing-default-smithi/

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/57087

    qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure

h3. 07 Mar 2023

https://pulpito.ceph.com/vshankar-2023-03-02_09:21:58-fs-wip-vshankar-testing-20230222.044949-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2023-03-07_05:15:12-fs-wip-vshankar-testing-20230307.030510-testing-default-smithi/

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/58220

    Command failed (workunit test fs/quota/quota.sh) on smithi081 with status 1:

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58934

    snaptest-git-ceph.sh failure with ceph-fuse

h3. 28 Feb 2023

https://pulpito.ceph.com/vshankar-2023-02-24_02:11:45-fs-wip-vshankar-testing-20230222.025426-testing-default-smithi/

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

(teuthology infra issues causing testing delays - merging PRs which have tests passing)

h3. 25 Jan 2023

https://pulpito.ceph.com/vshankar-2023-01-25_07:57:32-fs-wip-vshankar-testing-20230125.055346-testing-default-smithi/

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/57206

    libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/58220

    Command failed (workunit test fs/quota/quota.sh) on smithi081 with status 1:

* https://tracker.ceph.com/issues/58340

  mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/56011

    fs/thrash: snaptest-snap-rm-cmp.sh fails in mds5sum comparison

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

h3. 30 JAN 2023

run: http://pulpito.front.sepia.ceph.com/rishabh-2022-11-28_08:04:11-fs-wip-rishabh-testing-2022Nov24-1818-testing-default-smithi/

re-run: http://pulpito.front.sepia.ceph.com/rishabh-2023-01-13_12:08:33-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/

re-run of re-run: http://pulpito.front.sepia.ceph.com/rishabh-2023-01-23_18:53:32-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/

* https://tracker.ceph.com/issues/52624

  qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" 

* https://tracker.ceph.com/issues/56695

  [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57676

  qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/55332

  Failure in snaptest-git-ceph.sh

* https://tracker.ceph.com/issues/51964

  qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/56446

  Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/57655 

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58340

  mds: fsstress.sh hangs with multimds

* https://tracker.ceph.com/issues/58219

  Command crashed: 'ceph-dencoder type inode_backtrace_t import - decode dump_json'

* "Failed to load ceph-mgr modules: prometheus" in cluster log"

  http://pulpito.front.sepia.ceph.com/rishabh-2023-01-23_18:53:32-fs-wip-rishabh-testing-2022Nov24-11Jan2023-distro-default-smithi/7134086

  Acc to Venky this was fixed in https://github.com/ceph/ceph/commit/cf6089200d96fc56b08ee17a4e31f19823370dc8

* Created https://tracker.ceph.com/issues/58564

  workunit test suites/dbench.sh failed error code 1

h3. 15 Dec 2022

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20221215.112736

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/58219

* https://tracker.ceph.com/issues/57655

* qa: fs:mixed-clients kernel_untar_build failure

    Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration)

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/58340

    mds: fsstress.sh hangs with multimds

h3. 08 Dec 2022

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20221130.043104

https://pulpito.ceph.com/?branch=wip-vshankar-testing-20221209.043803

(lots of transient git.ceph.com failures)

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/57655

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/58219

    Test failure: test_journal_migration (tasks.cephfs.test_journal_migration.TestJournalMigration)

* https://tracker.ceph.com/issues/58220

    Command failed (workunit test fs/quota/quota.sh) on smithi081 with status 1:

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/53859

    qa: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/58244

    Test failure: test_rebuild_inotable (tasks.cephfs.test_data_scan.TestDataScan)

h3. 14 Oct 2022

https://pulpito.ceph.com/vshankar-2022-10-12_04:56:59-fs-wip-vshankar-testing-20221011-145847-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2022-10-14_04:04:57-fs-wip-vshankar-testing-20221014-072608-testing-default-smithi/

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/55804

    Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/57682

    client: ERROR: test_reconnect_after_blocklisted

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

h3. 10 Oct 2022

http://pulpito.front.sepia.ceph.com/rishabh-2022-09-30_19:45:21-fs-wip-rishabh-testing-30Sep2022-testing-default-smithi/

reruns

* fs-thrash, passed: http://pulpito.front.sepia.ceph.com/rishabh-2022-10-04_13:19:47-fs-wip-rishabh-testing-30Sep2022-testing-default-smithi/

* fs-verify, passed: http://pulpito.front.sepia.ceph.com/rishabh-2022-10-05_12:25:37-fs-wip-rishabh-testing-30Sep2022-testing-default-smithi/

* cephadm failures also passed after many re-runs: http://pulpito.front.sepia.ceph.com/rishabh-2022-10-06_13:50:51-fs-wip-rishabh-testing-30Sep2022-2-testing-default-smithi/

    ** needed this PR to be merged in ceph-ci branch - https://github.com/ceph/ceph/pull/47458

known bugs

* https://tracker.ceph.com/issues/52624

  qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke

* https://tracker.ceph.com/issues/57299

  qa: test_dump_loads fails with JSONDecodeError

* https://tracker.ceph.com/issues/57655 [Exist in main as well]

  qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/57206

  libcephfs/test.sh: ceph_test_libcephfs_reclaim

h3. 2022 Sep 29

http://pulpito.front.sepia.ceph.com/rishabh-2022-09-14_12:48:43-fs-wip-rishabh-testing-2022Sep9-1708-testing-default-smithi/

* https://tracker.ceph.com/issues/55804

  Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/36593

  Command failed (workunit test fs/quota/quota.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/52624

  qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/51964

  qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/56632

  Test failure: test_subvolume_snapshot_clone_quota_exceeded

* https://tracker.ceph.com/issues/50821

  qa: untar_snap_rm failure during mds thrashing

h3. 2022 Sep 26

https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20220923.171109

* https://tracker.ceph.com/issues/55804

    qa failure: pjd link tests failed

* https://tracker.ceph.com/issues/57676

    qa: error during scrub thrashing: rank damage found: {'backtrace'}

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/57580

    Test failure: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps)

* https://tracker.ceph.com/issues/48773

    qa: scrub does not complete

* https://tracker.ceph.com/issues/57299

    qa: test_dump_loads fails with JSONDecodeError

* https://tracker.ceph.com/issues/57280

    qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package version from shaman

* https://tracker.ceph.com/issues/57205

    Test failure: test_subvolume_group_ls_filter_internal_directories (tasks.cephfs.test_volumes.TestSubvolumeGroups)

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/57677

    qa: "1 MDSs behind on trimming (MDS_TRIM)"

* https://tracker.ceph.com/issues/57206

    libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/57446

    qa: test_subvolume_snapshot_info_if_orphan_clone fails

* https://tracker.ceph.com/issues/57655 [Exist in main as well]

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/57682

    client: ERROR: test_reconnect_after_blocklisted

h3. 2022 Sep 22

https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20220920.234701

* https://tracker.ceph.com/issues/57299

    qa: test_dump_loads fails with JSONDecodeError

* https://tracker.ceph.com/issues/57205

    Test failure: test_subvolume_group_ls_filter_internal_directories (tasks.cephfs.test_volumes.TestSubvolumeGroups)

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/57580

    Test failure: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps)

* https://tracker.ceph.com/issues/57280

    qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package version from shaman

* https://tracker.ceph.com/issues/48773

    qa: scrub does not complete

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/57206

    libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/51267

    CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi096 with status 1:...

NEW:

* https://tracker.ceph.com/issues/57656

    [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)

* https://tracker.ceph.com/issues/57655 [Exist in main as well]

    qa: fs:mixed-clients kernel_untar_build failure

* https://tracker.ceph.com/issues/57657

    mds: scrub locates mismatch between child accounted_rstats and self rstats

Segfault probably caused by: https://github.com/ceph/ceph/pull/47795#issuecomment-1255724799

h3. 2022 Sep 16

https://pulpito.ceph.com/?branch=wip-vshankar-testing1-20220905-132828

* https://tracker.ceph.com/issues/57446

    qa: test_subvolume_snapshot_info_if_orphan_clone fails

* https://tracker.ceph.com/issues/57299

    qa: test_dump_loads fails with JSONDecodeError

* https://tracker.ceph.com/issues/50223

    client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/57205

    Test failure: test_subvolume_group_ls_filter_internal_directories (tasks.cephfs.test_volumes.TestSubvolumeGroups)

* https://tracker.ceph.com/issues/57280

    qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package version from shaman

* https://tracker.ceph.com/issues/51282

    pybind/mgr/mgr_util: .mgr pool may be created to early causing spurious PG_DEGRADED warnings

* https://tracker.ceph.com/issues/48203

  https://tracker.ceph.com/issues/36593

    qa: quota failure

    qa: quota failure caused by clients stepping on each other

* https://tracker.ceph.com/issues/57580

    Test failure: test_newops_getvxattr (tasks.cephfs.test_newops.TestNewOps)

h3. 2022 Aug 26

http://pulpito.front.sepia.ceph.com/rishabh-2022-08-22_17:49:59-fs-wip-rishabh-testing-2022Aug19-testing-default-smithi/

http://pulpito.front.sepia.ceph.com/rishabh-2022-08-24_11:56:51-fs-wip-rishabh-testing-2022Aug19-testing-default-smithi/

* https://tracker.ceph.com/issues/57206

  libcephfs/test.sh: ceph_test_libcephfs_reclaim

* https://tracker.ceph.com/issues/56632

  Test failure: test_subvolume_snapshot_clone_quota_exceeded (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)

* https://tracker.ceph.com/issues/56446

  Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/51964

  qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/53859

  qa: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/54462

  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/36593

  Command failed (workunit test fs/quota/quota.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/52624

  qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/55804

  Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke)

h3. 2022 Aug 22

https://pulpito.ceph.com/vshankar-2022-08-12_09:34:24-fs-wip-vshankar-testing1-20220812-072441-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2022-08-18_04:30:42-fs-wip-vshankar-testing1-20220818-082047-testing-default-smithi/ (drop problematic PR and re-run)

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/55804

    Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/51278

    mds: "FAILED ceph_assert(!segments.empty())"

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/57205

    Test failure: test_subvolume_group_ls_filter_internal_directories (tasks.cephfs.test_volumes.TestSubvolumeGroups)

* https://tracker.ceph.com/issues/57206

    ceph_test_libcephfs_reclaim crashes during test

* https://tracker.ceph.com/issues/53859

  Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/50223

    client.xxxx isn't responding to mclientcaps(revoke)

h3. 2022 Aug 12

https://pulpito.ceph.com/vshankar-2022-08-10_04:06:00-fs-wip-vshankar-testing-20220805-190751-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2022-08-11_12:16:58-fs-wip-vshankar-testing-20220811-145809-testing-default-smithi/ (drop problematic PR and re-run)

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/55804

    Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/50223

    client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/50821

    qa: untar_snap_rm failure during mds thrashing

* https://tracker.ceph.com/issues/54460

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

h3. 2022 Aug 04

https://pulpito.ceph.com/?branch=wip-vshankar-testing1-20220804-123835 (only mgr/volumes, mgr/stats)

Unrealted teuthology failure on rhel

h3. 2022 Jul 25

http://pulpito.front.sepia.ceph.com/rishabh-2022-07-22_11:34:20-fs-wip-rishabh-testing-2022Jul22-1400-testing-default-smithi/

1st re-run: http://pulpito.front.sepia.ceph.com/rishabh-2022-07-24_03:51:19-fs-wip-rishabh-testing-2022Jul22-1400-testing-default-smithi

2nd re-run: http://pulpito.front.sepia.ceph.com/rishabh-2022-07-24_08:53:36-fs-wip-rishabh-testing-2022Jul22-1400-testing-default-smithi/

3rd re-run: http://pulpito.front.sepia.ceph.com/rishabh-2022-07-24_08:53:36-fs-wip-rishabh-testing-2022Jul22-1400-testing-default-smithi/

4th (final) re-run: http://pulpito.front.sepia.ceph.com/rishabh-2022-07-28_03:59:01-fs-wip-rishabh-testing-2022Jul28-0143-testing-default-smithi/

* https://tracker.ceph.com/issues/55804

  Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/54460

  Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/36593

  Command failed (workunit test fs/quota/quota.sh) on smithixxx with status 1

* https://tracker.ceph.com/issues/54462

  Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status 128~

h3. 2022 July 22

https://pulpito.ceph.com/?branch=wip-pdonnell-testing-20220721.235756

MDS_HEALTH_DUMMY error in log fixed by followup commit.

transient selinux ping failure

* https://tracker.ceph.com/issues/56694

    qa: avoid blocking forever on hung umount

* https://tracker.ceph.com/issues/56695

    [RHEL stock] pjd test failures

* https://tracker.ceph.com/issues/56696

    admin keyring disappears during qa run

* https://tracker.ceph.com/issues/56697

    qa: fs/snaps fails for fuse

* https://tracker.ceph.com/issues/50222

    osd: 5.2s0 deep-scrub : stat mismatch

* https://tracker.ceph.com/issues/56698

    client: FAILED ceph_assert(_size == 0)

* https://tracker.ceph.com/issues/50223

    qa: "client.4737 isn't responding to mclientcaps(revoke)"

h3. 2022 Jul 15

http://pulpito.front.sepia.ceph.com/rishabh-2022-07-08_23:53:34-fs-wip-rishabh-testing-2022Jul08-1820-testing-default-smithi/

re-run: http://pulpito.front.sepia.ceph.com/rishabh-2022-07-15_06:42:04-fs-wip-rishabh-testing-2022Jul08-1820-testing-default-smithi/

* https://tracker.ceph.com/issues/53859

  Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/55804

  Command failed (workunit test suites/pjd.sh)

* https://tracker.ceph.com/issues/50223

  client.xxxx isn't responding to mclientcaps(revoke)

* https://tracker.ceph.com/issues/50222

  osd: deep-scrub : stat mismatch

* https://tracker.ceph.com/issues/56632

  Test failure: test_subvolume_snapshot_clone_quota_exceeded (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)

* https://tracker.ceph.com/issues/56634

  workunit test fs/snaps/snaptest-intodir.sh

* https://tracker.ceph.com/issues/56644

  Test failure: test_rapid_creation (tasks.cephfs.test_fragment.TestFragmentation)

h3. 2022 July 05

http://pulpito.front.sepia.ceph.com/rishabh-2022-07-02_14:14:52-fs-wip-rishabh-testing-20220702-1631-testing-default-smithi/

On 1st re-run some jobs passed - http://pulpito.front.sepia.ceph.com/rishabh-2022-07-03_15:10:28-fs-wip-rishabh-testing-20220702-1631-distro-default-smithi/

On 2nd re-run only few jobs failed -

http://pulpito.front.sepia.ceph.com/rishabh-2022-07-06_05:24:29-fs-wip-rishabh-testing-20220705-2132-distro-default-smithi/

http://pulpito.front.sepia.ceph.com/rishabh-2022-07-06_05:24:29-fs-wip-rishabh-testing-20220705-2132-distro-default-smithi/

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/55804

    Command failed (workunit test suites/pjd.sh) on smithi047 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/

* https://tracker.ceph.com/issues/56445

    Command failed on smithi080 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

* https://tracker.ceph.com/issues/51267

    Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps.sh) on smithi098 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1

* https://tracker.ceph.com/issues/50224

    Test failure: test_mirroring_init_failure_with_recovery (tasks.cephfs.test_mirroring.TestMirroring)

h3. 2022 July 04

https://pulpito.ceph.com/vshankar-2022-06-29_09:19:00-fs-wip-vshankar-testing-20220627-100931-testing-default-smithi/

(rhel runs were borked due to: https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/JSZQFUKVLDND4W33PXDGCABPHNSPT6SS/, tests ran with --filter-out=rhel)

* https://tracker.ceph.com/issues/56445

    Command failed on smithi162 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

* https://tracker.ceph.com/issues/56446

    Test failure: test_client_cache_size (tasks.cephfs.test_client_limits.TestClientLimits)

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

h3. 2022 June 20

https://pulpito.ceph.com/vshankar-2022-06-15_04:03:39-fs-wip-vshankar-testing1-20220615-072516-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2022-06-19_08:22:46-fs-wip-vshankar-testing1-20220619-102531-testing-default-smithi/

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/55804

    qa failure: pjd link tests failed

* https://tracker.ceph.com/issues/54108

    qa: iogen workunit: "The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'}"

* https://tracker.ceph.com/issues/55332

    Failure in snaptest-git-ceph.sh (it's an async unlink/create bug)

h3. 2022 June 13

https://pulpito.ceph.com/pdonnell-2022-06-12_05:08:12-fs:workload-wip-pdonnell-testing-20220612.004943-distro-default-smithi/

* https://tracker.ceph.com/issues/56024

    cephadm: removes ceph.conf during qa run causing command failure

* https://tracker.ceph.com/issues/48773

    qa: scrub does not complete

* https://tracker.ceph.com/issues/56012

    mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay())

h3. 2022 Jun 13

https://pulpito.ceph.com/vshankar-2022-06-07_00:25:50-fs-wip-vshankar-testing-20220606-223254-testing-default-smithi/

https://pulpito.ceph.com/vshankar-2022-06-10_01:04:46-fs-wip-vshankar-testing-20220609-175550-testing-default-smithi/

* https://tracker.ceph.com/issues/52624

    qa: "Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)"

* https://tracker.ceph.com/issues/51964

    qa: test_cephfs_mirror_restart_sync_on_blocklist failure

* https://tracker.ceph.com/issues/53859

    qa: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)

* https://tracker.ceph.com/issues/55804

    qa failure: pjd link tests failed

* https://tracker.ceph.com/issues/56003

    client: src/include/xlist.h: 81: FAILED ceph_assert(_size == 0)

* https://tracker.ceph.com/issues/56011

    fs/thrash: snaptest-snap-rm-cmp.sh fails in mds5sum comparison

* https://tracker.ceph.com/issues/56012

    mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay())

h3. 2022 Jun 07

https://pulpito.ceph.com/vshankar-2022-06-06_21:25:41-fs-wip-vshankar-testing1-20220606-230129-testing-default-smithi/