Project

General

Profile

Activity

From 08/01/2022 to 08/30/2022

08/30/2022

10:00 AM Backport #57274 (In Progress): pacific: mgr/stats: missing clients in perf stats command output.
Neeraj Pratap Singh
09:39 AM Backport #57273 (In Progress): quincy: mgr/stats: missing clients in perf stats command output.
Neeraj Pratap Singh
07:26 AM Backport #57252 (In Progress): pacific: Restore the AT_NO_ATTR_SYNC define in libcephfs
Xiubo Li
02:54 AM Backport #57253 (In Progress): quincy: Restore the AT_NO_ATTR_SYNC define in libcephfs
Xiubo Li
02:31 AM Backport #56055 (Resolved): quincy: crash: uint64_t CephFuse::Handle::fino_snap(uint64_t): assert...
Xiubo Li

08/29/2022

03:13 PM Backport #56055: quincy: crash: uint64_t CephFuse::Handle::fino_snap(uint64_t): assert(stag_snap_...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/46948
merged
Yuri Weinstein
12:49 PM Bug #57280 (Triaged): qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fet...
Venky Shankar
10:39 AM Backport #57330 (In Progress): quincy: Test failure: test_client_metrics_and_metadata (tasks.ceph...
Neeraj Pratap Singh
10:33 AM Backport #57330 (Resolved): quincy: Test failure: test_client_metrics_and_metadata (tasks.cephfs....
https://github.com/ceph/ceph/pull/47820 Backport Bot
10:38 AM Backport #57331 (In Progress): pacific: Test failure: test_client_metrics_and_metadata (tasks.cep...
Neeraj Pratap Singh
10:33 AM Backport #57331 (Resolved): pacific: Test failure: test_client_metrics_and_metadata (tasks.cephfs...
https://github.com/ceph/ceph/pull/47851 Backport Bot
10:21 AM Bug #55861 (Pending Backport): Test failure: test_client_metrics_and_metadata (tasks.cephfs.test_...
Venky Shankar

08/28/2022

09:20 AM Bug #57206: ceph_test_libcephfs_reclaim crashes during test
I"ve used https://github.com/ceph/ceph/blob/main/src/script/ceph-debug-docker.sh for deploying the build on container... Tamar Shacked

08/27/2022

11:16 AM Bug #50840 (Resolved): mds: CephFS kclient gets stuck when getattr() on a certain file
Konstantin Shalygin
11:16 AM Backport #51202 (Resolved): octopus: mds: CephFS kclient gets stuck when getattr() on a certain file
Konstantin Shalygin

08/26/2022

06:29 PM Bug #55825 (Duplicate): cluster [WRN] Health check failed: Degraded data redundancy: 1 pg degrade...
Laura Flores
05:33 PM Backport #57277 (In Progress): pacific: mgr/stats: 'perf stats' command shows incorrect output wi...
Neeraj Pratap Singh
05:30 PM Backport #57279 (In Progress): pacific: mgr/stats: add fs_name as field in perf stats command output
Neeraj Pratap Singh
05:24 PM Backport #57276 (In Progress): quincy: mgr/stats: 'perf stats' command shows incorrect output wit...
Neeraj Pratap Singh
05:21 PM Backport #57278 (In Progress): quincy: mgr/stats: add fs_name as field in perf stats command output
Neeraj Pratap Singh
02:48 PM Bug #57210: NFS client unable to see newly created files when listing directory contents in a FS ...
Also worth noting is that as per https://bugzilla.redhat.com/show_bug.cgi?id=2118263#c11 , this is a regression in th... Ramana Raja
12:36 PM Bug #57210: NFS client unable to see newly created files when listing directory contents in a FS ...
Disabled caching to match the original BZ setup https://bugzilla.redhat.com/show_bug.cgi?id=2118263#c2... Ramana Raja
01:16 PM Bug #56414 (Resolved): mounting subvolume shows size/used bytes for entire fs, not subvolume
Rishabh Dave
09:07 AM Bug #57072 (Fix Under Review): Quincy 17.2.3 pybind/mgr/status: assert metadata failed
Venky Shankar
07:56 AM Feature #56140 (Resolved): cephfs: tooling to identify inode (metadata) corruption
Venky Shankar
06:13 AM Bug #56988: mds: memory leak suspected
I have attached the current "ceph tell mds.<fs_name>:0 perf dump" log files.
Files with the name:
ceph_mds_perf_<...
Ramin Torabi
01:22 AM Bug #53597 (Resolved): mds: FAILED ceph_assert(dir->get_projected_version() == dir->get_version())
Xiubo Li
01:22 AM Backport #55928 (Resolved): quincy: mds: FAILED ceph_assert(dir->get_projected_version() == dir->...
Xiubo Li
01:20 AM Bug #56010 (Resolved): xfstests-dev generic/444 test failed
Xiubo Li
01:20 AM Backport #56464 (Resolved): quincy: xfstests-dev generic/444 test failed
Xiubo Li
01:14 AM Backport #56461 (Resolved): quincy: mds: crash due to seemingly unrecoverable metadata error
Xiubo Li

08/25/2022

10:06 PM Bug #57210: NFS client unable to see newly created files when listing directory contents in a FS ...
Greg Farnum wrote:
> I wonder if the real difference here is not the cloned subvolume, but whether the mount point h...
Ramana Raja
08:32 PM Backport #56461: quincy: mds: crash due to seemingly unrecoverable metadata error
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47432
merged
Yuri Weinstein
08:29 PM Backport #56464: quincy: xfstests-dev generic/444 test failed
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47058
merged
Yuri Weinstein
07:21 PM Bug #57280: qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package...
Created a new Tracker for Cephadm, but marked a relation to this tracker. Laura Flores
07:12 PM Bug #57280: qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package...
Several occurrences with cephadm tests on this run, such as:
/a/yuriw-2022-08-17_19:34:54-rados-wip-yuri7-testing-...
Laura Flores
05:53 PM Bug #57285 (Won't Fix): mds: removing data pool doesn't dissociate the pool from the cephfs appli...
https://github.com/ceph/ceph/pull/47801#issuecomment-1227520097 Milind Changire
05:38 PM Bug #57299 (Fix Under Review): qa: test_dump_loads fails with JSONDecodeError
Rishabh Dave
05:25 PM Bug #57299 (Pending Backport): qa: test_dump_loads fails with JSONDecodeError
@tasks.cephfs.test_admin.TestAdminCommandDumpLoads.test_dump_loads@ fails with @JSONDecodeError@. Traceback -
<pre...
Rishabh Dave
03:17 PM Backport #55928: quincy: mds: FAILED ceph_assert(dir->get_projected_version() == dir->get_version())
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47181
merged
Yuri Weinstein
03:14 PM Backport #57057: quincy: mgr/volumes: Handle internal metadata directories under '/volumes' prope...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47511
merged
Yuri Weinstein
01:11 PM Fix #57295 (Fix Under Review): qa: remove RHEL from job matrix
Patrick Donnelly
01:08 PM Fix #57295 (Rejected): qa: remove RHEL from job matrix
The last few months it has become a frequent source of avoidable problems. Patrick Donnelly
12:30 PM Feature #55463 (Duplicate): cephfs-top: allow users to chose sorting order
Marking this as duplicate of https://tracker.ceph.com/issues/55121 (that tracker has more context/discussions). Venky Shankar
12:30 PM Feature #55121: cephfs-top: new options to limit and order-by
Neeraj is working on this. Venky Shankar
05:23 AM Feature #55197 (In Progress): cephfs-top: make cephfs-top display scrollable like top
Venky Shankar wrote:
> Jos, I presume that you are working on this feature?
Yes.
Jos Collin
05:19 AM Feature #55197: cephfs-top: make cephfs-top display scrollable like top
Jos, I presume that you are working on this feature? Venky Shankar

08/24/2022

01:55 PM Fix #51177: pybind/mgr/volumes: investigate moving calls which may block on libcephfs into anothe...
Spoke to Kotersh today - we may want to introduce an async command execution interface in plugins that the finisher t... Venky Shankar
11:46 AM Bug #57285 (Won't Fix): mds: removing data pool doesn't dissociate the pool from the cephfs appli...
... Milind Changire
09:04 AM Bug #57072: Quincy 17.2.3 pybind/mgr/status: assert metadata failed
cld slnt wrote:
> Venky Shankar wrote:
> > Could you provide more info on how you ran into this. A reproducer would...
Venky Shankar
08:34 AM Bug #57072: Quincy 17.2.3 pybind/mgr/status: assert metadata failed
Venky Shankar wrote:
> Could you provide more info on how you ran into this. A reproducer would really help (or at l...
cld slnt
08:09 AM Bug #57280: qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fetch package...
It's ubuntu 22.04 Jammy JellyFish. The 'sha1' for client.0 is 'None', hence the failure. The client.0 is 'ceph-fuse'... Kotresh Hiremath Ravishankar
07:09 AM Bug #57280 (Resolved): qa: tasks/kernel_cfuse_workunits_untarbuild_blogbench fails - Failed to fe...
Seen in multiple runs:
https://pulpito.ceph.com/yuriw-2022-08-22_21:15:31-fs-wip-yuri7-testing-2022-08-17-0943-quinc...
Kotresh Hiremath Ravishankar
08:09 AM Backport #57283 (Resolved): quincy: cephfs-top:addition of filesystem menu(improving GUI)
https://github.com/ceph/ceph/pull/47820 Backport Bot
08:09 AM Backport #57282 (Resolved): pacific: cephfs-top:addition of filesystem menu(improving GUI)
https://github.com/ceph/ceph/pull/47998 Backport Bot
08:09 AM Backport #57281 (In Progress): quincy: mds/MDBalancer: add an arg to limit depth when dump loads ...
https://github.com/ceph/ceph/pull/54188 Backport Bot
08:08 AM Feature #54978 (Pending Backport): cephfs-top:addition of filesystem menu(improving GUI)
Venky Shankar
08:01 AM Feature #56058 (Pending Backport): mds/MDBalancer: add an arg to limit depth when dump loads for ...
Venky Shankar
05:54 AM Backport #57279 (Resolved): pacific: mgr/stats: add fs_name as field in perf stats command output
https://github.com/ceph/ceph/pull/47851 Backport Bot
05:54 AM Backport #57278 (Resolved): quincy: mgr/stats: add fs_name as field in perf stats command output
https://github.com/ceph/ceph/pull/47820 Backport Bot
05:54 AM Backport #57277 (Resolved): pacific: mgr/stats: 'perf stats' command shows incorrect output with ...
https://github.com/ceph/ceph/pull/47851 Backport Bot
05:54 AM Backport #57276 (Resolved): quincy: mgr/stats: 'perf stats' command shows incorrect output with n...
https://github.com/ceph/ceph/pull/47820 Backport Bot
05:44 AM Bug #56169 (Pending Backport): mgr/stats: 'perf stats' command shows incorrect output with non-ex...
Venky Shankar
05:43 AM Bug #56162 (Pending Backport): mgr/stats: add fs_name as field in perf stats command output
Venky Shankar
04:51 AM Backport #57274 (Resolved): pacific: mgr/stats: missing clients in perf stats command output.
https://github.com/ceph/ceph/pull/47866 Backport Bot
04:51 AM Backport #57273 (Resolved): quincy: mgr/stats: missing clients in perf stats command output.
https://github.com/ceph/ceph/pull/47820 Backport Bot
04:20 AM Bug #56483 (Pending Backport): mgr/stats: missing clients in perf stats command output.
Venky Shankar
01:11 AM Bug #56988 (Need More Info): mds: memory leak suspected
Patrick Donnelly
01:11 AM Bug #56988: mds: memory leak suspected
If you could capture... Patrick Donnelly
12:49 AM Backport #56465 (Resolved): pacific: xfstests-dev generic/444 test failed
Xiubo Li
12:49 AM Backport #55929 (Resolved): pacific: mds: FAILED ceph_assert(dir->get_projected_version() == dir-...
Xiubo Li

08/23/2022

06:57 PM Backport #57263 (In Progress): pacific: pybind/mgr/volumes: add basic introspection
Neeraj Pratap Singh
04:07 PM Backport #57263 (Resolved): pacific: pybind/mgr/volumes: add basic introspection
https://github.com/ceph/ceph/pull/47769 Backport Bot
06:17 PM Backport #57264 (In Progress): quincy: pybind/mgr/volumes: add basic introspection
Neeraj Pratap Singh
04:07 PM Backport #57264 (Resolved): quincy: pybind/mgr/volumes: add basic introspection
https://github.com/ceph/ceph/pull/47768 Backport Bot
05:49 PM Bug #57249 (Fix Under Review): mds: damage table only stores one dentry per dirfrag
Patrick Donnelly
12:43 PM Bug #57249 (Resolved): mds: damage table only stores one dentry per dirfrag
Some copy paste in
https://github.com/ceph/ceph/commit/c9cfaef104e9aaefad55583d7e54f8b4665904b3
caused DamageTa...
Patrick Donnelly
04:07 PM Backport #57262 (Rejected): octopus: standby-replay mds is removed from MDSMap unexpectedly
Backport Bot
04:07 PM Backport #57261 (Resolved): pacific: standby-replay mds is removed from MDSMap unexpectedly
https://github.com/ceph/ceph/pull/48270 Backport Bot
04:03 PM Feature #51434 (Pending Backport): pybind/mgr/volumes: add basic introspection
Venky Shankar
04:00 PM Bug #53811 (Pending Backport): standby-replay mds is removed from MDSMap unexpectedly
Venky Shankar
03:50 PM Bug #57154: kernel/fuse client using ceph ID with uid restricted MDS caps cannot update caps
I think we need to look at session->check_access() call in Locker::_do_cap_update() . During cap update, does the MDS... Ramana Raja
02:27 PM Backport #55929: pacific: mds: FAILED ceph_assert(dir->get_projected_version() == dir->get_versio...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47180
merged
Yuri Weinstein
02:26 PM Backport #56465: pacific: xfstests-dev generic/444 test failed
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47059
merged
Yuri Weinstein
02:11 PM Backport #57253 (Resolved): quincy: Restore the AT_NO_ATTR_SYNC define in libcephfs
https://github.com/ceph/ceph/pull/47861 Backport Bot
02:11 PM Backport #57252 (Resolved): pacific: Restore the AT_NO_ATTR_SYNC define in libcephfs
https://github.com/ceph/ceph/pull/47862 Backport Bot
02:09 PM Bug #56638 (Pending Backport): Restore the AT_NO_ATTR_SYNC define in libcephfs
Venky Shankar
12:20 PM Bug #57248 (Fix Under Review): qa: mirror tests should cleanup fs during unwind
Patrick Donnelly
12:10 PM Bug #57248 (Resolved): qa: mirror tests should cleanup fs during unwind
https://pulpito.ceph.com/vshankar-2022-08-18_04:30:42-fs-wip-vshankar-testing1-20220818-082047-testing-default-smithi... Patrick Donnelly
09:16 AM Bug #57244 (Fix Under Review): [WRN] : client.408214273 isn't responding to mclientcaps(revoke), ...
Xiubo Li
08:32 AM Bug #57244 (Resolved): [WRN] : client.408214273 isn't responding to mclientcaps(revoke), ino 0x10...
This is from cu case and there is no more debug logs. It's so strange that the MDS is waiting for the revoking respon... Xiubo Li
08:33 AM Bug #49434 (Duplicate): `client isn't responding to mclientcaps(revoke)` for hours
Xiubo Li
08:00 AM Backport #57242 (Resolved): quincy: mgr/volumes: Clone operations are failing with Assertion Error
https://github.com/ceph/ceph/pull/47894 Backport Bot
08:00 AM Backport #57241 (Resolved): pacific: mgr/volumes: Clone operations are failing with Assertion Error
https://github.com/ceph/ceph/pull/47112 Backport Bot
07:55 AM Bug #55976 (Pending Backport): mgr/volumes: Clone operations are failing with Assertion Error
Venky Shankar
07:52 AM Backport #57240 (Resolved): quincy: ceph-fs crashes on getfattr
https://github.com/ceph/ceph/pull/47890 Backport Bot
07:52 AM Backport #57239 (Resolved): pacific: ceph-fs crashes on getfattr
https://github.com/ceph/ceph/pull/47891 Backport Bot
07:52 AM Bug #56529 (Pending Backport): ceph-fs crashes on getfattr
Venky Shankar
07:14 AM Bug #57044: mds: add some debug logs for "crash during construction of internal request"
Debug PR merged. Waiting for logs in test runs... Venky Shankar
07:09 AM Bug #57205: Test failure: test_subvolume_group_ls_filter_internal_directories (tasks.cephfs.test_...
Nikhil, PTAL. Venky Shankar
07:07 AM Bug #57204: MDLog.h: 99: FAILED ceph_assert(!segments.empty())
Patrick Donnelly wrote:
> ?
Yeh. Thx!
Venky Shankar
04:56 AM Backport #57113 (In Progress): pacific: Intermittent ParsingError failure in mgr/volumes module ...
Kotresh Hiremath Ravishankar
04:50 AM Backport #57112 (In Progress): quincy: Intermittent ParsingError failure in mgr/volumes module d...
Kotresh Hiremath Ravishankar

08/22/2022

10:25 PM Bug #47643: mds: Segmentation fault in thread 7fcff3078700 thread_name:md_log_replay
I've ran into a very similar issue to this one on @17.2.0@. I tried to upgrade MDS to @17.2.3 (dff484dfc9e19a9819f375... Tomasz Kuzemko
08:42 PM Backport #56526: quincy: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay())
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47110
merged
Yuri Weinstein
06:57 PM Backport #55385 (In Progress): quincy: mgr/snap_schedule: include timezone information in schedul...
Neeraj Pratap Singh
06:49 PM Backport #57200 (In Progress): quincy: snap_schedule: replace .snap with the client configured sn...
Neeraj Pratap Singh
03:01 PM Documentation #57062: Document access patterns that have good/pathological performance on CephFS
Niklas Hambuechen wrote:
> Hi Venky, I'm using the kclient on Linux 5.10.88 in this cluster.
Thanks, Niklas. I'll...
Venky Shankar
01:34 PM Documentation #57062: Document access patterns that have good/pathological performance on CephFS
Hi Venky, I'm using the kclient on Linux 5.10.88 in this cluster. Niklas Hambuechen
01:22 PM Documentation #57062: Document access patterns that have good/pathological performance on CephFS
Hi Niklas,
Do you see this behavior with user-space and kclient?
Venky Shankar
01:36 PM Feature #56643: scrub: add one subcommand or option to add the missing objects back
Venky Shankar wrote:
> Xiubo Li wrote:
> > Venky Shankar wrote:
> > > Xiubo Li wrote:
> > > > When we are scrub r...
Xiubo Li
01:16 PM Feature #56643: scrub: add one subcommand or option to add the missing objects back
Xiubo Li wrote:
> Venky Shankar wrote:
> > Xiubo Li wrote:
> > > When we are scrub repairing the metadatas and som...
Venky Shankar
01:32 PM Bug #56634: qa: workunit snaptest-intodir.sh fails with MDS crash
Leaving this unassigned for now. Please bring this up if we hit this again. Venky Shankar
01:21 PM Bug #57204 (Duplicate): MDLog.h: 99: FAILED ceph_assert(!segments.empty())
? Patrick Donnelly
12:51 PM Bug #57204 (Triaged): MDLog.h: 99: FAILED ceph_assert(!segments.empty())
Venky Shankar
01:18 PM Bug #57072: Quincy 17.2.3 pybind/mgr/status: assert metadata failed
Could you provide more info on how you ran into this. A reproducer would really help (or at least the debug mgr logs). Venky Shankar
01:07 PM Bug #56261 (Triaged): crash: Migrator::import_notify_abort(CDir*, std::set<CDir*, std::less<CDir*...
Venky Shankar
01:06 PM Bug #56288 (Triaged): crash: Client::_readdir_cache_cb(dir_result_t*, int (*)(void*, dirent*, cep...
Venky Shankar
01:05 PM Bug #56697: qa: fs/snaps fails for fuse
Seen in this quincy run as well https://pulpito.ceph.com/yuriw-2022-08-17_18:46:04-fs-wip-yuri7-testing-2022-08-17-09... Kotresh Hiremath Ravishankar
01:05 PM Bug #56592 (Triaged): mds: crash when mounting a client during the scrub repair is going on
Venky Shankar
01:02 PM Bug #56644 (Triaged): qa: test_rapid_creation fails with "No space left on device"
Venky Shankar
09:52 AM Bug #56644: qa: test_rapid_creation fails with "No space left on device"
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-default-smith... Kotresh Hiremath Ravishankar
01:01 PM Bug #56698 (Triaged): client: FAILED ceph_assert(_size == 0)
Venky Shankar
01:00 PM Bug #57065 (Triaged): qa: test_query_client_ip_filter fails with latest 'perf stats' structure ch...
Venky Shankar
01:00 PM Bug #56988 (Triaged): mds: memory leak suspected
Venky Shankar
12:57 PM Bug #57084 (Triaged): Permissions of the .snap directory do not inherit ACLs
Venky Shankar
12:55 PM Bug #57087 (Triaged): qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) fa...
Venky Shankar
12:53 PM Bug #57154 (Triaged): kernel/fuse client using ceph ID with uid restricted MDS caps cannot update...
Venky Shankar
12:47 PM Bug #57206 (Triaged): ceph_test_libcephfs_reclaim crashes during test
Venky Shankar
12:45 PM Bug #57210 (Triaged): NFS client unable to see newly created files when listing directory content...
Venky Shankar
12:43 PM Bug #57218 (Triaged): qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
Venky Shankar
09:22 AM Bug #57218 (Triaged): qa: tasks/{1-thrash/mds 2-workunit/cfuse_workunit_suites_fsstress}} fails
Seen in https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-defau... Kotresh Hiremath Ravishankar
11:01 AM Backport #57201 (In Progress): pacific: snap_schedule: replace .snap with the client configured s...
Neeraj Pratap Singh
10:57 AM Bug #46609: mds: CDir.cc: 956: FAILED ceph_assert(auth_pins == 0)
Seen in recent pacific run: https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-... Kotresh Hiremath Ravishankar
10:09 AM Bug #57219 (Duplicate): qa: mds crashed while running workunit test fs/misc/dirfrag.sh
This should be a know issue, please see https://tracker.ceph.com/issues/46609. Xiubo Li
09:37 AM Bug #57219 (Duplicate): qa: mds crashed while running workunit test fs/misc/dirfrag.sh
Seen in https://pulpito.ceph.com/yuriw-2022-08-19_21:01:11-fs-wip-yuri10-testing-2022-08-18-1400-pacific-distro-defau... Kotresh Hiremath Ravishankar
09:51 AM Bug #51964: qa: test_cephfs_mirror_restart_sync_on_blocklist failure
Seen in this pacific run: https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pa... Kotresh Hiremath Ravishankar
09:51 AM Bug #48773: qa: scrub does not complete
Recent occurrence:
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacifi...
Kotresh Hiremath Ravishankar
09:49 AM Bug #51267: CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps...
Recent occurence:
https://pulpito.ceph.com/yuriw-2022-08-18_23:16:33-fs-wip-yuri10-testing-2022-08-18-1400-pacific...
Kotresh Hiremath Ravishankar
09:17 AM Backport #56112 (Resolved): pacific: Test failure: test_flush (tasks.cephfs.test_readahead.TestRe...
Dhairya Parmar
09:17 AM Backport #56111 (Resolved): quincy: Test failure: test_flush (tasks.cephfs.test_readahead.TestRea...
Dhairya Parmar
09:17 AM Bug #55538 (Resolved): Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
Dhairya Parmar
08:55 AM Cleanup #4744 (Fix Under Review): mds: pass around LogSegments via std::shared_ptr
PR: https://github.com/ceph/ceph/pull/47598 Tamar Shacked
06:19 AM Backport #57042 (Resolved): quincy: pybind/mgr/volumes: interface to check the presence of subvol...
Neeraj Pratap Singh

08/20/2022

01:33 AM Bug #54463 (Resolved): mds: flush mdlog if locked and still has wanted caps not satisfied
Xiubo Li
01:33 AM Backport #55756 (Resolved): quincy: mds: flush mdlog if locked and still has wanted caps not sati...
Xiubo Li
01:33 AM Cleanup #54362 (Resolved): client: do not release the global snaprealm until unmounting
Xiubo Li
01:32 AM Backport #55736 (Resolved): quincy: client: do not release the global snaprealm until unmounting
Xiubo Li

08/19/2022

11:00 PM Bug #57210: NFS client unable to see newly created files when listing directory contents in a FS ...
I wonder if the real difference here is not the cloned subvolume, but whether the mount point had files in it prior t... Greg Farnum
10:36 PM Bug #57210 (Resolved): NFS client unable to see newly created files when listing directory conten...
Tried the following in a vstart cluster on ceph-main that launches ganesha v3.5 containers... Ramana Raja
05:59 PM Bug #57206 (Rejected): ceph_test_libcephfs_reclaim crashes during test
/a/vshankar-2022-08-18_04:30:42-fs-wip-vshankar-testing1-20220818-082047-testing-default-smithi/6978421
Core is at...
Venky Shankar
05:51 PM Bug #57205 (Pending Backport): Test failure: test_subvolume_group_ls_filter_internal_directories ...
/a/vshankar-2022-08-18_04:30:42-fs-wip-vshankar-testing1-20220818-082047-testing-default-smithi/6978395... Venky Shankar
05:50 PM Bug #57204 (Duplicate): MDLog.h: 99: FAILED ceph_assert(!segments.empty())
/a/vshankar-2022-08-18_04:30:42-fs-wip-vshankar-testing1-20220818-082047-testing-default-smithi/6978343
MDS crashe...
Venky Shankar
04:41 PM Backport #55756: quincy: mds: flush mdlog if locked and still has wanted caps not satisfied
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/46494
merged
Yuri Weinstein
02:58 PM Backport #56111: quincy: Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
Dhairya Parmar wrote:
> https://github.com/ceph/ceph/pull/46899
merged
Yuri Weinstein
02:53 PM Backport #55736: quincy: client: do not release the global snaprealm until unmounting
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/46495
merged
Yuri Weinstein
01:28 PM Backport #57201 (Resolved): pacific: snap_schedule: replace .snap with the client configured snap...
https://github.com/ceph/ceph/pull/47726 Backport Bot
01:28 PM Backport #57200 (Resolved): quincy: snap_schedule: replace .snap with the client configured snap ...
https://github.com/ceph/ceph/pull/47734 Backport Bot
01:14 PM Bug #54283 (Resolved): qa/cephfs: is_mounted() depends on a mutable variable
Rishabh Dave
01:14 PM Bug #55234 (Pending Backport): snap_schedule: replace .snap with the client configured snap dir name
Rishabh Dave
01:10 PM Backport #57194 (Resolved): pacific: ceph pacific fails to perform fs/mirror test
https://github.com/ceph/ceph/pull/48269 Backport Bot
01:09 PM Backport #57193 (Resolved): quincy: ceph pacific fails to perform fs/mirror test
https://github.com/ceph/ceph/pull/48268 Backport Bot
01:03 PM Bug #55134 (Pending Backport): ceph pacific fails to perform fs/mirror test
Rishabh Dave

08/18/2022

04:24 PM Bug #53192: High cephfs MDS latency and CPU load with snapshots and unlink operations
This topic was discussed during the User + Dev meeting on Aug. 8th, 2022. One revelation that came of the meeting (as... Laura Flores
01:12 PM Bug #51964: qa: test_cephfs_mirror_restart_sync_on_blocklist failure
Seen during weekly QA run - http://pulpito.front.sepia.ceph.com/rishabh-2022-07-24_08:53:36-fs-wip-rishabh-testing-20... Rishabh Dave
09:01 AM Bug #53724 (Fix Under Review): mds: stray directories are not purged when all past parents are clear
Xiubo Li

08/17/2022

01:09 PM Feature #16745: mon: prevent allocating snapids allocated for CephFS
Greg, You mean to disable taking snaps for a pool if its in use with CephFS? Venky Shankar
08:46 AM Backport #57156 (In Progress): quincy: cephfs-top: wrong/infinitely changing wsp values
Jos Collin
05:47 AM Backport #57156 (Resolved): quincy: cephfs-top: wrong/infinitely changing wsp values
https://github.com/ceph/ceph/pull/47648 Backport Bot
08:42 AM Backport #57155 (In Progress): pacific: cephfs-top: wrong/infinitely changing wsp values
Jos Collin
05:47 AM Backport #57155 (Resolved): pacific: cephfs-top: wrong/infinitely changing wsp values
https://github.com/ceph/ceph/pull/47647 Backport Bot
07:08 AM Backport #57158 (Resolved): quincy: doc: update snap-schedule notes regarding 'start' time
https://github.com/ceph/ceph/pull/53577 Backport Bot
07:08 AM Backport #57157 (Resolved): pacific: doc: update snap-schedule notes regarding 'start' time
https://github.com/ceph/ceph/pull/53576 Backport Bot
07:06 AM Documentation #56730 (Pending Backport): doc: update snap-schedule notes regarding 'start' time
Venky Shankar
05:44 AM Bug #56537 (Pending Backport): cephfs-top: wrong/infinitely changing wsp values
Venky Shankar

08/16/2022

08:53 PM Bug #57154: kernel/fuse client using ceph ID with uid restricted MDS caps cannot update caps
This issue was first described in https://tracker.ceph.com/issues/56067#note-15 Ramana Raja
08:47 PM Bug #57154 (Pending Backport): kernel/fuse client using ceph ID with uid restricted MDS caps cann...
A kclient sends cap updates as caller_uid:caller_gid 0:0. A fuse client sends cap updates as caller_uid:caller_gid -1... Ramana Raja
08:51 PM Bug #56067: Cephfs data loss with root_squash enabled
Created a separate tracker ticket for the cap updates being dropped for clients using ceph IDs with uid restricted MD... Ramana Raja
02:41 PM Backport #57042: quincy: pybind/mgr/volumes: interface to check the presence of subvolumegroups/s...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47474
merged
Yuri Weinstein
02:35 PM Backport #56112: pacific: Test failure: test_flush (tasks.cephfs.test_readahead.TestReadahead)
Dhairya Parmar wrote:
> Venky Shankar wrote:
> > Dhairya, please do the backport.
>
> https://github.com/ceph/ce...
Yuri Weinstein
02:31 PM Bug #56666: mds: standby-replay daemon always removed in MDSMonitor::prepare_beacon
https://github.com/ceph/ceph/pull/47281 merged Yuri Weinstein
02:07 PM Backport #56590: quincy: qa: iogen workunit: "The following counters failed to be set on mds daem...
Ramana, please post the backport. Venky Shankar
02:06 PM Backport #56541: quincy: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = Sn...
Milind, please take this one. Venky Shankar
02:06 PM Bug #56269: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
Venky Shankar wrote:
> Milind, please take this one.
Sorry - I meant to update the backport tracker.
Venky Shankar
02:04 PM Bug #56269: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
Milind, please take this one. Venky Shankar
02:05 PM Backport #56542: pacific: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = S...
Milind, please take this one. Venky Shankar
01:46 PM Bug #50224: qa: test_mirroring_init_failure_with_recovery failure
This is seen recently in https://pulpito.ceph.com/yuriw-2022-08-11_16:57:01-fs-wip-yuri3-testing-2022-08-11-0809-paci... Kotresh Hiremath Ravishankar
04:20 AM Bug #56249 (Fix Under Review): crash: int Client::_do_remount(bool): abort
Xiubo Li

08/15/2022

03:21 PM Backport #56978: pacific: mgr/volumes: Subvolume creation failed on FIPs enabled system
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47369
merged
Yuri Weinstein
12:27 PM Bug #50834 (Resolved): MDS heartbeat timed out between during executing MDCache::start_files_to_r...
Konstantin Shalygin
12:27 PM Backport #50914 (Resolved): octopus: MDS heartbeat timed out between during executing MDCache::st...
Konstantin Shalygin
12:26 PM Bug #52123 (Resolved): mds sends cap updates with btime zeroed out
Konstantin Shalygin
12:26 PM Backport #52635 (Resolved): pacific: mds sends cap updates with btime zeroed out
Konstantin Shalygin
12:26 PM Backport #52634 (Resolved): octopus: mds sends cap updates with btime zeroed out
Konstantin Shalygin
12:26 PM Bug #48422 (Resolved): mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.count(mds->get_n...
Konstantin Shalygin
12:26 PM Backport #51933 (Resolved): octopus: mds: MDCache.cc:5319 FAILED ceph_assert(rejoin_ack_gather.co...
Konstantin Shalygin
12:25 PM Bug #48231 (Resolved): qa: test_subvolume_clone_in_progress_snapshot_rm is racy
Konstantin Shalygin
12:25 PM Backport #51201 (Resolved): octopus: qa: test_subvolume_clone_in_progress_snapshot_rm is racy
Konstantin Shalygin
12:23 PM Bug #41072 (Resolved): scheduled cephfs snapshots (via ceph manager)
Konstantin Shalygin
12:23 PM Backport #47200 (Rejected): octopus: scheduled cephfs snapshots (via ceph manager)
Konstantin Shalygin
12:22 PM Bug #53952 (Resolved): mds: mds_oft_prefetch_dirfrags default to false
Konstantin Shalygin
12:21 PM Backport #54194 (Resolved): pacific: mds: mds_oft_prefetch_dirfrags default to false
Konstantin Shalygin
12:21 PM Backport #54196 (Resolved): quincy: mds: mds_oft_prefetch_dirfrags default to false
Konstantin Shalygin
12:21 PM Backport #54195 (Resolved): octopus: mds: mds_oft_prefetch_dirfrags default to false
Konstantin Shalygin
12:21 PM Bug #53805 (Resolved): mds: seg fault in expire_recursive
Konstantin Shalygin
12:21 PM Backport #54407 (Resolved): quincy: mds: seg fault in expire_recursive
Konstantin Shalygin
12:20 PM Backport #54220 (Resolved): pacific: mds: seg fault in expire_recursive
Konstantin Shalygin
12:20 PM Backport #54219 (Resolved): octopus: mds: seg fault in expire_recursive
Konstantin Shalygin
09:35 AM Bug #56249 (In Progress): crash: int Client::_do_remount(bool): abort
Xiubo Li
09:35 AM Bug #56249: crash: int Client::_do_remount(bool): abort
Went through the kernel code I couldn't find any case in our case could cause the failure.
And from https://tracke...
Xiubo Li
07:05 AM Bug #56249: crash: int Client::_do_remount(bool): abort
Xiubo Li wrote:
> This only exist in the **v17.1.0** and the logic has been changed after [1][2][3] below. When tryi...
Xiubo Li
06:16 AM Bug #56249: crash: int Client::_do_remount(bool): abort
This only exist in the **v17.1.0** and the logic has been changed after [1][2][3] below. When trying remount to inval... Xiubo Li
06:04 AM Bug #56249: crash: int Client::_do_remount(bool): abort
Venky,
Please check this one https://tracker.ceph.com/issues/56532. It should be the same bug with this one.
Xiubo Li
07:18 AM Bug #57126 (Fix Under Review): client: abort the client daemons when we couldn't invalidate the d...
Xiubo Li
07:12 AM Bug #57126 (Resolved): client: abort the client daemons when we couldn't invalidate the dentry ca...
It was introduced by https://tracker.ceph.com/issues/54049.
From the options:...
Xiubo Li
05:54 AM Bug #54653: crash: uint64_t CephFuse::Handle::fino_snap(uint64_t): assert(stag_snap_map.count(stag))
I found one case could cause this, such as in the xfstests-dev's open_by_handle.c, which will use the name_to_handle_... Xiubo Li
05:53 AM Bug #56380: crash: Client::_get_vino(Inode*)
I found one case could cause this, such as in the xfstests-dev's open_by_handle.c, which will use the name_to_handle_... Xiubo Li
05:52 AM Bug #56774: crash: Client::_get_vino(Inode*)
I found one case could cause this, such as in the xfstests-dev's open_by_handle.c, which will use the name_to_handle_... Xiubo Li
05:18 AM Bug #56774 (Duplicate): crash: Client::_get_vino(Inode*)
Xiubo Li
05:52 AM Bug #56263: crash: Client::_get_vino(Inode*)
I found one case could cause this, such as in the xfstests-dev's open_by_handle.c, which will use the name_to_handle_... Xiubo Li

08/12/2022

05:22 PM Documentation #57062: Document access patterns that have good/pathological performance on CephFS
I think that a good place for this info to be added would be https://docs.ceph.com/en/quincy/cephfs/app-best-practice... Niklas Hambuechen
12:03 PM Documentation #57115 (New): Explanation for cache pressure
Following up on the "thread":https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/DOUQNI5YQ74YB3FS5ZOQI2MS... Eugen Block
09:54 AM Bug #56996: Transient data read corruption from other machine
Confirmed with Venky, when the **CInode::filelock** is in **LOCK_MIX** state we won't guarantee the data consistency ... Xiubo Li
09:31 AM Feature #40633 (Resolved): mds: dump recent log events for extraordinary events
Venky Shankar
09:23 AM Backport #57113 (Resolved): pacific: Intermittent ParsingError failure in mgr/volumes module dur...
https://github.com/ceph/ceph/pull/47112 Backport Bot
09:23 AM Backport #57112 (In Progress): quincy: Intermittent ParsingError failure in mgr/volumes module d...
https://github.com/ceph/ceph/pull/47747 Backport Bot
09:12 AM Bug #55583 (Pending Backport): Intermittent ParsingError failure in mgr/volumes module during "c...
Venky Shankar
09:11 AM Backport #57111 (Resolved): quincy: mds: handle deferred client request core when mds reboot
https://github.com/ceph/ceph/pull/53363 Backport Bot
09:11 AM Backport #57110 (Resolved): pacific: mds: handle deferred client request core when mds reboot
https://github.com/ceph/ceph/pull/53362 Backport Bot
09:10 AM Bug #56116 (Pending Backport): mds: handle deferred client request core when mds reboot
Venky Shankar
05:54 AM Bug #54460: snaptest-multiple-capsnaps.sh test failure
Milind Changire wrote:
> client-type: fuse
>
> * Iteratively running shell scripts under *qa/workunits/fs/snaps/*...
Venky Shankar

08/11/2022

04:08 PM Bug #57048: osdc/Journaler: better handle ENOENT during replay as up:standby-replay
Greg Farnum wrote:
> Patrick Donnelly wrote:
> > Venky Shankar wrote:
> > > Patrick,
> > >
> > > Do you mean a ...
Patrick Donnelly
03:12 PM Bug #57048: osdc/Journaler: better handle ENOENT during replay as up:standby-replay
Patrick Donnelly wrote:
> Venky Shankar wrote:
> > Patrick,
> >
> > Do you mean a standby-replay MDS should tole...
Greg Farnum
01:37 PM Bug #57048: osdc/Journaler: better handle ENOENT during replay as up:standby-replay
Venky Shankar wrote:
> Patrick,
>
> Do you mean a standby-replay MDS should tolerate missing journal objects?
...
Patrick Donnelly
01:40 PM Backport #51337 (Rejected): nautilus: mds: avoid journaling overhead for setxattr("ceph.dir.subvo...
Patrick Donnelly
06:56 AM Bug #57087: qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure
Note that the test successfully passed on the re-run
https://pulpito.ceph.com/yuriw-2022-08-10_20:34:29-fs-wip-yuri6...
Kotresh Hiremath Ravishankar
03:44 AM Bug #54253: Avoid OOM exceeding 10x MDS cache limit on restart after many files were opened
Unfortunately I must report that I'm still hitting this issue even with Ceph 16.2.7 and... Niklas Hambuechen

08/10/2022

05:30 PM Feature #56140 (Fix Under Review): cephfs: tooling to identify inode (metadata) corruption
Patrick Donnelly
05:20 PM Feature #57091 (Resolved): mds: modify scrub to catch dentry corruption
Such as "first" snapshot being an invalid value. Patrick Donnelly
05:01 PM Feature #57090 (Resolved): MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
During some recovery situations, it would be useful to have MDS up but prevent clients from establishing sessions. Us... Patrick Donnelly
03:18 PM Backport #56979: quincy: mgr/volumes: Subvolume creation failed on FIPs enabled system
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47368
mergedReviewed-by: Ramana Raja <rraja@redhat.com>
Yuri Weinstein
02:34 PM Bug #55216 (Resolved): cephfs-shell: creates directories in local file system even if file not found
PR along with backport PRs merged. Marking as resolved. Dhairya Parmar
02:30 PM Backport #55627 (Resolved): pacific: cephfs-shell: creates directories in local file system even ...
merged Dhairya Parmar
01:59 PM Feature #55715 (Resolved): pybind/mgr/cephadm/upgrade: allow upgrades without reducing max_mds
Adam King
11:13 AM Bug #54271: mds/OpenFileTable.cc: 777: FAILED ceph_assert(omap_num_objs == num_objs)
We will wait for this to happen in recent versions. Kotresh Hiremath Ravishankar
11:11 AM Bug #54271: mds/OpenFileTable.cc: 777: FAILED ceph_assert(omap_num_objs == num_objs)
Lowering the priority as this is seen only in nautilus and not seen in supported versions. Kotresh Hiremath Ravishankar
10:48 AM Bug #56644: qa: test_rapid_creation fails with "No space left on device"
https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-default-smithi... Kotresh Hiremath Ravishankar
10:35 AM Bug #57087 (Pending Backport): qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDat...
Seen in https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-defaul... Kotresh Hiremath Ravishankar
10:01 AM Bug #51276 (Resolved): mds: avoid journaling overhead for setxattr("ceph.dir.subvolume") for no-o...
Konstantin Shalygin
10:00 AM Backport #51337 (Resolved): nautilus: mds: avoid journaling overhead for setxattr("ceph.dir.subvo...
Nautilus is EOL Konstantin Shalygin
09:49 AM Bug #51267: CommandFailedError: Command failed (workunit test fs/snaps/snaptest-multiple-capsnaps...
Seen in https://pulpito.ceph.com/yuriw-2022-08-04_20:54:08-fs-wip-yuri6-testing-2022-08-04-0617-pacific-distro-defaul... Kotresh Hiremath Ravishankar
08:18 AM Bug #57083 (Fix Under Review): ceph-fuse: monclient(hunting): handle_auth_bad_method server allow...
Xiubo Li
07:56 AM Bug #57083: ceph-fuse: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but ...
The **nautilus** is using **python2**, while the **pacific** qa suite is using **python3** and the qa test suite seem... Xiubo Li
07:37 AM Bug #57083: ceph-fuse: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but ...
From **remote/smithi029/log/ceph-mon.a.log.gz**: ... Xiubo Li
07:26 AM Bug #57083: ceph-fuse: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but ...
The root cause is that in **nautilus** the **qa/workunits/fs/upgrade/volume_client** script is using **python2** to r... Xiubo Li
07:21 AM Bug #57083: ceph-fuse: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but ...
From **remote/smithi029/log/ceph-mon.a.log.gz**: ... Xiubo Li
07:10 AM Bug #57083 (Resolved): ceph-fuse: monclient(hunting): handle_auth_bad_method server allowed_metho...
From https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pacific-distro-default-s... Xiubo Li
07:54 AM Bug #53360 (Duplicate): pacific: client: "handle_auth_bad_method server allowed_methods [2] but i...
Missed this existing tracker. Will track this in https://tracker.ceph.com/issues/57083 tracker. Have found root cause... Xiubo Li
07:37 AM Bug #57084 (Resolved): Permissions of the .snap directory do not inherit ACLs
when using CephFS with POSIX ACLs I noticed that the .snap directory does not inherit the ACLs from its parent but on... Robert Sander
07:26 AM Backport #53861: pacific: qa: tasks.cephfs.fuse_mount:mount command failed
Created a new tracker to fix it https://tracker.ceph.com/issues/57083. Xiubo Li
06:50 AM Backport #53861: pacific: qa: tasks.cephfs.fuse_mount:mount command failed
Xiubo Li wrote:
> Kotresh Hiremath Ravishankar wrote:
> > Xiubo,
> >
> > Looks like this is seen again in this p...
Xiubo Li
07:23 AM Bug #55572: qa/cephfs: omit_sudo doesn't have effect when passed to run_shell()
I think this needs to be backported. Nikhil mentioned that the PR https://github.com/ceph/ceph/pull/47112 in pacific ... Kotresh Hiremath Ravishankar
07:08 AM Bug #57071 (Fix Under Review): mds: consider mds_cap_revoke_eviction_timeout for get_late_revokin...
Venky Shankar

08/09/2022

04:13 PM Backport #56527: pacific: mds: src/mds/MDLog.cc: 283: FAILED ceph_assert(!mds->is_ any_replay())
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47111
merged
Yuri Weinstein
04:12 PM Backport #56152: pacific: mgr/snap_schedule: schedule updates are not persisted across mgr restart
https://github.com/ceph/ceph/pull/46797 merged Yuri Weinstein
12:55 PM Bug #56529: ceph-fs crashes on getfattr
Frank Schilder wrote:
> Thanks for the quick answer. Then, I guess, the patch to the ceph-fs clients will handle thi...
Venky Shankar
12:54 PM Bug #56529: ceph-fs crashes on getfattr
Frank Schilder wrote:
> Thanks for the quick answer. Then, I guess, the patch to the ceph-fs clients will handle thi...
Xiubo Li
12:47 PM Bug #56529: ceph-fs crashes on getfattr
Thanks for the quick answer. Then, I guess, the patch to the ceph-fs clients will handle this once it is approved? I ... Frank Schilder
12:40 PM Bug #56529: ceph-fs crashes on getfattr
Frank Schilder wrote:
> Hi all,
>
> this story continues, this time with a _valid_ vxattr name. I just observed e...
Xiubo Li
12:33 PM Bug #56529: ceph-fs crashes on getfattr
Hi all,
this story continues, this time with a _valid_ vxattr name. I just observed exactly the same problem now w...
Frank Schilder
11:40 AM Bug #57072 (Pending Backport): Quincy 17.2.3 pybind/mgr/status: assert metadata failed
`ceph fs status` return AssertionError
Error EINVAL: Traceback (most recent call last):
File "/usr/share/ceph/m...
cld slnt
10:24 AM Backport #53861: pacific: qa: tasks.cephfs.fuse_mount:mount command failed
Kotresh Hiremath Ravishankar wrote:
> Xiubo,
>
> Looks like this is seen again in this pacific run ?
>
> https...
Xiubo Li
10:13 AM Backport #53861: pacific: qa: tasks.cephfs.fuse_mount:mount command failed
Xiubo,
Looks like this is seen again in this pacific run ?
https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-f...
Kotresh Hiremath Ravishankar
10:24 AM Bug #56644: qa: test_rapid_creation fails with "No space left on device"
Seen in recent pacific run https://pulpito.ceph.com/yuriw-2022-07-24_15:34:38-fs-wip-yuri2-testing-2022-07-15-0755-pa... Kotresh Hiremath Ravishankar
09:27 AM Bug #57071 (Fix Under Review): mds: consider mds_cap_revoke_eviction_timeout for get_late_revokin...
Even though mds_cap_revoke_eviction_timeout is set to zero, ceph-mon reports some clients failing to respond to capab... Yongseok Oh
09:01 AM Bug #57048: osdc/Journaler: better handle ENOENT during replay as up:standby-replay
Patrick,
Do you mean a standby-replay MDS should tolerate missing journal objects? How can it end up in such a sit...
Venky Shankar
08:58 AM Bug #56808: crash: LogSegment* MDLog::get_current_segment(): assert(!segments.empty())
Looks similar to https://tracker.ceph.com/issues/51589 which was fixed a while ago.
Kotresh, please RCA this.
Venky Shankar
08:16 AM Backport #57058 (In Progress): pacific: mgr/volumes: Handle internal metadata directories under '...
Nikhilkumar Shelke
08:06 AM Backport #57057 (In Progress): quincy: mgr/volumes: Handle internal metadata directories under '/...
Nikhilkumar Shelke
07:07 AM Bug #54462: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status...
This maybe duplicated to https://tracker.ceph.com/issues/55332. Xiubo Li
06:55 AM Bug #54462: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi055 with status...
Seen in this run too.
https://pulpito.ceph.com/yuriw-2022-08-02_21:20:37-fs-wip-yuri7-testing-2022-07-27-0808-quin...
Kotresh Hiremath Ravishankar
06:51 AM Bug #56592: mds: crash when mounting a client during the scrub repair is going on
More info:
I was just simulating the cu case we hit by just removing one object of the directory from the metadata...
Xiubo Li
06:47 AM Bug #56592: mds: crash when mounting a client during the scrub repair is going on
Venky Shankar wrote:
> Xiubo,
>
> Were you trying to mount /mydir when it was getting repaired?
No, I was just...
Xiubo Li
06:23 AM Bug #56592: mds: crash when mounting a client during the scrub repair is going on
Xiubo,
Were you trying to mount /mydir when it was getting repaired?
Venky Shankar
06:47 AM Bug #51964: qa: test_cephfs_mirror_restart_sync_on_blocklist failure
Seen in this quincy run https://pulpito.ceph.com/yuriw-2022-08-02_21:20:37-fs-wip-yuri7-testing-2022-07-27-0808-quinc... Kotresh Hiremath Ravishankar
06:30 AM Bug #56830: crash: cephfs::mirror::PeerReplayer::pick_directory()
Dhairya,
Please take a look at this. I think there is some sort of race that is causing this crash while iterating...
Venky Shankar
06:25 AM Bug #57014: cephfs-top: add an option to dump the computed values to stdout
Jos, please take this one. Venky Shankar
05:54 AM Bug #56996 (In Progress): Transient data read corruption from other machine
Xiubo Li
04:48 AM Bug #56996: Transient data read corruption from other machine
Witold Baryluk wrote:
> Ok. I still do not understand why this can happen:
>
> writer: write("a"); write("b"); wr...
Xiubo Li
04:59 AM Bug #57065 (Closed): qa: test_query_client_ip_filter fails with latest 'perf stats' structure cha...
test_query_client_ip_filter fails with the below error in tests [1] and [2]. This happens when PR [3] is tested.
<...
Jos Collin
04:41 AM Bug #57064 (Need More Info): qa: test_add_ancestor_and_child_directory failure
Seen in recent quincy run https://pulpito.ceph.com/yuriw-2022-08-04_11:54:20-fs-wip-yuri8-testing-2022-08-03-1028-qui... Kotresh Hiremath Ravishankar
02:47 AM Bug #56067: Cephfs data loss with root_squash enabled
Patrick Donnelly wrote:
> Please open a PR for discussion.
https://github.com/ceph/ceph/pull/47506 . Please take ...
Ramana Raja
02:45 AM Bug #56067 (Fix Under Review): Cephfs data loss with root_squash enabled
Ramana Raja

08/08/2022

07:03 PM Documentation #57062 (New): Document access patterns that have good/pathological performance on C...
I have a CephFS 16.2.7 with 200 M small files (between 1 KB and 100 KB; ther are a few larger ones up to 200 MB) and ... Niklas Hambuechen
03:28 PM Bug #56048: ceph.mirror.info is not removed from target FS when mirroring is disabled
Hi Venky,
I tried it again, now with 17.2.1, and I could reproduce the issue. The mgr debug log is below.
As fa...
Andreas Teuchert
01:08 PM Bug #56048: ceph.mirror.info is not removed from target FS when mirroring is disabled
Andreas Teuchert wrote:
> When disabling mirroring on a FS with "ceph fs snapshot mirror disable <source-fs>" the "c...
Venky Shankar
02:32 PM Bug #56996: Transient data read corruption from other machine
Ok. I still do not understand why this can happen:
writer: write("a"); write("b"); write("c");
reader (other cl...
Witold Baryluk
06:33 AM Bug #56996: Transient data read corruption from other machine
Witold Baryluk wrote:
> What about when there is one writer and one reader?
This will depend on whether they are ...
Xiubo Li
01:18 PM Feature #56643: scrub: add one subcommand or option to add the missing objects back
Venky Shankar wrote:
> Xiubo Li wrote:
> > When we are scrub repairing the metadatas and some objects may get lost ...
Xiubo Li
01:02 PM Feature #56643: scrub: add one subcommand or option to add the missing objects back
Xiubo Li wrote:
> When we are scrub repairing the metadatas and some objects may get lost due to some reasons. After...
Venky Shankar
01:01 PM Bug #56249: crash: int Client::_do_remount(bool): abort
Xiubo Li wrote:
> Should be fixed by https://tracker.ceph.com/issues/54049.
Looks the same. However, I'm not sure...
Venky Shankar
09:41 AM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
Milind Changire wrote:
> Adding any more condition to the assertion expression and passing the assertion is not goin...
Venky Shankar
08:07 AM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
Adding any more condition to the assertion expression and passing the assertion is not going to do any good.
Since M...
Milind Changire
05:37 AM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
Never mind - I see the err coming from JournalPointer. If the MDS is respawning/shutting down could that condition ad... Venky Shankar
05:29 AM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
Milind Changire wrote:
> This seems to be a race between an mds respawn and the MDLog::_recovery_thread()
> In Paci...
Venky Shankar
08:55 AM Backport #57058 (Resolved): pacific: mgr/volumes: Handle internal metadata directories under '/vo...
https://github.com/ceph/ceph/pull/47512 Backport Bot
08:55 AM Backport #57057 (Resolved): quincy: mgr/volumes: Handle internal metadata directories under '/vol...
https://github.com/ceph/ceph/pull/47511 Backport Bot
08:54 AM Bug #55762 (Pending Backport): mgr/volumes: Handle internal metadata directories under '/volumes'...
Venky Shankar

08/05/2022

09:26 PM Bug #56067: Cephfs data loss with root_squash enabled
Greg Farnum wrote:
>
>
> But now I have another question -- does this mean that a kclient which only has access ...
Ramana Raja
04:21 PM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
This seems to be a race between an mds respawn and the MDLog::_recovery_thread()
In Pacific, the MDLog::_recovery_th...
Milind Changire
01:15 PM Bug #57048 (Pending Backport): osdc/Journaler: better handle ENOENT during replay as up:standby-r...
... Patrick Donnelly
06:37 AM Backport #57042 (In Progress): quincy: pybind/mgr/volumes: interface to check the presence of sub...
Neeraj Pratap Singh
04:42 AM Bug #48673: High memory usage on standby replay MDS
We seem to be running into this pretty frequently and easily with standby-replay configuration. Venky Shankar

08/04/2022

11:43 PM Bug #57044 (Fix Under Review): mds: add some debug logs for "crash during construction of interna...
Xiubo Li
11:42 PM Bug #57044 (Resolved): mds: add some debug logs for "crash during construction of internal request"
... Xiubo Li
07:26 PM Bug #56802 (Duplicate): crash: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*): assert(!...
Patrick Donnelly
03:33 PM Bug #55897: test_nfs: update of export's access type should not trigger NFS service restart
/a/yuriw-2022-08-03_20:33:43-rados-wip-yuri8-testing-2022-08-03-1028-quincy-distro-default-smithi/6957515 Laura Flores
02:37 PM Backport #57041 (In Progress): pacific: pybind/mgr/volumes: interface to check the presence of su...
Neeraj Pratap Singh
01:15 PM Backport #57041 (Resolved): pacific: pybind/mgr/volumes: interface to check the presence of subvo...
https://github.com/ceph/ceph/pull/47460 Backport Bot
01:15 PM Backport #57042 (Resolved): quincy: pybind/mgr/volumes: interface to check the presence of subvol...
https://github.com/ceph/ceph/pull/47474 Backport Bot
01:10 PM Feature #55821 (Pending Backport): pybind/mgr/volumes: interface to check the presence of subvolu...
Venky Shankar
12:19 PM Bug #56996: Transient data read corruption from other machine
What about when there is one writer and one reader? Witold Baryluk
12:36 AM Bug #56996: Transient data read corruption from other machine
I am not very sure this is a bug.
If there are multiple clients and they are in any of:...
Xiubo Li
10:59 AM Fix #51177: pybind/mgr/volumes: investigate moving calls which may block on libcephfs into anothe...
Kotresh, please take a look at this. Venky Shankar

08/03/2022

02:46 PM Bug #56644: qa: test_rapid_creation fails with "No space left on device"
Rishabh,
Do we know why the space issue started to show up recently?
Venky Shankar
02:19 PM Bug #56517 (Resolved): fuse_ll.cc: error: expected identifier before ‘{’ token 1379 | {
Venky Shankar
10:36 AM Bug #57014 (Resolved): cephfs-top: add an option to dump the computed values to stdout
It would be nice if cephfs-top dumps it's computed values to stdout in json format. The json should contain all the f... Jos Collin
08:16 AM Backport #56462 (In Progress): pacific: mds: crash due to seemingly unrecoverable metadata error
Xiubo Li
08:15 AM Backport #56462 (Need More Info): pacific: mds: crash due to seemingly unrecoverable metadata error
Xiubo Li
08:12 AM Backport #56461 (In Progress): quincy: mds: crash due to seemingly unrecoverable metadata error
Xiubo Li
06:13 AM Bug #56506: pacific: Test failure: test_rebuild_backtraceless (tasks.cephfs.test_data_scan.TestDa...
Milind, please RCA this. Venky Shankar
12:04 AM Fix #51177: pybind/mgr/volumes: investigate moving calls which may block on libcephfs into anothe...
Downstream BZ - https://bugzilla.redhat.com/show_bug.cgi?id=2114615 Vikhyat Umrao

08/02/2022

02:09 PM Bug #56626 (Closed): "ceph fs volume create" fails with error ERANGE
Closing the bug. Changes in devstack-plugin-ceph, https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/85152... Ramana Raja
02:03 PM Bug #55858: Pacific 16.2.7 MDS constantly crashing
I've noticed a commonality when this is being triggered, Singularity is being used https://en.wikipedia.org/wiki/Sing... Mike Lowe
08:15 AM Bug #56802: crash: void MDLog::_submit_entry(LogEvent*, MDSLogContextBase*): assert(!mds->is_any_...
Maybe this is relevant information to reproduce the crash:
I have NFS Ganesha running to export CephFS and when I ...
Andreas Teuchert
06:47 AM Bug #56988: mds: memory leak suspected
Here is a graph of the memory summary without and with the automated restart. Ramin Torabi
06:34 AM Bug #56988: mds: memory leak suspected
I have automated restarting a single MDS-Server when MDS memory consumption is 80GB (roughly twice the configured mds... Ramin Torabi
06:28 AM Bug #56695 (Fix Under Review): [RHEL stock] pjd test failures(a bug that need to wait the unlink ...
Xiubo Li
05:42 AM Bug #56695: [RHEL stock] pjd test failures(a bug that need to wait the unlink to finish)
Patrick Donnelly wrote:
> [...]
>
> /ceph/teuthology-archive/pdonnell-2022-07-22_19:42:58-fs-wip-pdonnell-testing...
Xiubo Li
02:50 AM Bug #56695: [RHEL stock] pjd test failures(a bug that need to wait the unlink to finish)
Xiubo Li wrote:
> Tried **4.18.0-348.20.1.el8_5.x86_64** and couldn't reproduce it.
>
> Will try the exact same ...
Xiubo Li
02:37 AM Bug #56695: [RHEL stock] pjd test failures(a bug that need to wait the unlink to finish)
Tried **4.18.0-348.20.1.el8_5.x86_64** and couldn't reproduce it.
Will try the exact same version of **kernel-4.1...
Xiubo Li

08/01/2022

04:34 PM Bug #56996 (In Progress): Transient data read corruption from other machine
Kernel cephfs on both sides.
* ceph version 15.2.15 (2dfb18841cfecc2f7eb7eb2afd65986ca4d95985) octopus (stable)
*...
Witold Baryluk
09:47 AM Bug #56695: [RHEL stock] pjd test failures(a bug that need to wait the unlink to finish)
Test this with the latest **testing** kclient branch, I couldn't reproduce it.
Will switch to use the distro kerne...
Xiubo Li
09:46 AM Bug #56695: [RHEL stock] pjd test failures(a bug that need to wait the unlink to finish)
Xiubo Li wrote:
> Currently the kclient's **testing** branch has merged the fscryption name related patches, which w...
Xiubo Li
09:10 AM Bug #56695 (In Progress): [RHEL stock] pjd test failures(a bug that need to wait the unlink to fi...
Currently the kclient's **testing** branch has merged the fscryption name related patches, which will limit the **MAX... Xiubo Li
09:08 AM Bug #56633 (Need More Info): mds: crash during construction of internal request
Locally I couldn't reproduce it. And by reading the code I couldn't figure out in which case will the internal reques... Xiubo Li
08:59 AM Bug #53573: qa: test new clients against older Ceph clusters
Xiubo Li wrote:
> The tracker [1] has done the test for new clients with nautilus ceph simultaneously.
>
> [1] ht...
Xiubo Li
08:51 AM Bug #53573: qa: test new clients against older Ceph clusters
The tracker [1] has done the test for new clients with nautilus ceph simultaneously.
[1] https://tracker.ceph.com/...
Xiubo Li
07:01 AM Bug #56988 (Need More Info): mds: memory leak suspected
We are runnung a cephfs pacific cluster in production:
MDS version: ceph version 16.2.9 (4c3647a322c0ff5a1dd2344...
Ramin Torabi
 

Also available in: Atom