Project

General

Profile

Activity

From 11/03/2022 to 12/02/2022

12/02/2022

02:12 PM Feature #58154 (Resolved): mds: add minor segment boundaries
See PR/commits. Patrick Donnelly
10:58 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> Is there a good way to monitor the read/write speed of the fuse and kernel client?
Is this ...
Venky Shankar
09:48 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Is there a good way to monitor the read/write speed of the fuse and kernel client?
xianpao chen

12/01/2022

01:33 PM Feature #58129: mon/FSCommands: support swapping file systems by name
Venky Shankar wrote:
> The operation also needs to swap the fsid and no clients should we interfering when the swap ...
Patrick Donnelly
04:10 AM Feature #58129: mon/FSCommands: support swapping file systems by name
The operation also needs to swap the fsid and no clients should we interfering when the swap is under execution. Venky Shankar
11:04 AM Bug #58138 (In Progress): "ceph nfs cluster info" shows junk data for non-existent cluster
Dhairya Parmar
09:18 AM Bug #58138 (Resolved): "ceph nfs cluster info" shows junk data for non-existent cluster
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2149415
Steps to Reproduce(we will use a non-existent cluster name...
Dhairya Parmar
01:28 AM Feature #58133 (Fix Under Review): qa: add test cases for fscrypt feature in kernel CephFS client
Xiubo Li
01:22 AM Feature #58133 (Resolved): qa: add test cases for fscrypt feature in kernel CephFS client
As per the documentation fscrypt is a (kernel) "library which filesystems can hook into to support transparent encryp... Xiubo Li

11/30/2022

09:48 PM Bug #54643 (Duplicate): crash: void Server::_unlink_local(MDRequestRef&, CDentry*, CDentry*): ass...
Patrick Donnelly
09:48 PM Bug #53179 (Duplicate): Crash when unlink in corrupted cephfs
Patrick Donnelly
09:47 PM Bug #38452 (Need More Info): mds: assert crash loop while unlinking file
Patrick Donnelly
05:05 PM Feature #58129 (Pending Backport): mon/FSCommands: support swapping file systems by name
Storage operators like Rook constantly do "reconciliation" to ensure that the desired state of the system (e.g. file ... Patrick Donnelly
03:38 PM Bug #24403: mon failed to return metadata for mds
Was discussion about this tracker with Patrick - there are separate paxos proposals for fsmap update and the metadata... Venky Shankar
02:27 PM Bug #24403: mon failed to return metadata for mds
The MDS is identified using a nonce as well as an IP in the map, right? After the containerized OSDs managed to clobb... Greg Farnum
02:23 PM Bug #24403: mon failed to return metadata for mds
Venky Shankar wrote:
> It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadat...
Patrick Donnelly
12:52 PM Bug #24403: mon failed to return metadata for mds
It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadata and includes that in t... Venky Shankar
09:25 AM Bug #57014 (Fix Under Review): cephfs-top: add an option to dump the computed values to stdout
Jos Collin

11/29/2022

07:13 AM Feature #56489: qa: test mgr plugins with standby mgr failover
New pull request with mgr thrasher. Milind Changire
05:10 AM Bug #58109 (Pending Backport): ceph-fuse: doesn't work properly when the version of libfuse is 3....
I want to use ceph-fuse with libfuse which version is 3.6 or later, because it supports for fuse kernel feature `max_... Zhansong Gao
04:50 AM Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
Venky Shankar wrote:
> Milind Changire wrote:
> > The most common mistake that users tend to do is include the moun...
Milind Changire
02:04 AM Bug #58090: Non-existent pending clone shows up in snapshot info
The `/volumes/_index/clone/` directory is empty, by the way. But that's after the snapshot was deleted successfully. ... Sebastian Hasler
02:01 AM Bug #58090: Non-existent pending clone shows up in snapshot info
Now the snapshot is deleted (finally). From the logs of our CSI provisioner, it seems that the snapshot was deleted s... Sebastian Hasler
01:26 AM Feature #58070: qa: add test suite to test old kernels
Patrick Donnelly wrote:
> This is certainly a good thing to add. Where do we want to put it? fs:workload?
I was ...
Xiubo Li
01:16 AM Feature #58070: qa: add test suite to test old kernels
This is certainly a good thing to add. Where do we want to put it? fs:workload? We need to be careful to avoid testin... Patrick Donnelly
01:18 AM Feature #58072: enable 'ceph fs new' use 'ceph fs set' options
I think at this point we should consider making it possible to set arbitrary settings on a fs during creation. i.e. a... Patrick Donnelly

11/28/2022

05:19 PM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
Milind Changire wrote:
> Due to unavailability of debug logs, there has been some speculation about the issue during...
Venky Shankar
04:58 PM Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
Milind Changire wrote:
> The most common mistake that users tend to do is include the mount point path along with th...
Venky Shankar
03:39 PM Bug #58095 (Resolved): snap-schedule: handle non-existent path gracefully during snapshot creation
The most common mistake that users tend to do is include the mount point path along with the file-system path when us... Milind Changire
03:26 PM Bug #54017: Problem with ceph fs snapshot mirror and read-only folders
Milind, this was discussed here - https://www.mail-archive.com/ceph-users@ceph.io/msg14364.html
Related bz - https...
Venky Shankar
03:06 PM Bug #58090: Non-existent pending clone shows up in snapshot info
Hi Sebastian,
There is a stray index causing this issue. Could you list the contents of `/volumes/_index/clone/` (...
Venky Shankar
02:44 PM Bug #58058: CephFS Snapshot Mirroring slow due to repeating attribute sync
Mathias Kuhring wrote:
> We might have found a major performance bug in the cephfs snapshot mirroring.
> We already...
Venky Shankar
01:41 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
Venky Shankar
05:32 AM Bug #58082 (Fix Under Review): cephfs:filesystem became read only after Quincy upgrade
Konstantin Shalygin

11/27/2022

02:08 PM Bug #58090 (New): Non-existent pending clone shows up in snapshot info
Ceph version: v17.2.5
My CephFS somehow got in a state where a snapshot has a pending clone, but the pending clone...
Sebastian Hasler

11/26/2022

05:23 PM Bug #58088 (Fix Under Review): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd()...
Ramana Raja
05:08 PM Bug #58088 (Resolved): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd() takes 3...
Hit this error,... Ramana Raja

11/25/2022

10:12 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
Due to unavailability of debug logs, there has been some speculation about the issue during discussion with Venky.
T...
Milind Changire
05:07 AM Bug #58082: cephfs:filesystem became read only after Quincy upgrade
From the logs, the *dir(0x1)* will submit the *volumes* Dentry to metadata pool: ... Xiubo Li
04:51 AM Bug #58082 (Resolved): cephfs:filesystem became read only after Quincy upgrade
Copy the info from ceph-user mail list by Adrien:... Xiubo Li
04:54 AM Bug #52260 (Duplicate): 1 MDSs are read only | pacific 16.2.5
Will tracker and fix it in https://tracker.ceph.com/issues/58082. Xiubo Li

11/24/2022

05:27 PM Backport #58079 (Resolved): quincy: cephfs-top: Sorting doesn't work when the filesystems are rem...
https://github.com/ceph/ceph/pull/50151 Backport Bot
05:27 PM Backport #58078 (Resolved): pacific: cephfs-top: Sorting doesn't work when the filesystems are re...
https://github.com/ceph/ceph/pull/49303 Backport Bot
05:26 PM Bug #58028 (Pending Backport): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Venky Shankar
01:39 PM Backport #58074 (Resolved): quincy: cephfs-top: sorting/limit excepts when the filesystems are re...
https://github.com/ceph/ceph/pull/50151 Backport Bot
01:39 PM Backport #58073 (Resolved): pacific: cephfs-top: sorting/limit excepts when the filesystems are r...
https://github.com/ceph/ceph/pull/49303 Backport Bot
01:36 PM Bug #58031 (Pending Backport): cephfs-top: sorting/limit excepts when the filesystems are removed...
Venky Shankar
01:05 PM Feature #58072 (Fix Under Review): enable 'ceph fs new' use 'ceph fs set' options
As discussed in PR [1], this flag would come handy in situations like 'ceph fs new --recover'. Need to push this enha... Dhairya Parmar
07:33 AM Feature #58070 (New): qa: add test suite to test old kernels
Currently there is test case will test old ceph-fuse clients with new ceph, but we also need to test the old kclient ... Xiubo Li
05:36 AM Feature #55940 (Fix Under Review): quota: accept values in human readable format as well
Dhairya Parmar

11/23/2022

05:31 PM Bug #24403: mon failed to return metadata for mds
FYI - restarting the MDS fixes the issue. Venky Shankar
05:30 PM Bug #24403: mon failed to return metadata for mds
This was seen in pacific installation. MDS entries in FSMap are fine - that serves `fs dump` and `fs status` commands... Venky Shankar
12:05 PM Bug #58031 (Fix Under Review): cephfs-top: sorting/limit excepts when the filesystems are removed...
Neeraj Pratap Singh
08:05 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory...
Venky Shankar
07:40 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory_limit = 16GB, no speci... xianpao chen
07:25 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Any reason you are using mds_cache_memory_limit = 16GB when you have memory to spare? Venky Shankar
06:32 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
the "free -h" of the mds node(after restart the mds): ... xianpao chen
06:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, do you see any performance degradation on clients in general over the course and/or when the MDS is about to get... Venky Shankar
05:27 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I checked the session info, there is just one client which is holding ~1M caps. But that should not bother the MDS th... Venky Shankar

11/22/2022

03:42 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I happened to have a memory problem today, then I changed mds_session_cache_liveness_decay_rate to 150s, tried "ceph ... xianpao chen
11:19 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Did you get to applying the suggested config?
Thanks for your suggestion, I will try it to...
xianpao chen
11:07 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Did you get to applying the suggested config? Venky Shankar
10:19 AM Bug #57523: CephFS performance degredation in mountpoint
Guys this can't be only a thing in our setup. Every time a connection puts more than a few GB into cephfs the perform... Vincent Hermes
01:19 AM Bug #58056: ceph-fuse - fuse failed to start on CentOS 7 host machine
It seems passing invalidate flag ?... Xiubo Li

11/21/2022

08:17 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
We might have found a major performance bug in the cephfs snapshot mirroring.
We already reported it to the mailing ...
Mathias Kuhring
06:11 PM Support #38374: Crash when using cephfs as /var/lib/docker in devicemapper mode
We're not using this kind of setup anymore and won't be troubleshooting further. We can close this for now, probably ... Jérôme Poulin
01:53 PM Feature #58057: cephfs-top: enhance fstop tests to cover testing displayed data
The Dashboard folks could point us to tools for testing console UI based apps. Milind Changire
11:30 AM Feature #58057 (Resolved): cephfs-top: enhance fstop tests to cover testing displayed data
Right now the tests are pretty rudimentary. cephfs-top is a UI tool and writing tests can be a bit hard. Due to this ... Venky Shankar
09:05 AM Bug #58056 (New): ceph-fuse - fuse failed to start on CentOS 7 host machine
Hello,
We were previously using ceph v16.2.10 (docker container) with rook under kubernetes which was installed on...
Razvan Ghitescu
06:31 AM Bug #57014 (In Progress): cephfs-top: add an option to dump the computed values to stdout
Jos Collin

11/18/2022

11:38 AM Bug #58028 (Fix Under Review): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Jos Collin

11/17/2022

12:05 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Hey,
Thanks for the update. You should try adjusting `mds_session_cache_liveness_decay_rate` to a lower value (def...
Venky Shankar
10:17 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
and another side note, the crash was seen when a directory pin was removed from rank-0 mds. Pinning it back again cea... Venky Shankar
10:16 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
oh, and btw this was seen in ceph-16.2.8. Venky Shankar
10:15 AM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
... Venky Shankar
09:21 AM Feature #55215 (Fix Under Review): mds: fragment directory snapshots
Venky Shankar

11/15/2022

01:49 PM Bug #58031 (Resolved): cephfs-top: sorting/limit excepts when the filesystems are removed and cre...
This happens in the main branch. Please check.
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
01:42 PM Bug #58000 (Fix Under Review): mds: switch submit_mutex to fair mutex for MDLog
Venky Shankar
01:41 PM Bug #58008 (Fix Under Review): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate...
Venky Shankar
01:41 PM Bug #58028 (Triaged): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Venky Shankar
10:12 AM Bug #58028 (Resolved): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Sorting doesn't work in the following scenario
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
11:08 AM Bug #58030 (Resolved): mds: avoid ~mdsdir's scrubbing and reporting damage health status
We are supposed to handle the case of mdsdir, where we
are not having any backtrace actually.We should prevent the
...
Neeraj Pratap Singh
10:49 AM Bug #58029 (Fix Under Review): cephfs-data-scan: multiple data pools are not supported
Mykola Golub
10:46 AM Bug #58029 (Resolved): cephfs-data-scan: multiple data pools are not supported
The tool cannot properly recover if a fs has extra data pools. We need access to all data pools on `scan_extents` ste... Mykola Golub

11/14/2022

09:32 PM Fix #58023 (Pending Backport): mds: do not evict clients if OSDs are laggy
Monitoring perf dumps from the MDS can sometimes show that OSDs are laggy, "objecter.op_laggy" and "objecter.osd_lagg... Patrick Donnelly
01:27 PM Bug #58018 (Fix Under Review): mount.ceph: will fail with old kernels
Xiubo Li
10:09 AM Bug #58018 (Pending Backport): mount.ceph: will fail with old kernels
... Xiubo Li

11/11/2022

02:11 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> xianpao chen wrote:
> > Venky Shankar wrote:
> > > Could you share the output of
> > >
>...
xianpao chen
01:02 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> Venky Shankar wrote:
> > Could you share the output of
> >
> > [...]
> >
> > Also, does...
Venky Shankar
09:14 AM Bug #58008: mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
When increasing filer_max_purge_ops on a pacific version mds, pq_executing_ops/pq_executing_ops_high_water of purge_q... yixing hao
09:13 AM Bug #58008 (Resolved): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
_calculate_ops relying on a config which can be modified on the fly will cause a bug. e.g.
# A file has 20 objects...
yixing hao

11/10/2022

08:18 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> BTW, are you *not* seeing any "oversized cache" warning for the MDS?
there is no "oversize...
xianpao chen
04:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, are you *not* seeing any "oversized cache" warning for the MDS? Venky Shankar
02:42 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Do you have lots of small files and frequently scan them? Venky Shankar
01:12 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Have you tried running `heap release`?
yes,but it didn't seem to work.
xianpao chen
01:45 AM Bug #58000: mds: switch submit_mutex to fair mutex for MDLog
From Patrick's comment in https://github.com/ceph/ceph/pull/44180#pullrequestreview-1174516711. Xiubo Li
01:44 AM Bug #58000 (Resolved): mds: switch submit_mutex to fair mutex for MDLog
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the ...
Xiubo Li

11/09/2022

07:08 PM Feature #57090 (Fix Under Review): MDSMonitor,mds: add MDSMap flag to prevent clients from connec...
Dhairya Parmar
01:22 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Have you tried running `heap release`? Venky Shankar
09:35 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
09:23 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
08:56 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Could you share the output of... Venky Shankar

11/07/2022

01:48 PM Bug #57985 (Triaged): mds: warning `clients failing to advance oldest client/flush tid` seen with...
Venky Shankar
09:06 AM Bug #57985 (Pending Backport): mds: warning `clients failing to advance oldest client/flush tid` ...
https://bugzilla.redhat.com/show_bug.cgi?id=2134709
Generally seen when the MDS is heavily loaded with I/Os. Inter...
Venky Shankar

11/04/2022

07:48 PM Bug #49132: mds crashed "assert_condition": "state == LOCK_XLOCK || state == LOCK_XLOCKDONE",
Alternative fix is available at https://github.com/ceph/ceph/pull/48743 Igor Fedotov
08:54 AM Backport #57974 (In Progress): pacific: cephfs-top: make cephfs-top display scrollable like top
Jos Collin
08:46 AM Backport #57974 (Resolved): pacific: cephfs-top: make cephfs-top display scrollable like top
https://github.com/ceph/ceph/pull/48734 Jos Collin
03:51 AM Backport #57971 (Resolved): pacific: cephfs-top: new options to limit and order-by
https://github.com/ceph/ceph/pull/49303 Backport Bot
03:50 AM Backport #57970 (Resolved): quincy: cephfs-top: new options to limit and order-by
https://github.com/ceph/ceph/pull/50151 Backport Bot
03:25 AM Feature #55121 (Pending Backport): cephfs-top: new options to limit and order-by
Jos Collin

11/03/2022

12:45 PM Feature #44455 (In Progress): cephfs: add recursive unlink RPC
Patrick Donnelly
09:30 AM Feature #57090 (In Progress): MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
Dhairya Parmar
07:34 AM Feature #57090: MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
Patrick Donnelly wrote:
> Dhairya, status on this?
Hi Patrick, i'm on this completely now. Will try bring somethi...
Dhairya Parmar
09:20 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
If you're running into this bug after upgrading from Pacific to Quincy, you can manually delete the legacy schedule D... Andreas Teuchert
08:49 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
{"log":"debug 2022-11-03T08:38:12.502+0000 7f46270f5700 -1 mgr load Failed to construct class in 'snap_schedule'\n","... Alexander Mamonov
08:46 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
How to fix it? Alexander Mamonov
 

Also available in: Atom