Project

General

Profile

Activity

From 10/30/2022 to 11/28/2022

11/28/2022

05:19 PM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
Milind Changire wrote:
> Due to unavailability of debug logs, there has been some speculation about the issue during...
Venky Shankar
04:58 PM Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
Milind Changire wrote:
> The most common mistake that users tend to do is include the mount point path along with th...
Venky Shankar
03:39 PM Bug #58095 (Resolved): snap-schedule: handle non-existent path gracefully during snapshot creation
The most common mistake that users tend to do is include the mount point path along with the file-system path when us... Milind Changire
03:26 PM Bug #54017: Problem with ceph fs snapshot mirror and read-only folders
Milind, this was discussed here - https://www.mail-archive.com/ceph-users@ceph.io/msg14364.html
Related bz - https...
Venky Shankar
03:06 PM Bug #58090: Non-existent pending clone shows up in snapshot info
Hi Sebastian,
There is a stray index causing this issue. Could you list the contents of `/volumes/_index/clone/` (...
Venky Shankar
02:44 PM Bug #58058: CephFS Snapshot Mirroring slow due to repeating attribute sync
Mathias Kuhring wrote:
> We might have found a major performance bug in the cephfs snapshot mirroring.
> We already...
Venky Shankar
01:41 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
Venky Shankar
05:32 AM Bug #58082 (Fix Under Review): cephfs:filesystem became read only after Quincy upgrade
Konstantin Shalygin

11/27/2022

02:08 PM Bug #58090 (New): Non-existent pending clone shows up in snapshot info
Ceph version: v17.2.5
My CephFS somehow got in a state where a snapshot has a pending clone, but the pending clone...
Sebastian Hasler

11/26/2022

05:23 PM Bug #58088 (Fix Under Review): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd()...
Ramana Raja
05:08 PM Bug #58088 (Resolved): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd() takes 3...
Hit this error,... Ramana Raja

11/25/2022

10:12 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
Due to unavailability of debug logs, there has been some speculation about the issue during discussion with Venky.
T...
Milind Changire
05:07 AM Bug #58082: cephfs:filesystem became read only after Quincy upgrade
From the logs, the *dir(0x1)* will submit the *volumes* Dentry to metadata pool: ... Xiubo Li
04:51 AM Bug #58082 (Resolved): cephfs:filesystem became read only after Quincy upgrade
Copy the info from ceph-user mail list by Adrien:... Xiubo Li
04:54 AM Bug #52260 (Duplicate): 1 MDSs are read only | pacific 16.2.5
Will tracker and fix it in https://tracker.ceph.com/issues/58082. Xiubo Li

11/24/2022

05:27 PM Backport #58079 (Resolved): quincy: cephfs-top: Sorting doesn't work when the filesystems are rem...
https://github.com/ceph/ceph/pull/50151 Backport Bot
05:27 PM Backport #58078 (Resolved): pacific: cephfs-top: Sorting doesn't work when the filesystems are re...
https://github.com/ceph/ceph/pull/49303 Backport Bot
05:26 PM Bug #58028 (Pending Backport): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Venky Shankar
01:39 PM Backport #58074 (Resolved): quincy: cephfs-top: sorting/limit excepts when the filesystems are re...
https://github.com/ceph/ceph/pull/50151 Backport Bot
01:39 PM Backport #58073 (Resolved): pacific: cephfs-top: sorting/limit excepts when the filesystems are r...
https://github.com/ceph/ceph/pull/49303 Backport Bot
01:36 PM Bug #58031 (Pending Backport): cephfs-top: sorting/limit excepts when the filesystems are removed...
Venky Shankar
01:05 PM Feature #58072 (Fix Under Review): enable 'ceph fs new' use 'ceph fs set' options
As discussed in PR [1], this flag would come handy in situations like 'ceph fs new --recover'. Need to push this enha... Dhairya Parmar
07:33 AM Feature #58070 (New): qa: add test suite to test old kernels
Currently there is test case will test old ceph-fuse clients with new ceph, but we also need to test the old kclient ... Xiubo Li
05:36 AM Feature #55940 (Fix Under Review): quota: accept values in human readable format as well
Dhairya Parmar

11/23/2022

05:31 PM Bug #24403: mon failed to return metadata for mds
FYI - restarting the MDS fixes the issue. Venky Shankar
05:30 PM Bug #24403: mon failed to return metadata for mds
This was seen in pacific installation. MDS entries in FSMap are fine - that serves `fs dump` and `fs status` commands... Venky Shankar
12:05 PM Bug #58031 (Fix Under Review): cephfs-top: sorting/limit excepts when the filesystems are removed...
Neeraj Pratap Singh
08:05 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory...
Venky Shankar
07:40 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory_limit = 16GB, no speci... xianpao chen
07:25 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Any reason you are using mds_cache_memory_limit = 16GB when you have memory to spare? Venky Shankar
06:32 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
the "free -h" of the mds node(after restart the mds): ... xianpao chen
06:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, do you see any performance degradation on clients in general over the course and/or when the MDS is about to get... Venky Shankar
05:27 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I checked the session info, there is just one client which is holding ~1M caps. But that should not bother the MDS th... Venky Shankar

11/22/2022

03:42 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I happened to have a memory problem today, then I changed mds_session_cache_liveness_decay_rate to 150s, tried "ceph ... xianpao chen
11:19 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Did you get to applying the suggested config?
Thanks for your suggestion, I will try it to...
xianpao chen
11:07 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Did you get to applying the suggested config? Venky Shankar
10:19 AM Bug #57523: CephFS performance degredation in mountpoint
Guys this can't be only a thing in our setup. Every time a connection puts more than a few GB into cephfs the perform... Vincent Hermes
01:19 AM Bug #58056: ceph-fuse - fuse failed to start on CentOS 7 host machine
It seems passing invalidate flag ?... Xiubo Li

11/21/2022

08:17 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
We might have found a major performance bug in the cephfs snapshot mirroring.
We already reported it to the mailing ...
Mathias Kuhring
06:11 PM Support #38374: Crash when using cephfs as /var/lib/docker in devicemapper mode
We're not using this kind of setup anymore and won't be troubleshooting further. We can close this for now, probably ... Jérôme Poulin
01:53 PM Feature #58057: cephfs-top: enhance fstop tests to cover testing displayed data
The Dashboard folks could point us to tools for testing console UI based apps. Milind Changire
11:30 AM Feature #58057 (In Progress): cephfs-top: enhance fstop tests to cover testing displayed data
Right now the tests are pretty rudimentary. cephfs-top is a UI tool and writing tests can be a bit hard. Due to this ... Venky Shankar
09:05 AM Bug #58056 (New): ceph-fuse - fuse failed to start on CentOS 7 host machine
Hello,
We were previously using ceph v16.2.10 (docker container) with rook under kubernetes which was installed on...
Razvan Ghitescu
06:31 AM Bug #57014 (In Progress): cephfs-top: add an option to dump the computed values to stdout
Jos Collin

11/18/2022

11:38 AM Bug #58028 (Fix Under Review): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Jos Collin

11/17/2022

12:05 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Hey,
Thanks for the update. You should try adjusting `mds_session_cache_liveness_decay_rate` to a lower value (def...
Venky Shankar
10:17 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
and another side note, the crash was seen when a directory pin was removed from rank-0 mds. Pinning it back again cea... Venky Shankar
10:16 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
oh, and btw this was seen in ceph-16.2.8. Venky Shankar
10:15 AM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
... Venky Shankar
09:21 AM Feature #55215 (Fix Under Review): mds: fragment directory snapshots
Venky Shankar

11/15/2022

01:49 PM Bug #58031 (Resolved): cephfs-top: sorting/limit excepts when the filesystems are removed and cre...
This happens in the main branch. Please check.
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
01:42 PM Bug #58000 (Fix Under Review): mds: switch submit_mutex to fair mutex for MDLog
Venky Shankar
01:41 PM Bug #58008 (Fix Under Review): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate...
Venky Shankar
01:41 PM Bug #58028 (Triaged): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Venky Shankar
10:12 AM Bug #58028 (Resolved): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Sorting doesn't work in the following scenario
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
11:08 AM Bug #58030 (Resolved): mds: avoid ~mdsdir's scrubbing and reporting damage health status
We are supposed to handle the case of mdsdir, where we
are not having any backtrace actually.We should prevent the
...
Neeraj Pratap Singh
10:49 AM Bug #58029 (Fix Under Review): cephfs-data-scan: multiple data pools are not supported
Mykola Golub
10:46 AM Bug #58029 (Resolved): cephfs-data-scan: multiple data pools are not supported
The tool cannot properly recover if a fs has extra data pools. We need access to all data pools on `scan_extents` ste... Mykola Golub

11/14/2022

09:32 PM Fix #58023 (Pending Backport): mds: do not evict clients if OSDs are laggy
Monitoring perf dumps from the MDS can sometimes show that OSDs are laggy, "objecter.op_laggy" and "objecter.osd_lagg... Patrick Donnelly
01:27 PM Bug #58018 (Fix Under Review): mount.ceph: will fail with old kernels
Xiubo Li
10:09 AM Bug #58018 (Pending Backport): mount.ceph: will fail with old kernels
... Xiubo Li

11/11/2022

02:11 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> xianpao chen wrote:
> > Venky Shankar wrote:
> > > Could you share the output of
> > >
>...
xianpao chen
01:02 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> Venky Shankar wrote:
> > Could you share the output of
> >
> > [...]
> >
> > Also, does...
Venky Shankar
09:14 AM Bug #58008: mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
When increasing filer_max_purge_ops on a pacific version mds, pq_executing_ops/pq_executing_ops_high_water of purge_q... yixing hao
09:13 AM Bug #58008 (Resolved): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
_calculate_ops relying on a config which can be modified on the fly will cause a bug. e.g.
# A file has 20 objects...
yixing hao

11/10/2022

08:18 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> BTW, are you *not* seeing any "oversized cache" warning for the MDS?
there is no "oversize...
xianpao chen
04:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, are you *not* seeing any "oversized cache" warning for the MDS? Venky Shankar
02:42 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Do you have lots of small files and frequently scan them? Venky Shankar
01:12 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Have you tried running `heap release`?
yes,but it didn't seem to work.
xianpao chen
01:45 AM Bug #58000: mds: switch submit_mutex to fair mutex for MDLog
From Patrick's comment in https://github.com/ceph/ceph/pull/44180#pullrequestreview-1174516711. Xiubo Li
01:44 AM Bug #58000 (Resolved): mds: switch submit_mutex to fair mutex for MDLog
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the ...
Xiubo Li

11/09/2022

07:08 PM Feature #57090 (Fix Under Review): MDSMonitor,mds: add MDSMap flag to prevent clients from connec...
Dhairya Parmar
01:22 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Have you tried running `heap release`? Venky Shankar
09:35 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
09:23 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
08:56 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Could you share the output of... Venky Shankar

11/07/2022

01:48 PM Bug #57985 (Triaged): mds: warning `clients failing to advance oldest client/flush tid` seen with...
Venky Shankar
09:06 AM Bug #57985 (Pending Backport): mds: warning `clients failing to advance oldest client/flush tid` ...
https://bugzilla.redhat.com/show_bug.cgi?id=2134709
Generally seen when the MDS is heavily loaded with I/Os. Inter...
Venky Shankar

11/04/2022

07:48 PM Bug #49132: mds crashed "assert_condition": "state == LOCK_XLOCK || state == LOCK_XLOCKDONE",
Alternative fix is available at https://github.com/ceph/ceph/pull/48743 Igor Fedotov
08:54 AM Backport #57974 (In Progress): pacific: cephfs-top: make cephfs-top display scrollable like top
Jos Collin
08:46 AM Backport #57974 (Resolved): pacific: cephfs-top: make cephfs-top display scrollable like top
https://github.com/ceph/ceph/pull/48734 Jos Collin
03:51 AM Backport #57971 (Resolved): pacific: cephfs-top: new options to limit and order-by
https://github.com/ceph/ceph/pull/49303 Backport Bot
03:50 AM Backport #57970 (Resolved): quincy: cephfs-top: new options to limit and order-by
https://github.com/ceph/ceph/pull/50151 Backport Bot
03:25 AM Feature #55121 (Pending Backport): cephfs-top: new options to limit and order-by
Jos Collin

11/03/2022

12:45 PM Feature #44455 (In Progress): cephfs: add recursive unlink RPC
Patrick Donnelly
09:30 AM Feature #57090 (In Progress): MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
Dhairya Parmar
07:34 AM Feature #57090: MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
Patrick Donnelly wrote:
> Dhairya, status on this?
Hi Patrick, i'm on this completely now. Will try bring somethi...
Dhairya Parmar
09:20 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
If you're running into this bug after upgrading from Pacific to Quincy, you can manually delete the legacy schedule D... Andreas Teuchert
08:49 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
{"log":"debug 2022-11-03T08:38:12.502+0000 7f46270f5700 -1 mgr load Failed to construct class in 'snap_schedule'\n","... Alexander Mamonov
08:46 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
How to fix it? Alexander Mamonov

11/01/2022

02:32 AM Support #57952 (New): Pacific: the buffer_anon_bytes of ceph-mds is too large
The buffer_anon_bytes will reach 200+GB, then run out of machine memory.It does not seem to be able to effectively fr... xianpao chen

10/31/2022

09:50 AM Bug #57920: mds:ESubtreeMap event size is too large
Venky Shankar wrote:
> Hi,
>
> Could the list of PRs that try to address this issue be linked? (so, that we don't...
zhikuo du
09:36 AM Bug #57920: mds:ESubtreeMap event size is too large
Hi,
Could the list of PRs that try to address this issue be linked? (so, that we don't loose track of them).
As...
Venky Shankar
04:50 AM Bug #57920: mds:ESubtreeMap event size is too large
zhikuo du wrote:
> > I am afraid this won't work. As I remembered from my test before, the size of ESubtreeMap could...
Xiubo Li
02:47 AM Bug #57920: mds:ESubtreeMap event size is too large
> I am afraid this won't work. As I remembered from my test before, the size of ESubtreeMap could reach up to several... zhikuo du
02:37 AM Bug #57920: mds:ESubtreeMap event size is too large
zhikuo du wrote:
> > May I ask you a question:
> > What factors decide how many event must have a ESubtreeMap e...
Xiubo Li
04:35 AM Backport #57946 (In Progress): quincy: cephfs-top: make cephfs-top display scrollable like top
Jos Collin
04:26 AM Backport #57946 (Resolved): quincy: cephfs-top: make cephfs-top display scrollable like top
https://github.com/ceph/ceph/pull/48677 Backport Bot
04:21 AM Feature #55197 (Pending Backport): cephfs-top: make cephfs-top display scrollable like top
Venky Shankar

10/30/2022

02:18 PM Bug #57920: mds:ESubtreeMap event size is too large
> @Xiubo Li @Venky Shankar
>
> I readed the codes about: how the segment is trimmed and how ESubtreeMap/EImportSt...
zhikuo du
01:10 PM Bug #57920: mds:ESubtreeMap event size is too large
> May I ask you a question:
> What factors decide how many event must have a ESubtreeMap event? And what is the...
zhikuo du
 

Also available in: Atom