Activity
From 11/03/2022 to 12/02/2022
12/02/2022
- 02:12 PM Feature #58154 (Resolved): mds: add minor segment boundaries
- See PR/commits.
- 10:58 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- xianpao chen wrote:
> Is there a good way to monitor the read/write speed of the fuse and kernel client?
Is this ... - 09:48 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Is there a good way to monitor the read/write speed of the fuse and kernel client?
12/01/2022
- 01:33 PM Feature #58129: mon/FSCommands: support swapping file systems by name
- Venky Shankar wrote:
> The operation also needs to swap the fsid and no clients should we interfering when the swap ... - 04:10 AM Feature #58129: mon/FSCommands: support swapping file systems by name
- The operation also needs to swap the fsid and no clients should we interfering when the swap is under execution.
- 11:04 AM Bug #58138 (In Progress): "ceph nfs cluster info" shows junk data for non-existent cluster
- 09:18 AM Bug #58138 (Resolved): "ceph nfs cluster info" shows junk data for non-existent cluster
- BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2149415
Steps to Reproduce(we will use a non-existent cluster name... - 01:28 AM Feature #58133 (Fix Under Review): qa: add test cases for fscrypt feature in kernel CephFS client
- 01:22 AM Feature #58133 (Resolved): qa: add test cases for fscrypt feature in kernel CephFS client
- As per the documentation fscrypt is a (kernel) "library which filesystems can hook into to support transparent encryp...
11/30/2022
- 09:48 PM Bug #54643 (Duplicate): crash: void Server::_unlink_local(MDRequestRef&, CDentry*, CDentry*): ass...
- 09:48 PM Bug #53179 (Duplicate): Crash when unlink in corrupted cephfs
- 09:47 PM Bug #38452 (Need More Info): mds: assert crash loop while unlinking file
- 05:05 PM Feature #58129 (Pending Backport): mon/FSCommands: support swapping file systems by name
- Storage operators like Rook constantly do "reconciliation" to ensure that the desired state of the system (e.g. file ...
- 03:38 PM Bug #24403: mon failed to return metadata for mds
- Was discussion about this tracker with Patrick - there are separate paxos proposals for fsmap update and the metadata...
- 02:27 PM Bug #24403: mon failed to return metadata for mds
- The MDS is identified using a nonce as well as an IP in the map, right? After the containerized OSDs managed to clobb...
- 02:23 PM Bug #24403: mon failed to return metadata for mds
- Venky Shankar wrote:
> It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadat... - 12:52 PM Bug #24403: mon failed to return metadata for mds
- It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadata and includes that in t...
- 09:25 AM Bug #57014 (Fix Under Review): cephfs-top: add an option to dump the computed values to stdout
11/29/2022
- 07:13 AM Feature #56489: qa: test mgr plugins with standby mgr failover
- New pull request with mgr thrasher.
- 05:10 AM Bug #58109 (Pending Backport): ceph-fuse: doesn't work properly when the version of libfuse is 3....
- I want to use ceph-fuse with libfuse which version is 3.6 or later, because it supports for fuse kernel feature `max_...
- 04:50 AM Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
- Venky Shankar wrote:
> Milind Changire wrote:
> > The most common mistake that users tend to do is include the moun... - 02:04 AM Bug #58090: Non-existent pending clone shows up in snapshot info
- The `/volumes/_index/clone/` directory is empty, by the way. But that's after the snapshot was deleted successfully. ...
- 02:01 AM Bug #58090: Non-existent pending clone shows up in snapshot info
- Now the snapshot is deleted (finally). From the logs of our CSI provisioner, it seems that the snapshot was deleted s...
- 01:26 AM Feature #58070: qa: add test suite to test old kernels
- Patrick Donnelly wrote:
> This is certainly a good thing to add. Where do we want to put it? fs:workload?
I was ... - 01:16 AM Feature #58070: qa: add test suite to test old kernels
- This is certainly a good thing to add. Where do we want to put it? fs:workload? We need to be careful to avoid testin...
- 01:18 AM Feature #58072: enable 'ceph fs new' use 'ceph fs set' options
- I think at this point we should consider making it possible to set arbitrary settings on a fs during creation. i.e. a...
11/28/2022
- 05:19 PM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
- Milind Changire wrote:
> Due to unavailability of debug logs, there has been some speculation about the issue during... - 04:58 PM Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
- Milind Changire wrote:
> The most common mistake that users tend to do is include the mount point path along with th... - 03:39 PM Bug #58095 (Resolved): snap-schedule: handle non-existent path gracefully during snapshot creation
- The most common mistake that users tend to do is include the mount point path along with the file-system path when us...
- 03:26 PM Bug #54017: Problem with ceph fs snapshot mirror and read-only folders
- Milind, this was discussed here - https://www.mail-archive.com/ceph-users@ceph.io/msg14364.html
Related bz - https... - 03:06 PM Bug #58090: Non-existent pending clone shows up in snapshot info
- Hi Sebastian,
There is a stray index causing this issue. Could you list the contents of `/volumes/_index/clone/` (... - 02:44 PM Bug #58058: CephFS Snapshot Mirroring slow due to repeating attribute sync
- Mathias Kuhring wrote:
> We might have found a major performance bug in the cephfs snapshot mirroring.
> We already... - 01:41 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
- 05:32 AM Bug #58082 (Fix Under Review): cephfs:filesystem became read only after Quincy upgrade
11/27/2022
- 02:08 PM Bug #58090 (New): Non-existent pending clone shows up in snapshot info
- Ceph version: v17.2.5
My CephFS somehow got in a state where a snapshot has a pending clone, but the pending clone...
11/26/2022
- 05:23 PM Bug #58088 (Fix Under Review): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd()...
- 05:08 PM Bug #58088 (Resolved): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd() takes 3...
- Hit this error,...
11/25/2022
- 10:12 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
- Due to unavailability of debug logs, there has been some speculation about the issue during discussion with Venky.
T... - 05:07 AM Bug #58082: cephfs:filesystem became read only after Quincy upgrade
- From the logs, the *dir(0x1)* will submit the *volumes* Dentry to metadata pool: ...
- 04:51 AM Bug #58082 (Resolved): cephfs:filesystem became read only after Quincy upgrade
- Copy the info from ceph-user mail list by Adrien:...
- 04:54 AM Bug #52260 (Duplicate): 1 MDSs are read only | pacific 16.2.5
- Will tracker and fix it in https://tracker.ceph.com/issues/58082.
11/24/2022
- 05:27 PM Backport #58079 (Resolved): quincy: cephfs-top: Sorting doesn't work when the filesystems are rem...
- https://github.com/ceph/ceph/pull/50151
- 05:27 PM Backport #58078 (Resolved): pacific: cephfs-top: Sorting doesn't work when the filesystems are re...
- https://github.com/ceph/ceph/pull/49303
- 05:26 PM Bug #58028 (Pending Backport): cephfs-top: Sorting doesn't work when the filesystems are removed ...
- 01:39 PM Backport #58074 (Resolved): quincy: cephfs-top: sorting/limit excepts when the filesystems are re...
- https://github.com/ceph/ceph/pull/50151
- 01:39 PM Backport #58073 (Resolved): pacific: cephfs-top: sorting/limit excepts when the filesystems are r...
- https://github.com/ceph/ceph/pull/49303
- 01:36 PM Bug #58031 (Pending Backport): cephfs-top: sorting/limit excepts when the filesystems are removed...
- 01:05 PM Feature #58072 (Fix Under Review): enable 'ceph fs new' use 'ceph fs set' options
- As discussed in PR [1], this flag would come handy in situations like 'ceph fs new --recover'. Need to push this enha...
- 07:33 AM Feature #58070 (New): qa: add test suite to test old kernels
- Currently there is test case will test old ceph-fuse clients with new ceph, but we also need to test the old kclient ...
- 05:36 AM Feature #55940 (Fix Under Review): quota: accept values in human readable format as well
11/23/2022
- 05:31 PM Bug #24403: mon failed to return metadata for mds
- FYI - restarting the MDS fixes the issue.
- 05:30 PM Bug #24403: mon failed to return metadata for mds
- This was seen in pacific installation. MDS entries in FSMap are fine - that serves `fs dump` and `fs status` commands...
- 12:05 PM Bug #58031 (Fix Under Review): cephfs-top: sorting/limit excepts when the filesystems are removed...
- 08:05 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- xianpao chen wrote:
> I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory... - 07:40 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory_limit = 16GB, no speci...
- 07:25 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Any reason you are using mds_cache_memory_limit = 16GB when you have memory to spare?
- 06:32 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- the "free -h" of the mds node(after restart the mds): ...
- 06:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- BTW, do you see any performance degradation on clients in general over the course and/or when the MDS is about to get...
- 05:27 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- I checked the session info, there is just one client which is holding ~1M caps. But that should not bother the MDS th...
11/22/2022
- 03:42 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- I happened to have a memory problem today, then I changed mds_session_cache_liveness_decay_rate to 150s, tried "ceph ...
- 11:19 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> Did you get to applying the suggested config?
Thanks for your suggestion, I will try it to... - 11:07 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Did you get to applying the suggested config?
- 10:19 AM Bug #57523: CephFS performance degredation in mountpoint
- Guys this can't be only a thing in our setup. Every time a connection puts more than a few GB into cephfs the perform...
- 01:19 AM Bug #58056: ceph-fuse - fuse failed to start on CentOS 7 host machine
- It seems passing invalidate flag ?...
11/21/2022
- 08:17 PM Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
- We might have found a major performance bug in the cephfs snapshot mirroring.
We already reported it to the mailing ... - 06:11 PM Support #38374: Crash when using cephfs as /var/lib/docker in devicemapper mode
- We're not using this kind of setup anymore and won't be troubleshooting further. We can close this for now, probably ...
- 01:53 PM Feature #58057: cephfs-top: enhance fstop tests to cover testing displayed data
- The Dashboard folks could point us to tools for testing console UI based apps.
- 11:30 AM Feature #58057 (Resolved): cephfs-top: enhance fstop tests to cover testing displayed data
- Right now the tests are pretty rudimentary. cephfs-top is a UI tool and writing tests can be a bit hard. Due to this ...
- 09:05 AM Bug #58056 (New): ceph-fuse - fuse failed to start on CentOS 7 host machine
- Hello,
We were previously using ceph v16.2.10 (docker container) with rook under kubernetes which was installed on... - 06:31 AM Bug #57014 (In Progress): cephfs-top: add an option to dump the computed values to stdout
11/18/2022
- 11:38 AM Bug #58028 (Fix Under Review): cephfs-top: Sorting doesn't work when the filesystems are removed ...
11/17/2022
- 12:05 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Hey,
Thanks for the update. You should try adjusting `mds_session_cache_liveness_decay_rate` to a lower value (def... - 10:17 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
- and another side note, the crash was seen when a directory pin was removed from rank-0 mds. Pinning it back again cea...
- 10:16 AM Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
- oh, and btw this was seen in ceph-16.2.8.
- 10:15 AM Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
- ...
- 09:21 AM Feature #55215 (Fix Under Review): mds: fragment directory snapshots
11/15/2022
- 01:49 PM Bug #58031 (Resolved): cephfs-top: sorting/limit excepts when the filesystems are removed and cre...
- This happens in the main branch. Please check.
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms... - 01:42 PM Bug #58000 (Fix Under Review): mds: switch submit_mutex to fair mutex for MDLog
- 01:41 PM Bug #58008 (Fix Under Review): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate...
- 01:41 PM Bug #58028 (Triaged): cephfs-top: Sorting doesn't work when the filesystems are removed and created
- 10:12 AM Bug #58028 (Resolved): cephfs-top: Sorting doesn't work when the filesystems are removed and created
- Sorting doesn't work in the following scenario
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms... - 11:08 AM Bug #58030 (Resolved): mds: avoid ~mdsdir's scrubbing and reporting damage health status
- We are supposed to handle the case of mdsdir, where we
are not having any backtrace actually.We should prevent the
... - 10:49 AM Bug #58029 (Fix Under Review): cephfs-data-scan: multiple data pools are not supported
- 10:46 AM Bug #58029 (Resolved): cephfs-data-scan: multiple data pools are not supported
- The tool cannot properly recover if a fs has extra data pools. We need access to all data pools on `scan_extents` ste...
11/14/2022
- 09:32 PM Fix #58023 (Pending Backport): mds: do not evict clients if OSDs are laggy
- Monitoring perf dumps from the MDS can sometimes show that OSDs are laggy, "objecter.op_laggy" and "objecter.osd_lagg...
- 01:27 PM Bug #58018 (Fix Under Review): mount.ceph: will fail with old kernels
- 10:09 AM Bug #58018 (Pending Backport): mount.ceph: will fail with old kernels
- ...
11/11/2022
- 02:11 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> xianpao chen wrote:
> > Venky Shankar wrote:
> > > Could you share the output of
> > >
>... - 01:02 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- xianpao chen wrote:
> Venky Shankar wrote:
> > Could you share the output of
> >
> > [...]
> >
> > Also, does... - 09:14 AM Bug #58008: mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
- When increasing filer_max_purge_ops on a pacific version mds, pq_executing_ops/pq_executing_ops_high_water of purge_q...
- 09:13 AM Bug #58008 (Resolved): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
- _calculate_ops relying on a config which can be modified on the fly will cause a bug. e.g.
# A file has 20 objects...
11/10/2022
- 08:18 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> BTW, are you *not* seeing any "oversized cache" warning for the MDS?
there is no "oversize... - 04:06 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- BTW, are you *not* seeing any "oversized cache" warning for the MDS?
- 02:42 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Do you have lots of small files and frequently scan them?
- 01:12 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> Have you tried running `heap release`?
yes,but it didn't seem to work. - 01:45 AM Bug #58000: mds: switch submit_mutex to fair mutex for MDLog
- From Patrick's comment in https://github.com/ceph/ceph/pull/44180#pullrequestreview-1174516711.
- 01:44 AM Bug #58000 (Resolved): mds: switch submit_mutex to fair mutex for MDLog
- The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the ...
11/09/2022
- 07:08 PM Feature #57090 (Fix Under Review): MDSMonitor,mds: add MDSMap flag to prevent clients from connec...
- 01:22 PM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Have you tried running `heap release`?
- 09:35 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu... - 09:23 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu... - 08:56 AM Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
- Could you share the output of...
11/07/2022
- 01:48 PM Bug #57985 (Triaged): mds: warning `clients failing to advance oldest client/flush tid` seen with...
- 09:06 AM Bug #57985 (Pending Backport): mds: warning `clients failing to advance oldest client/flush tid` ...
- https://bugzilla.redhat.com/show_bug.cgi?id=2134709
Generally seen when the MDS is heavily loaded with I/Os. Inter...
11/04/2022
- 07:48 PM Bug #49132: mds crashed "assert_condition": "state == LOCK_XLOCK || state == LOCK_XLOCKDONE",
- Alternative fix is available at https://github.com/ceph/ceph/pull/48743
- 08:54 AM Backport #57974 (In Progress): pacific: cephfs-top: make cephfs-top display scrollable like top
- 08:46 AM Backport #57974 (Resolved): pacific: cephfs-top: make cephfs-top display scrollable like top
- https://github.com/ceph/ceph/pull/48734
- 03:51 AM Backport #57971 (Resolved): pacific: cephfs-top: new options to limit and order-by
- https://github.com/ceph/ceph/pull/49303
- 03:50 AM Backport #57970 (Resolved): quincy: cephfs-top: new options to limit and order-by
- https://github.com/ceph/ceph/pull/50151
- 03:25 AM Feature #55121 (Pending Backport): cephfs-top: new options to limit and order-by
11/03/2022
- 12:45 PM Feature #44455 (In Progress): cephfs: add recursive unlink RPC
- 09:30 AM Feature #57090 (In Progress): MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
- 07:34 AM Feature #57090: MDSMonitor,mds: add MDSMap flag to prevent clients from connecting
- Patrick Donnelly wrote:
> Dhairya, status on this?
Hi Patrick, i'm on this completely now. Will try bring somethi... - 09:20 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
- If you're running into this bug after upgrading from Pacific to Quincy, you can manually delete the legacy schedule D...
- 08:49 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
- {"log":"debug 2022-11-03T08:38:12.502+0000 7f46270f5700 -1 mgr load Failed to construct class in 'snap_schedule'\n","...
- 08:46 AM Bug #56270: crash: File "mgr/snap_schedule/module.py", in __init__: self.client = SnapSchedClient...
- How to fix it?
Also available in: Atom