Bug #46176
mgr/dashboard: NFS list timeout error
0%
Description
The NFS overview list is returning a timeout error if the related pool contains too many objects. The error seems to occur because of the following code lines: https://github.com/ceph/ceph/blob/nautilus/src/pybind/mgr/dashboard/services/ganesha.py#L795-L814 or https://github.com/ceph/ceph/blob/master/src/pybind/mgr/nfs/export.py#L295-L303
The for-loop needs to iterate all objects returned by `ioctx.list_objects()` which may take a lot of time if the pool contains many objects. Would it be possible to get the information in another way than iterating the whole list of objects?
Related issues
History
#1 Updated by Tatjana Dehler almost 4 years ago
- Description updated (diff)
#2 Updated by Tatjana Dehler over 3 years ago
If looping through all the objects in a pool (https://github.com/ceph/ceph/blob/master/src/pybind/mgr/dashboard/services/ganesha.py#L798-L817) is the real issue, there might be an idea for environments deployed by the Orchestrator:
If it's possible to get a list of the config files e.g. `conf-x` from the Orchestrator, then we might be able to get a list of the export names (export-y) from these files. We can then use the export name to look it up directly by calling the `Ioctx.read` function: https://docs.ceph.com/docs/master/rados/api/python/#rados.Ioctx.read . That way we can get the exports directly by name without looping through all objects of the pool.
#3 Updated by Kiefer Chang over 3 years ago
Thanks @Tatjana, the idea is good and we can avoid the object enumeration with it. Here are some problems we might encounter:
- Users can create exports without associating any daemons. That is, `conf-<daemon>` object has no `%url ...` entry, but there still are some `export-<id>` objects. In this case, those exports become orphans and are not visible in the Dashboard.
- How to get all daemons in order to resolve those `conf-<daemon>` objects if the Ceph cluster is deployed by downstream tools? Even we only support this after Octopus, we can't assume the cluster is deployed by the Orchestrator.
- Dashboard can't manage daemons deployed outside the Orchestrator.
#4 Updated by Kiefer Chang over 3 years ago
In the comment of issue #46327 and some chats, CLI in `volumes` module is also suggested to be used by Dashboard.
In this case, we can assure there is only one way to manage NFS Ganesha exports.
The module provides an interface to manage NFS clusters. It:
- manipulates RADOS objects for exports like the Dashboard does.
- asks Orchestrator to create NFS service if it needs.
- Create FS volumes and subvolumes
- Currently it can't manage RGW exports. (Thanks Varsha for the info).
See the following links for more information.
[1] https://github.com/ceph/ceph/blob/0afbdca6932ca77aefac3dc3c2ee5e165429a8c9/src/pybind/mgr/volumes/module.py#L257-L334
[2] https://github.com/ceph/ceph/blob/master/doc/cephfs/fs-nfs-exports.rst
[3] https://pad.ceph.com/p/nfs
The implementation suffers the same problem if there are huge amounts of objects in a pool. Because it iterates all objects to get `conf-xxx` and `export-xxx` objects.
#5 Updated by Tatjana Dehler over 3 years ago
- Related to Documentation #46530: mgr/dashboard: NFS ganesha object organization added
#6 Updated by Varsha Rao over 3 years ago
Kiefer Chang wrote:
In the comment of issue #46327 and some chats, CLI in `volumes` module is also suggested to be used by Dashboard.
In this case, we can assure there is only one way to manage NFS Ganesha exports.
The module provides an interface to manage NFS clusters. It:
- manipulates RADOS objects for exports like the Dashboard does.
- asks Orchestrator to create NFS service if it needs.
- Create FS volumes and subvolumes
- Currently it can't manage RGW exports. (Thanks Varsha for the info).See the following links for more information.
[1] https://github.com/ceph/ceph/blob/0afbdca6932ca77aefac3dc3c2ee5e165429a8c9/src/pybind/mgr/volumes/module.py#L257-L334
[2] https://github.com/ceph/ceph/blob/master/doc/cephfs/fs-nfs-exports.rst
[3] https://pad.ceph.com/p/nfsThe implementation suffers the same problem if there are huge amounts of objects in a pool. Because it iterates all objects to get `conf-xxx` and `export-xxx` objects.
No, it does not as volume's nfs interface creates a separate pool for nfs-ganesha conf, export and grace objects. In the same pool cephfs or any other data is not saved. We assume not many exports will be created.
#7 Updated by Ernesto Puerta almost 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 144 to Component - NFS
#8 Updated by Volker Theile almost 2 years ago
- Description updated (diff)
- Priority changed from Normal to High
- Affected Versions v14.2.9, v18.0.0 added
#9 Updated by Ernesto Puerta almost 2 years ago
Is this still an issue in Pacific/Quincy NFS? It looks like related to the old NFS way. If no longer happening, can we close it?
#10 Updated by Tatjana Dehler almost 2 years ago
- Status changed from New to Closed
Ernesto Puerta wrote:
Is this still an issue in Pacific/Quincy NFS? It looks like related to the old NFS way. If no longer happening, can we close it?
Oh! Good question. I can't tell if it's still an issue in Pacific or Quincy. I think we can close and re-open it if needed.