Project

General

Profile

Bug #46176

mgr/dashboard: NFS list timeout error

Added by Tatjana Dehler 4 months ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
dashboard/nfs-ganesha
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

The NFS overview list is returning a timeout error if the related pool contains too many objects. The error seems to occur because of the following code lines: https://github.com/ceph/ceph/blob/master/src/pybind/mgr/dashboard/services/ganesha.py#L798-L817
The for-loop needs to iterate all objects returned by `ioctx.list_objects()` which may take a lot of time if the pool contains many objects. Would it be possible to get the information in another way than iterating the whole list of objects?


Related issues

Related to mgr - Documentation #46530: mgr/dashboard: NFS ganesha object organization Resolved

History

#1 Updated by Tatjana Dehler 4 months ago

  • Description updated (diff)

#2 Updated by Tatjana Dehler 4 months ago

If looping through all the objects in a pool (https://github.com/ceph/ceph/blob/master/src/pybind/mgr/dashboard/services/ganesha.py#L798-L817) is the real issue, there might be an idea for environments deployed by the Orchestrator:

If it's possible to get a list of the config files e.g. `conf-x` from the Orchestrator, then we might be able to get a list of the export names (export-y) from these files. We can then use the export name to look it up directly by calling the `Ioctx.read` function: https://docs.ceph.com/docs/master/rados/api/python/#rados.Ioctx.read . That way we can get the exports directly by name without looping through all objects of the pool.

#3 Updated by Kiefer Chang 4 months ago

Thanks @Tatjana, the idea is good and we can avoid the object enumeration with it. Here are some problems we might encounter:

  • Users can create exports without associating any daemons. That is, `conf-<daemon>` object has no `%url ...` entry, but there still are some `export-<id>` objects. In this case, those exports become orphans and are not visible in the Dashboard.
  • How to get all daemons in order to resolve those `conf-<daemon>` objects if the Ceph cluster is deployed by downstream tools? Even we only support this after Octopus, we can't assume the cluster is deployed by the Orchestrator.
  • Dashboard can't manage daemons deployed outside the Orchestrator.

#4 Updated by Kiefer Chang 4 months ago

In the comment of issue #46327 and some chats, CLI in `volumes` module is also suggested to be used by Dashboard.

In this case, we can assure there is only one way to manage NFS Ganesha exports.

The module provides an interface to manage NFS clusters. It:
- manipulates RADOS objects for exports like the Dashboard does.
- asks Orchestrator to create NFS service if it needs.
- Create FS volumes and subvolumes
- Currently it can't manage RGW exports. (Thanks Varsha for the info).

See the following links for more information.

[1] https://github.com/ceph/ceph/blob/0afbdca6932ca77aefac3dc3c2ee5e165429a8c9/src/pybind/mgr/volumes/module.py#L257-L334
[2] https://github.com/ceph/ceph/blob/master/doc/cephfs/fs-nfs-exports.rst
[3] https://pad.ceph.com/p/nfs

The implementation suffers the same problem if there are huge amounts of objects in a pool. Because it iterates all objects to get `conf-xxx` and `export-xxx` objects.

#5 Updated by Tatjana Dehler 3 months ago

#6 Updated by Varsha Rao 3 months ago

Kiefer Chang wrote:

In the comment of issue #46327 and some chats, CLI in `volumes` module is also suggested to be used by Dashboard.

In this case, we can assure there is only one way to manage NFS Ganesha exports.

The module provides an interface to manage NFS clusters. It:
- manipulates RADOS objects for exports like the Dashboard does.
- asks Orchestrator to create NFS service if it needs.
- Create FS volumes and subvolumes
- Currently it can't manage RGW exports. (Thanks Varsha for the info).

See the following links for more information.

[1] https://github.com/ceph/ceph/blob/0afbdca6932ca77aefac3dc3c2ee5e165429a8c9/src/pybind/mgr/volumes/module.py#L257-L334
[2] https://github.com/ceph/ceph/blob/master/doc/cephfs/fs-nfs-exports.rst
[3] https://pad.ceph.com/p/nfs

The implementation suffers the same problem if there are huge amounts of objects in a pool. Because it iterates all objects to get `conf-xxx` and `export-xxx` objects.

No, it does not as volume's nfs interface creates a separate pool for nfs-ganesha conf, export and grace objects. In the same pool cephfs or any other data is not saved. We assume not many exports will be created.

Also available in: Atom PDF