Bug #45834
closedcephadm: "fs volume create cephfs" overwrites existing placement specification
0%
Description
The orchestrator behaves unexpectedly with apply mds. Consider the following:
I have a ceph cluster running and want cephfs
ceph orch apply mds cephfs 3
gives me 3 MDS daemons but no cephfs. The "cephfs" argument doesn't seem to do anything.
Then I created an fs vie
ceph fs volume create cephfs
. This reduces the number of MDS daemons to 2.
A subsequent call to
ceph orch apply mds cephfs 3
recreates the initially desired state.
Updated by Sebastian Wagner almost 4 years ago
Easy solution would be to validate the existence of this CephFS, before starting the MDS daemons.
Updated by Sebastian Wagner almost 4 years ago
- Tracker changed from Bug to Documentation
Updated by Sebastian Wagner about 3 years ago
- Tracker changed from Documentation to Bug
- Project changed from Orchestrator to CephFS
- Subject changed from cephadm: ceph orch apply mds <fsname> <n> to cephadm: "fs volume create cephfs" overwrites existing placement specification
- Description updated (diff)
- Regression set to No
- Severity set to 3 - minor
- Component(FS) mgr/volumes added
The problem is caused by volumes.fs.fs_util.create_mds replacing the existing placement spec with None and thus is overwriting it with the default.
The orchestrator API is declarative. We should probably avoid applying an MDS service here, if it already exists.
Updated by Patrick Donnelly about 3 years ago
- Category set to Administration/Usability
- Priority changed from Normal to Urgent
- Target version set to v17.0.0
- Source set to Development
- Tags deleted (
ux) - Backport set to pacific
- Component(FS) mgr/mds_autoscaler added
Updated by Patrick Donnelly about 3 years ago
- Status changed from New to Triaged
- Assignee set to Milind Changire
Updated by Milind Changire about 3 years ago
Jan,
Could you attach the fs dump just after creating 'cephfs'
eg.
$ ceph fs dump --format=json-pretty
I suspect that after creating the 'cephfs' file-system, max_mds is set to 1 and standby_count_wanted is also set to 1. So, in total there would be 2 mds instances running. Which is what you get to see in your case.
I'm not sure how the orch commands affect the fsmap when "ceph orch apply mds cephfs 3" is executed after the 'cephfs' fs is created. But the MDS Autoscaler only responds to fsmap changes to max_mds and standby_count_wanted tunables.
The very first "ceph orch apply mds cephfs 3" could be a no-op, as there wasn't any fs named 'cephfs' existing when that command was executed. Sebastian could clarify it further.
The mds count as 2 could be the default behavior on new fs creation. Need to confirm it with your "fs dump" output.
Updated by Milind Changire over 1 year ago
- Status changed from Triaged to Closed
- Priority changed from Urgent to Low
This behavior may be because the placement spec needs a valid fs to apply the spec to.
closing tracker for now
lowering priority to low