Bug #58744
closedqa: intermittent nfs test failures at nfs cluster creation
100%
Description
While working on https://github.com/ceph/ceph/pull/49460, I found any random test would fail with "AssertionError: NFS Ganesha cluster deployment failed". In order to confirm that it's not my code doing something, i ran test_nfs against main and found this issue still coming up. Running 'nfs clustur ls' before running 'nfs cluster create test' cmd, the output would show cluster 'test' existent while the previous test does delete it and this was confirmed by running 'nfs cluster ls' after every cluster deletion cmd.
http://pulpito.front.sepia.ceph.com/dparmar-2023-02-13_15:52:09-orch:cephadm-wip-58228-distro-default-smithi/7170593/
http://pulpito.front.sepia.ceph.com/dparmar-2023-02-15_09:23:57-orch:cephadm-main-distro-default-smithi/7174810/
Running the command set(check nfs server status -> nfs cluster create test -> check status) in a loop (max three iteration) fixes the issue, i think it takes sometime to update the cluster data and because the time gap between running cluster deletion - cluster creation is too small, its sometimes a bit slow to update the cluster data and flush the old data in time.