Project

General

Profile

Bug #37727

ceph-deploy mon destroy making ceph unresponsive

Added by Joao Aguiar over 5 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-deploy
Crash signature (v1):
Crash signature (v2):

Description

I started by creating a cluster with one monitor with "create-deploy new"… "create-initial”...
And ended up with ceph,conf like:
...
mon_initial_members = node0
mon_host = 10.2.2.2
….

Later I try to deploy a new monitor (ceph-deploy mon create node1), wait for it to get in quorum and then destroy the node0 (ceph-deploy mon destroy node0).

Result: Ceph gets unresponsive.

Some considerations:
1) node0, is the only in mon_initial_members, also he is the leader.
2) before deleting node0, node1 state is peon but after node0 deletions goes to probing
3) After node1 deletion I can still see node0 in the monmap

History

#1 Updated by Alfredo Deza over 5 years ago

  • Status changed from New to Rejected

You are describing the behavior of having an uneven number of monitors (2) and removing one, which is advised against. See: http://docs.ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-monitors

Specifically this section:

For instance, on a 2 monitor deployment, no failures can be tolerated in order to maintain a quorum

#2 Updated by Joao Aguiar over 5 years ago

I also tried adding 3 additional nodes, and destroy the first one....same problem.....

#3 Updated by Alfredo Deza over 5 years ago

  • Status changed from Rejected to New

Also available in: Atom PDF