Project

General

Profile

Actions

Bug #6797

closed

ceph osd out does not migrate properly

Added by Zoltan Arnold Nagy over 10 years ago. Updated about 10 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

ok, the subject might be misleading; here is what is happening:

- ceph osd out $id
- wait until migration finishes
- initctl stop ceph-osd id=$id to mark it down
- ceph osd crush remove osd.$id causes a migration again, even tho there wasn't any data on that disk at this point

according to #ceph@irc, this is a known issue; I currently run into this on dumpling, not sure about the status on emperor.

Actions #1

Updated by Andrey Korolyov over 10 years ago

+1, migration overhead may be reduced by doing these actions in a short order, but generally it introduces two peering processes instead of one which is definitely should be avoided

Actions #2

Updated by Greg Farnum over 10 years ago

Yeah, odd as it seems this is actually a user experience conflict — marking an OSD out does not change the CRUSH weight of the bucket it resides in, but removing it from the hierarchy does. The expectation being that you're likely to replace an out OSD, so you want to keep recovery local instead of adding a bunch of network traffic.
That doesn't mean it might not make sense to change! But we'd like to not just make the change but come up with a better model that covers all the cases instead of the first one (not even necessarily the most common one) people hit. :/

Actions #3

Updated by Andrey Korolyov over 10 years ago

I cannot agree that the question is just about user experience. Drive replacement almost always don`t mean any remapping in the middle, so 'out' action should not happen if expected replacement will not take very long time and out -> down flow will happen only on a matter of permanent rebalance. Speaking so, issuing rebalance twice is a bit more evil,

Actions #4

Updated by Dmitry Borodaenko over 10 years ago

Arguably the most common use case is drive replacement, and Ceph documentation recommends setting noout flag when you want to avoid rebalancing during maintenance:
http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-osd/#stopping-w-out-rebalancing

This means that if you're actually taking the OSD out, you plan on removing from the cluster altogether, not just replacing it. Is there a use case I'm missing?

Actions #5

Updated by Sage Weil about 10 years ago

  • Status changed from New to Won't Fix
Actions

Also available in: Atom PDF