Project

General

Profile

Feature #8667

CRUSH: override 'crush location' in the ceph config so that modifications to OSD position in the CRUSH map stick

Added by Christina Meno about 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Category:
Backend (REST API)
Target version:
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:

Description

TBD

History

#1 Updated by Christina Meno about 7 years ago

  • Target version set to 1.3 backlog

#2 Updated by Christina Meno almost 7 years ago

  • Subject changed from override 'crush location' in the ceph config so that modifications to OSD position in the CRUSH map stick to CRUSH: override 'crush location' in the ceph config so that modifications to OSD position in the CRUSH map stick

#3 Updated by Christina Meno over 6 years ago

  • Target version changed from 1.3 backlog to 1.3-dev12

#4 Updated by Christina Meno over 6 years ago

I cannot figure out what was intended when I created this ticket.

was it:
[osd.0]
osd host = osd-host-a

[osd.1]
osd host = osd-host-b

That doesn't make sense because it seems like osd id is chosen at creation time. ceph-deploy/ceph-disk don't seem to honor it.

#5 Updated by John Spray over 6 years ago

In normal operation, there are two sources of modifications to the crush map:

A is what Calamari currently does. B is designed to make OSDs somewhat hot pluggable between hosts.

The trouble is that if A is doing something clever like assigning an SSD into a different CRUSH root than the default, then B will come and trample all over it the next time the OSD starts. This ticket is about solving that problem.

The crush_location setting is a way of telling the B path (ceph-crush-location) "Instead of inferring your location from $hostname, set it to this".

#6 Updated by Christina Meno over 6 years ago

So the way I see it there are several ways to change this:

A. set the value to false.
B. set the value to a calamari-specific script

1. in the package post-hook type scripts:
- on install of calamari-server set A or B
- on removal of calamari-server restore to previous value, or do nothing

2. when cthulhu detects that the api is making a change to the crush map that requires this and the setting is not false
- it sets A or B

3. we helpfully explain all the details and allow the user to choose.

What would the calamari-specific script do?
It could just read the crush map and offer a startup stasis

I recommend going with 1A it is the simple solution, it is least surprising, and requires the least effort.

#7 Updated by Christina Meno over 6 years ago

  • Status changed from New to Need More Info

after talking to Dan the solution that seems best is:

Have calamari set "osd crush location hook" to a script that asks either calamari or the cluster for the OSDs last known location in the CRUSH map if this is a new OSD fallback to a sensible default e.g. the behavior as it "osd update on start" were true

The thing I like most about this approach is that we edit the config file one time.

#8 Updated by Christina Meno over 6 years ago

  • Target version changed from 1.3-dev12 to 1.3-dev13

#9 Updated by Christina Meno over 6 years ago

  • Status changed from Need More Info to In Progress

This ticket now depends on http://tracker.ceph.com/issues/10844 to work.
I can complete the code, but will be unable to have passing test code until that ticket is in place.

#10 Updated by Christina Meno over 6 years ago

  • Status changed from In Progress to Fix Under Review
  • Assignee set to Christina Meno

#11 Updated by Christina Meno over 6 years ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF