Project

General

Profile

Bug #10012

Configuration parameters not picked up outside of the [global] section.

Added by Christian Balzer over 9 years ago. Updated almost 3 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
-
Category:
common
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Certain osd* and radosgw* parameters are not picked up outside of the [global] section in the ceph.conf file, as per the following thread:
https://www.mail-archive.com/ceph-users@lists.ceph.com/msg13859.html

This was confirmed for all firefly versions and I expect this to be still present in giant.

Aside from fixing this bug the following things come to mind:

1. Ceph should report any issues (ignored lines, typos) with the config file during startup into the respective log file(s) at normal debug levels.
2. Parameters of a certain type should ALWAYS work inside their respective section, so any osd* param ought to work as expected in the [osd] section.

History

#1 Updated by Chris Armstrong over 9 years ago

A Deis user confirmed this still happens on Giant. For him, the radosgw was only ever talking to the single monitor in the mon initial members clause. His ceph.conf looked like this:

root@deis-store-gateway:/# more /etc/ceph/ceph.conf
[global]
fsid = fc0e2e09-ade3-4ff6-b23e-f789775b2515
mon initial members = nodo-3
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 3
osd pool default min_size = 1
osd pool default pg_num = 128
osd pool default pgp_num = 128
osd recovery delay start = 15
log file = /dev/stdout
mon_clock_drift_allowed = 1

[mon.nodo-1]
host = nodo-1
mon addr = 192.168.2.200:6789

[mon.nodo-2]
host = nodo-2
mon addr = 192.168.2.201:6789

[mon.nodo-3]
host = nodo-3
mon addr = 192.168.2.202:6789

[client.radosgw.gateway]
host = deis-store-gateway
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /dev/stdout

If nodo-3 was down, radosgw started spewing errors even though two other monitors were healthy. When we instead removed the various monitor configuration blocks and made the config file look like this:

root@deis-store-gateway:/# more /etc/ceph/ceph.conf
[global]
fsid = fc0e2e09-ade3-4ff6-b23e-f789775b2515
mon initial members = nodo-3
mon host = nodo-1, nodo-2, nodo-3
mon addr = 192.168.2.200:6789, 192.168.2.201:6789, 192.168.2.202:6789
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd pool default size = 3
osd pool default min_size = 1
osd pool default pg_num = 128
osd pool default pgp_num = 128
osd recovery delay start = 15
log file = /dev/stdout
mon_clock_drift_allowed = 1

[client.radosgw.gateway]
host = deis-store-gateway
keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /dev/stdout

Everything worked fine. See https://github.com/deis/deis/issues/2711 for more details.

#2 Updated by Greg Farnum about 9 years ago

  • Category set to common
  • Priority changed from Normal to High
  • Source changed from other to Community (user)

This is continuing to cause trouble for users in various ways. Here's a report of a value in [global] not impacting the monitors until moved to the [mon] section: https://www.mail-archive.com/ceph-users@lists.ceph.com/msg16514.html

#3 Updated by Sage Weil almost 3 years ago

  • Status changed from New to Can't reproduce

Also available in: Atom PDF