Project

General

Profile

Feature #6143

OSD: kill filestore_xattr_use_omap, leave it enabled forever, adjust xattr threshholds based on fs by default

Added by Samuel Just almost 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
08/28/2013
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
User Impact:
Affected Versions:
Release:
Needs Doc:
No

Description

Otherwise you might get a corrupt osd.


Related issues

Duplicated by Ceph - Tasks #6184: filestore should record if filestore_xattr_use_omap has ever been enabled and never disable it thereafter Closed 08/30/2013

Associated revisions

Revision dc0dfb9e (diff)
Added by David Zafman over 3 years ago

common,os: Remove filestore_xattr_use_omap option

Now we operate just like when this was set to true

Fixes: #6143

Signed-off-by: David Zafman <>

Revision bb9b9c89 (diff)
Added by David Zafman over 3 years ago

common, os: Perform xattr handling based on detected fs type

In FileStore::_detect_fs() store discovered filesystem type in m_fs_type
Add per-filesystem filestore_max_inline_xattr_size_* variants
Add per-filesystem filestore_max_inline_xattrs_* variants
New function set_xattr_limits_via_conf()
Set m_filestore_max_inline_xattr_size based on override or fs type
Set m_filestore_max_inline_xattrs based on override or fs type
Handle conf change of any relevant value by calling set_xattr_limits_via_conf()
Change filestore_max_inline_xattr_size to override if non-zero
Change filestore_max_inline_xattrs to override if non-zero

Fixes: #6143

Signed-off-by: David Zafman <>

Revision 3e473d42 (diff)
Added by David Zafman over 3 years ago

common,os: Remove filestore_xattr_use_omap option

Now we operate just like when this was set to true

Fixes: #6143

Signed-off-by: David Zafman <>
(cherry picked from commit dc0dfb9e01d593afdd430ca776cf4da2c2240a20)

History

#1 Updated by Sage Weil almost 4 years ago

  • translation missing: en.field_story_points set to 2.00

#2 Updated by Ian Colle almost 4 years ago

  • Target version changed from v0.69 to v0.70

#3 Updated by Ian Colle almost 4 years ago

  • Assignee set to David Zafman

#4 Updated by Samuel Just over 3 years ago

  • Target version changed from v0.70 to v0.71

#5 Updated by Samuel Just over 3 years ago

  • Subject changed from OSD: remember if filestore_xattr_use_omap and always enable in the future to OSD: kill filestore_xattr_use_omap, leave it enabled forever, adjust xattr threshholds based on fs by default

#6 Updated by David Zafman over 3 years ago

  • Status changed from New to Need Review

#7 Updated by David Zafman over 3 years ago

  • Status changed from Need Review to Resolved

5258c9c4e5bd61e6b1a1fdbdb781b9dee702fda3

#8 Updated by Florian Haas over 3 years ago

This issue has caused data corruption in at least one user (Inktank PS is well aware of this).

Since
  • https://github.com/ceph/ceph/commit/dc0dfb9e01d593afdd430ca776cf4da2c2240a20 was never backported to Dumpling,
  • there is no known fix for when the issue does occur (which is possible with rgw or a large rados xattr set), other than manual filestore hackery,
  • the proposed workaround (to keep the problem from happening in the first place) is to always set filestore_xattr_use_omap,
  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,
  • Dumpling is the designated stable/Enterprise/better-than-sliced-bread release,
  • sane defaults should be a primary concern,

I suggest that the default in Dumpling be flipped. Not sure if that counts as a backport.

AFAICS, that's a one-line patch in src/common/config_opts.h.

#9 Updated by Sage Weil over 3 years ago

Florian Haas wrote:

  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,

We haven't confirmed, but it is expected that this will degrade performance on some workloads, especially when leveldb has a bunch of data in it. Conveniently, those users are probably the ones using rgw who would need this change.

In any case, data safety trumps performance, so this will go into dumpling shortly along with some noise in the release notes about the change.

#10 Updated by Florian Haas over 3 years ago

  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,

We haven't confirmed, but it is expected that this will degrade performance on some workloads, especially when leveldb has a bunch of data in it. Conveniently, those users are probably the ones using rgw who would need this change.

OK, let me put it this way then:

  • filestore_xattr_use_omap can be enabled in a running cluster without any known side effects that would outweigh the impact of multiple OSDs crashing with no automated mode of repair or recovery.

In any case, data safety trumps performance, so this will go into dumpling shortly along with some noise in the release notes about the change.

Agreed on the first part. Thanks on the second.

Also available in: Atom PDF