Project

General

Profile

Actions
Copy link

Feature #6143

closed

OSD: kill filestore_xattr_use_omap, leave it enabled forever, adjust xattr threshholds based on fs by default

Added by Samuel Just over 10 years ago. Updated about 10 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
David Zafman
Category:
-
Target version:
v0.71
% Done:

0%

Source:
other
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

Otherwise you might get a corrupt osd.

Related issues 1 (0 open — 1 closed)

Has duplicate Ceph - Tasks #6184: filestore should record if filestore_xattr_use_omap has ever been enabled and never disable it thereafterClosed08/30/2013

Actions
Actions
Copy link
 #1

Updated by Sage Weil over 10 years ago

  • Translation missing: en.field_story_points set to 2.00
Actions
Copy link
 #2

Updated by Ian Colle over 10 years ago

  • Target version changed from v0.69 to v0.70
Actions
Copy link
 #3

Updated by Ian Colle over 10 years ago

  • Assignee set to David Zafman
Actions
Copy link
 #4

Updated by Samuel Just over 10 years ago

  • Target version changed from v0.70 to v0.71
Actions
Copy link
 #5

Updated by Samuel Just over 10 years ago

  • Subject changed from OSD: remember if filestore_xattr_use_omap and always enable in the future to OSD: kill filestore_xattr_use_omap, leave it enabled forever, adjust xattr threshholds based on fs by default
Actions
Copy link
 #6

Updated by David Zafman over 10 years ago

  • Status changed from New to Fix Under Review
Actions
Copy link
 #7

Updated by David Zafman over 10 years ago

  • Status changed from Fix Under Review to Resolved

5258c9c4e5bd61e6b1a1fdbdb781b9dee702fda3

Actions
Copy link
 #8

Updated by Florian Haas about 10 years ago

This issue has caused data corruption in at least one user (Inktank PS is well aware of this).

Since
  • https://github.com/ceph/ceph/commit/dc0dfb9e01d593afdd430ca776cf4da2c2240a20 was never backported to Dumpling,
  • there is no known fix for when the issue does occur (which is possible with rgw or a large rados xattr set), other than manual filestore hackery,
  • the proposed workaround (to keep the problem from happening in the first place) is to always set filestore_xattr_use_omap,
  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,
  • Dumpling is the designated stable/Enterprise/better-than-sliced-bread release,
  • sane defaults should be a primary concern,

I suggest that the default in Dumpling be flipped. Not sure if that counts as a backport.

AFAICS, that's a one-line patch in src/common/config_opts.h.

Actions
Copy link
 #9

Updated by Sage Weil about 10 years ago

Florian Haas wrote:

  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,

We haven't confirmed, but it is expected that this will degrade performance on some workloads, especially when leveldb has a bunch of data in it. Conveniently, those users are probably the ones using rgw who would need this change.

In any case, data safety trumps performance, so this will go into dumpling shortly along with some noise in the release notes about the change.

Actions
Copy link
 #10

Updated by Florian Haas about 10 years ago

  • filestore_xattr_use_omap can be enabled in a running cluster with no known side effects,

We haven't confirmed, but it is expected that this will degrade performance on some workloads, especially when leveldb has a bunch of data in it. Conveniently, those users are probably the ones using rgw who would need this change.

OK, let me put it this way then:

  • filestore_xattr_use_omap can be enabled in a running cluster without any known side effects that would outweigh the impact of multiple OSDs crashing with no automated mode of repair or recovery.

In any case, data safety trumps performance, so this will go into dumpling shortly along with some noise in the release notes about the change.

Agreed on the first part. Thanks on the second.

Actions
Copy link

Also available in: Atom PDF