Project

General

Profile

Bug #61950

mds/OpenFileTable: match MAX_ITEMS_PER_OBJ does not honor osd_deep_scrub_large_omap_object_key_threshold

Added by Vikhyat Umrao 8 months ago. Updated 3 months ago.

Status:
Need More Info
Priority:
High
Assignee:
-
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Support
Tags:
Backport:
reef,quincy,pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The changes implemented in [1] should make sure that we never have an openfiletable objects omap keys above osd_deep_scrub_large_omap_object_key_threshold( default 200000) but in Ceph version: 14.2.22-110.el8cp (2e0d97dbe192cca7419bbf3f8ee6b7abb42965c4) nautilus

We see the following message in `ceph -s` and the following log in cluster logs:

LARGE_OMAP_OBJECTS

2023-07-09 10:42:34.177821 osd.123 (osd.123) 16631 : cluster [WRN] Large omap object found. Object: 3:2adbd678:::mds0_openfiles.9:head PG: 3.1e6bdb54 (3.154) Key count: 200001 Size (bytes): 12824872

In this situation the mds0_openfiles.9 object has just 1 key extra from the threshold(200000).

[1] Old Tracker and PR:
https://tracker.ceph.com/issues/42515
https://github.com/ceph/ceph/pull/31232


Related issues

Related to CephFS - Bug #42515: fs: OpenFileTable object shards have too many k/v pairs Resolved

History

#1 Updated by Venky Shankar 7 months ago

Ok, so an off-by-one error - should be relatively easy to figure...

#2 Updated by Patrick Donnelly 7 months ago

  • Category set to Correctness/Safety
  • Status changed from New to In Progress
  • Assignee set to Patrick Donnelly
  • Priority changed from Normal to High
  • Target version set to v19.0.0
  • Source set to Support
  • Backport set to reef,quincy,pacific
  • Component(FS) MDS added

#3 Updated by Venky Shankar 7 months ago

Patrick - I can take this one if you haven't started on it yet.

#4 Updated by Patrick Donnelly 7 months ago

Venky Shankar wrote:

Patrick - I can take this one if you haven't started on it yet.

I have started on it. Should be quick.

#5 Updated by Patrick Donnelly 7 months ago

  • Related to Bug #42515: fs: OpenFileTable object shards have too many k/v pairs added

#6 Updated by Patrick Donnelly 7 months ago

Venky Shankar wrote:

Patrick - I can take this one if you haven't started on it yet.

https://github.com/ceph/ceph/pull/52415#issuecomment-1632803909

I haven't found the cause. If you'd like to have a stab at it be my guest. :)

#7 Updated by Patrick Donnelly 7 months ago

  • Pull request ID set to 52415

#8 Updated by Venky Shankar 6 months ago

Patrick Donnelly wrote:

Venky Shankar wrote:

Patrick - I can take this one if you haven't started on it yet.

https://github.com/ceph/ceph/pull/52415#issuecomment-1632803909

I haven't found the cause. If you'd like to have a stab at it be my guest. :)

I'll have a look. Assigning to myself.

#9 Updated by Venky Shankar 6 months ago

  • Assignee changed from Patrick Donnelly to Venky Shankar

#10 Updated by Patrick Donnelly 3 months ago

  • Status changed from In Progress to Need More Info
  • Assignee deleted (Venky Shankar)
  • Pull request ID deleted (52415)

cannot reproduce

Also available in: Atom PDF