Project

General

Profile

Feature #23979

Limit pg log length during recovery/backfill so that we don't run out of memory.

Added by David Zafman 7 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
Start date:
05/02/2018
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
mimic, luminous
Reviewed:
Affected Versions:
Component(RADOS):
Pull request ID:

Description

This means if there's another failure, we'll need to restart backfill or go from recovery to backfill, but that's better than running out of memory.

Treating osd_max_pg_log_entries as a hard cap should be sufficient.


Related issues

Related to RADOS - Bug #25198: FAILED assert(trim_to <= info.last_complete) in PGLog::trim() Resolved 07/31/2018
Related to RADOS - Bug #26868: PGLog.cc: saw valgrind issues while accessing complete_to->version Resolved 08/06/2018
Copied to RADOS - Backport #24988: luminous: Limit pg log length during recovery/backfill so that we don't run out of memory. Resolved
Copied to RADOS - Backport #24989: mimic: Limit pg log length during recovery/backfill so that we don't run out of memory. Resolved

History

#1 Updated by David Zafman 7 months ago

  • Source set to Development
  • Backport set to luminous

#2 Updated by David Zafman 7 months ago

  • Assignee changed from David Zafman to Josh Durgin

#3 Updated by Josh Durgin 7 months ago

  • Assignee changed from Josh Durgin to Neha Ojha

Initial testing is referenced here: https://github.com/ceph/ceph/pull/21508

#4 Updated by Josh Durgin 7 months ago

  • Backport changed from luminous to mimic, luminous

#5 Updated by Neha Ojha 5 months ago

  • Status changed from Verified to Need Review

#6 Updated by Josh Durgin 5 months ago

  • Status changed from Need Review to Pending Backport

#7 Updated by Nathan Cutler 5 months ago

  • Copied to Backport #24988: luminous: Limit pg log length during recovery/backfill so that we don't run out of memory. added

#8 Updated by Nathan Cutler 5 months ago

  • Copied to Backport #24989: mimic: Limit pg log length during recovery/backfill so that we don't run out of memory. added

#9 Updated by Neha Ojha 4 months ago

  • Related to Bug #25198: FAILED assert(trim_to <= info.last_complete) in PGLog::trim() added

#10 Updated by Neha Ojha 4 months ago

  • Related to Bug #26868: PGLog.cc: saw valgrind issues while accessing complete_to->version added

#11 Updated by Nathan Cutler 3 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF