Project

General

Profile

Subtask #5046

Feature #4929: Erasure encoded placement group

Factor out PG logs, PG missing

Added by Loic Dachary over 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
OSD
Target version:
-
Start date:
05/16/2013
Due date:
% Done:

0%

Estimated time:
0.00 h
Spent time:
35.00 h (Total: 55.50 h)
Source:
Development
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

PG logs, PG missing: The logic for merging an authoritative PG log with another PG log while filling in the missing set would benefit massively from being testable seperately from a PG instance. It's possible that the stripes involved in ErasureCodedPG will make this impractical to generalize.

Discussions


Subtasks

Subtask #5085: PG::merge_log should not have side effects other than on the log & missing data membersRejectedLoic Dachary

Subtask #5213: unit tests for src/osd/PGLog.{cc,h}ResolvedLoic Dachary

Associated revisions

Revision ec7731f7 (diff)
Added by Loic Dachary over 6 years ago

move log, ondisklog, missing from PG to PGLog

PG::log, PG::ondisklog, PG::missing are moved from PG to a new PGLog
class and are made protected data members. It is a preliminary step
before writing unit tests to cover the methods that have side effects
on these data members and define a clean PGLog API. It improves
encapsulation and does not change any of the logic already in
place.

Possible issues :

  • an additional reference (PG->PGLog->IndexedLog instead of
    PG->IndexedLog for instance) is introduced : is it optimized ?
  • rewriting log.log into pg_log.get_log().log affects the readability
    but should be optimized and have no impact on performances

The guidelines followed for this patch are:

  • const access to the data members are preserved, no attempt is made
    to define accessors
  • all non const methods are in PGLog, no access to non const methods of
    PGLog::log, PGLog::logondisk and PGLog::missing are provided
  • when methods are moved from PG to PGLog the change to their
    implementation is restricted to the minimum.
  • the PG::OndiskLog and PG::IndexedLog sub classes are moved
    to PGLog sub classes unmodified and remain public

A const version of the pg_log_t::find_entry method was added.

A const accessor is provided for PGLog::get_log, PGLog::get_missing,
PGLog::get_ondisklog but no non-const accessor.

Arguments are added to most of the methods moved from PG to PGLog so
that they can get access to PG data members such as info or log_oid.

The PGLog method are sorted according to the data member they modify.

//////////////////// missing ////////////////////

  • The pg_missing_t::{got,have,need,add,rm} methods are wrapped as
    PGLog::missing_{got,have,need,add,rm}

//////////////////// log ////////////////////

  • PGLog::get_tail, PGLog::get_head getters are created
  • PGLog::set_tail, PGLog::set_head, PGLog::set_last_requested setters
    are created
  • PGLog::index, PGLog::unindex, PGLog::add wrappers,
    PGLog::reset_recovery_pointers are created
  • PGLog::clear_info_log replaces PG::clear_info_log
  • PGLog::trim replaces PG::trim

//////////////////// log & missing ////////////////////

  • PGLog::claim_log is created with code extracted from
    PG::RecoveryState::Stray::react.
  • PGLog::split_into is created with code extracted from
    PG::split_into.
  • PGLog::recover_got is created with code extracted from
    ReplicatedPG::recover_got.
  • PGLog::activate_not_complete is created with code extracted
    from PG::active
  • PGLog:proc_replica_log is created with code extracted from
    PG::proc_replica_log
  • PGLog:write_log is created with code extracted from
    PG::write_log
  • PGLog::merge_old_entry replaces PG::merge_old_entry
    The remove_snap argument is used to collect hobject_t
  • PGLog::rewind_divergent_log replaces PG::rewind_divergent_log
    The remove_snap argument is used to collect hobject_t
    A new PG::rewind_divergent_log method is added to call
    remove_snap_mapped_object on each of the remove_snap
    elements
  • PGLog::merge_log replaces PG::merge_log
    The remove_snap argument is used to collect hobject_t
    A new PG::merge_log method is added to call
    remove_snap_mapped_object on each of the remove_snap
    elements
  • PGLog:write_log is created with code extracted from PG::write_log. A
    non-static version is created for convenience but is a simple
    wrapper.
  • PGLog:read_log replaces PG::read_log. A non-static version is
    created for convenience but is a simple wrapper.
  • PGLog:read_log_old replaces PG::read_log_old.

http://tracker.ceph.com/issues/5046 refs #5046

Signed-off-by: Loic Dachary <>

History

#1 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#2 Updated by Loic Dachary over 6 years ago

read PG::merge_log PG::merge_old_entry PG::rewind_divergent_log PG::proc_replica_log
pg_log_t + pg_missing_t
Read SnapMapper & MapCacher
create PGLog.{cc,h} and move code there, including all IndexedLog

#3 Updated by Loic Dachary over 6 years ago

pg_info_t ( including pg_stat_t ) is modified during the log merging phase but it should not be the case. When factoring out the log merging, the modifications of pg_info_t must be moved out of the merge code.

#4 Updated by Loic Dachary over 6 years ago

Move code to PGLog.cc PGLog.h so that missing, log and ondisklog are protected. Fix what breaks.

#5 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)
  • Source set to Development

#6 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#7 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#8 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#9 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#12 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#13 Updated by Loic Dachary over 6 years ago

  • Description updated (diff)

#14 Updated by Loic Dachary about 6 years ago

  • Category changed from common to OSD

#15 Updated by Loic Dachary about 6 years ago

  • Assignee deleted (Loic Dachary)

#16 Updated by Samuel Just about 6 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF