Project

General

Profile

Actions

Feature #41073

closed

cephfs-sync: tool for synchronizing cephfs snapshots to remote target

Added by Venky Shankar over 4 years ago. Updated about 3 years ago.

Status:
Rejected
Priority:
Normal
Category:
Administration/Usability
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
Labels (FS):
Pull request ID:

Description

Introduce a rsync like tool (cephfs-sync) for mirroring scheduled and temp cephfs snapshots to sync targets. sync targets could be ceph filesystem on a remote site or even a traditional filesystem. We may want to sync to remote ceph filesystem initially.

We would start with scanning the filesystem based on rctime and feed the file list to rsync. Later, implement (or provide an option) to switch using various sync mechanisms that are much more efficient (such as identifying renames and propagating that to the remote target rather than a copy and delete).


Related issues 1 (0 open1 closed)

Related to CephFS - Feature #41074: pybind/mgr/volumes: mirror (scheduled) snapshots to remote targetResolvedVenky Shankar

Actions
Actions #1

Updated by Venky Shankar over 4 years ago

  • Related to Feature #41074: pybind/mgr/volumes: mirror (scheduled) snapshots to remote target added
Actions #2

Updated by Patrick Donnelly over 4 years ago

  • Assignee set to Daniel Oliveira
Actions #3

Updated by Patrick Donnelly over 4 years ago

  • Target version deleted (v15.0.0)
Actions #4

Updated by Patrick Donnelly about 4 years ago

  • Assignee deleted (Daniel Oliveira)
  • Target version set to v16.0.0
Actions #5

Updated by Milind Changire almost 4 years ago

  • Assignee set to Milind Changire
Actions #6

Updated by Milind Changire almost 4 years ago

Feature Requirements:
  • tool should be implemented as a mgr plugin
  • tool should mirror User Defined as well as Scheduled Snapshots
  • tool should optimize on number of snapshot processes running concurrently
  • tool should run sync process in a containerized environment
  • tool should verify which snapshots are pending and which are completed
  • tool should interact with the snapshot retention mechanism to block purging of snapshots which haven't been synced / mirrored yet
  • sync process should transfer data over a secure connection eg. ssh tunnel
  • sync process should limit use of network bandwidth for data transfers
Actions #7

Updated by Patrick Donnelly about 3 years ago

Milind what's the status of this tickeT?

Actions #8

Updated by Patrick Donnelly about 3 years ago

ping

Actions #9

Updated by Venky Shankar about 3 years ago

I think this can be closed. Right Milind?

(We dropped relying on rsync)

Actions #10

Updated by Patrick Donnelly about 3 years ago

  • Status changed from New to Rejected
Actions

Also available in: Atom PDF