Rados - improve ex-import functionality


There are several problems and missing functionality with 'rados export/import' tools.


  • Danny Al-Gaaf (Deutsche Telekom)
  • Name (Affiliation)
  • Name

Interested Parties

  • Marc Koderer ( Deutsche Telekom)
  • Name (Affiliation)
  • Name

Current Status

Detailed Description

The current implementation of 'rados export/import' has some issues:
  • export/import speed is very slow
  • export of large objects fails (e.g. big objects put via S3 into the cluster pool) due to limitation of extended attributes on the target filesystem
  • depends on extended attributes support on local target filesystem
There are missing features:
  • direct export from cluster1 and re-import in cluster2 in one run (could be faster and needed in some cases)
  • verification, checksums

Work items

Coding tasks

  • export performance:
    • random shuffle "ExportLocalFileWQ export_object_wq" before start export should increase the performance (dalgaaf: already implemented and tested, need to rebase and review)
  • extended attributes:
    • use different approach than extended attributes or at least add option to export without
  • direct export/import
    • add functions and options to export/import in one run
  • verification/checksums
    • ...

Build / release tasks

  1. Task 1
  2. Task 2
  3. Task 3

Documentation tasks

  1. Task 1
  2. Task 2
  3. Task 3

Deprecation tasks

  1. Support already exported pools to be imported again also with new implementation.