Rados - improve ex-import functionality¶

Summary¶

There are several problems and missing functionality with 'rados export/import' tools.

Owners¶

• Danny Al-Gaaf (Deutsche Telekom)
• Name (Affiliation)
• Name

Interested Parties¶

• Marc Koderer ( Deutsche Telekom)
• Name (Affiliation)
• Name

Current Status¶

Detailed Description¶

The current implementation of 'rados export/import' has some issues:

• export/import speed is very slow
• export of large objects fails (e.g. big objects put via S3 into the cluster pool) due to limitation of extended attributes on the target filesystem
• depends on extended attributes support on local target filesystem

There are missing features:

• direct export from cluster1 and re-import in cluster2 in one run (could be faster and needed in some cases)
• verification, checksums

Work items¶

Coding tasks¶

• export performance:
  • random shuffle "ExportLocalFileWQ export_object_wq" before start export should increase the performance (dalgaaf: already implemented and tested, need to rebase and review)
• extended attributes:
  • use different approach than extended attributes or at least add option to export without
• direct export/import
  • add functions and options to export/import in one run
• verification/checksums
  • ...

Build / release tasks¶

1. Task 1
2. Task 2
3. Task 3

Documentation tasks¶

1. Task 1
2. Task 2
3. Task 3

Deprecation tasks¶

1. Support already exported pools to be imported again also with new implementation.