Project

General

Profile

Rados - improve ex-import functionality » History » Version 1

Jessica Mack, 07/03/2015 09:28 PM

1 1 Jessica Mack
h1. Rados - improve ex-import functionality
2 1 Jessica Mack
3 1 Jessica Mack
h3. Summary
4 1 Jessica Mack
5 1 Jessica Mack
There are several problems and missing functionality with 'rados export/import' tools. 
6 1 Jessica Mack
7 1 Jessica Mack
h3. Owners
8 1 Jessica Mack
9 1 Jessica Mack
* Danny Al-Gaaf (Deutsche Telekom)
10 1 Jessica Mack
* Name (Affiliation)
11 1 Jessica Mack
* Name
12 1 Jessica Mack
13 1 Jessica Mack
h3. Interested Parties
14 1 Jessica Mack
15 1 Jessica Mack
* Marc Koderer ( Deutsche Telekom)
16 1 Jessica Mack
* Name (Affiliation)
17 1 Jessica Mack
* Name
18 1 Jessica Mack
19 1 Jessica Mack
h3. Current Status
20 1 Jessica Mack
 
21 1 Jessica Mack
h3. Detailed Description
22 1 Jessica Mack
23 1 Jessica Mack
The current implementation of 'rados export/import' has some issues:
24 1 Jessica Mack
* export/import speed is very slow
25 1 Jessica Mack
* export of large objects fails (e.g. big objects put via S3 into the cluster pool) due to limitation of extended attributes on the target filesystem
26 1 Jessica Mack
* depends on extended attributes support on local target filesystem
27 1 Jessica Mack
28 1 Jessica Mack
There are missing features:
29 1 Jessica Mack
* direct export from cluster1 and re-import in cluster2 in one run (could be faster and needed in some cases)
30 1 Jessica Mack
* verification, checksums
31 1 Jessica Mack
32 1 Jessica Mack
h3. Work items
33 1 Jessica Mack
34 1 Jessica Mack
h4. Coding tasks
35 1 Jessica Mack
36 1 Jessica Mack
* export performance:
37 1 Jessica Mack
** random shuffle "ExportLocalFileWQ export_object_wq" before start export should increase the performance (dalgaaf: already implemented and tested, need to rebase and review)
38 1 Jessica Mack
* extended attributes:
39 1 Jessica Mack
** use different approach than extended attributes or at least add option to export without
40 1 Jessica Mack
* direct export/import
41 1 Jessica Mack
** add functions and options to export/import in one run
42 1 Jessica Mack
* verification/checksums
43 1 Jessica Mack
** ...
44 1 Jessica Mack
45 1 Jessica Mack
h4. Build / release tasks
46 1 Jessica Mack
47 1 Jessica Mack
# Task 1
48 1 Jessica Mack
# Task 2
49 1 Jessica Mack
# Task 3
50 1 Jessica Mack
51 1 Jessica Mack
h4. Documentation tasks
52 1 Jessica Mack
53 1 Jessica Mack
# Task 1
54 1 Jessica Mack
# Task 2
55 1 Jessica Mack
# Task 3
56 1 Jessica Mack
57 1 Jessica Mack
h4. Deprecation tasks
58 1 Jessica Mack
59 1 Jessica Mack
# Support already exported pools to be imported again also with new implementation.