Feature #7063
closedReplace lfn_find call in omap* functions with more lighter implematation
0%
Description
find_index plays a important role in handling transactions with FileStore. Although it's not a heave implementation, it's called several times each transaction and the cumulative time is still striking.
In a latest test with all SSD backend, the actual IO operations don't occur the main consuming time and find_index takes over 25% time of the whole transaction.
Systemtap results:
----lfn_find--------------------------------------------------
value |-------------------------------------------------- count
20 | 0
30 | 0
40 |@@@@@@ 5290@
50 |@@@@@@@@@@@@@@@@@@@@@@@ 18867
60 |@@@@@@ 4544
70 |@@@@ 3673@
80 |@@ 2279
90 | 467@
100 | 117
>100 |@@ 2275
sum: 2646862
----do_transactions-------------------------------------------
value |-------------------------------------------------- count
225 | 0
250 | 0
>250 |@@@@@@@@@@@@@@@@@@@@@@@@@ 9377
sum: 10785520
--------------------------------------------------------------
I simply remove all lfn_find() in omap* functions, it decrease op latency hugely.
----do_transactions-------------------------------------------
value |-------------------------------------------------- count
225 | 0
250 | 0
>250 |@@@@@@@@@@@@@@@@@@@@@@@@@ 20212
39359311
--------------------------------------------------------------
Nearly improve 2x throughout.
So I think we need to a way to avoid or lighten find_index call. Maybe cache object existing status, remove object or move object both are rarely thing.
Updated by Haomai Wang over 10 years ago
Maybe we can maintain a hashmap<object, state>, "state" can be int and 0 mean unknown, 1 mean existing. If unknown, it may need to call "lfn_find" to check. Because most of "lfn_find" return value are existing state, it will be a little percentage of caller needed to check filesystem again.
Atomic operation can be used to modify state.
Updated by Haomai Wang over 10 years ago
- Subject changed from Replace find_index call in omap* functions with more lighter implematation to Replace lfn_find call in omap* functions with more lighter implematation
Updated by Haomai Wang over 10 years ago
It seemed that "IndexManager::get_index" protect concurrent write operations on the same object. So if remove it, multi write operations on the same object may be happened?