Project

General

Profile

Feature #2087

lightweight filestore workload generator

Added by Sage Weil almost 8 years ago. Updated over 7 years ago.

Status:
Resolved
Priority:
Normal
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

simple program that uses FileStore and generates something that looks vaguely like what an OSD does. e.g.,

- stream of transactions with something like
- small(or large) write to new object
- setxattr on that object
- append to per-collection log object
- set collection xattr

do that with say 50 parallel streams to different collections.

History

#1 Updated by Sage Weil almost 8 years ago

  • Target version changed from v0.43 to v0.44

#2 Updated by Sage Weil almost 8 years ago

  • translation missing: en.field_position set to 1138

#3 Updated by Sage Weil almost 8 years ago

  • translation missing: en.field_story_points set to 5
  • translation missing: en.field_position deleted (1138)
  • translation missing: en.field_position set to 1138

#4 Updated by Sage Weil almost 8 years ago

  • Assignee set to Samuel Just

#5 Updated by Samuel Just almost 8 years ago

  • Assignee changed from Samuel Just to Joao Eduardo Luis

Mark suggested that this would be a good talk for Joao.

#6 Updated by Samuel Just almost 8 years ago

*task

#7 Updated by Sage Weil almost 8 years ago

  • Status changed from New to In Progress

#8 Updated by Sage Weil almost 8 years ago

  • Target version changed from v0.44 to v0.45
  • translation missing: en.field_position deleted (1150)
  • translation missing: en.field_position set to 8

#9 Updated by Sage Weil almost 8 years ago

  • translation missing: en.field_position deleted (13)
  • translation missing: en.field_position set to 12

#10 Updated by Sage Weil almost 8 years ago

  • translation missing: en.field_position deleted (10)
  • translation missing: en.field_position set to 1
  • translation missing: en.field_position changed from 1 to 1167

#11 Updated by Joao Eduardo Luis over 7 years ago

  • Status changed from In Progress to 4

Pushed a working version to ceph's git repository, branch wip-2087 [1]. Feedback would be appreciated.

[1] - https://github.com/ceph/ceph/commit/6158e0ccb48b0598cffdbf3333a9c38afc78d5ae

#12 Updated by Joao Eduardo Luis over 7 years ago

Pushed a new commit to [1], making the code compliant with the CodeStyle and with Sage's suggestions on github.

[1] - https://github.com/ceph/ceph/commit/8aa0e798160e4400bec0d9f975c1f2fb4b3e1ba2

#13 Updated by Joao Eduardo Luis over 7 years ago

  • Status changed from 4 to 7

#14 Updated by Joao Eduardo Luis over 7 years ago

  • Status changed from 7 to In Progress

Looks like some memory should be leaking bad, such that valgrind hangs on exit.

19080
19080 HEAP SUMMARY:
19080 in use at exit: 9,454,160,343 bytes in 68,984 blocks
19080 total heap usage: 973,116 allocs, 904,132 frees, 19,001,241,545 bytes allocated
19080
^C^C^C^C^C^C

#15 Updated by Joao Eduardo Luis over 7 years ago

  • Status changed from In Progress to 7

Memory leak fixed.

Apparently, the FileStore does not cleanup after transactions once they are applied, which may lead to huge memory leaks.

Now we simply 'delete m_tx' in the transaction's callback class.

Valgrind still reports memory leaks, but they seem to come from the FileStore class instead of being directly related with our WorkloadGenerator class.

#16 Updated by Sage Weil over 7 years ago

  • Status changed from 7 to Resolved

Also available in: Atom PDF