Average Latency:        1.708
Stddev Latency:         2.00759
Max latency:            19.4718
Min latency:            0.0769
common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f67723d8780 time 2013-09-17 15:02:12.808953
common/WorkQueue.cc: 65: FAILED assert(_threads.empty())
 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 2: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 3: (main()+0xd97) [0x4a9db7]
 4: (__libc_start_main()+0xed) [0x7f67702ee76d]
 5: rest-bench() [0x4ab179]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2013-09-17 15:02:12.809282 7f67723d8780 -1 common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f67723d8780 time 2013-09-17 15:02:12.808953
common/WorkQueue.cc: 65: FAILED assert(_threads.empty())

 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 2: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 3: (main()+0xd97) [0x4a9db7]
 4: (__libc_start_main()+0xed) [0x7f67702ee76d]
 5: rest-bench() [0x4ab179]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
   -12> 2013-09-17 15:01:06.168029 7f67723d8780  5 asok(0x1ecd220) register_command perfcounters_dump hook 0x1ecd3f0
   -11> 2013-09-17 15:01:06.168048 7f67723d8780  5 asok(0x1ecd220) register_command 1 hook 0x1ecd3f0
   -10> 2013-09-17 15:01:06.168053 7f67723d8780  5 asok(0x1ecd220) register_command perf dump hook 0x1ecd3f0
    -9> 2013-09-17 15:01:06.168058 7f67723d8780  5 asok(0x1ecd220) register_command perfcounters_schema hook 0x1ecd3f0
    -8> 2013-09-17 15:01:06.168061 7f67723d8780  5 asok(0x1ecd220) register_command 2 hook 0x1ecd3f0
    -7> 2013-09-17 15:01:06.168116 7f67723d8780  5 asok(0x1ecd220) register_command perf schema hook 0x1ecd3f0
    -6> 2013-09-17 15:01:06.168120 7f67723d8780  5 asok(0x1ecd220) register_command config show hook 0x1ecd3f0
    -5> 2013-09-17 15:01:06.168123 7f67723d8780  5 asok(0x1ecd220) register_command config set hook 0x1ecd3f0
    -4> 2013-09-17 15:01:06.168127 7f67723d8780  5 asok(0x1ecd220) register_command config get hook 0x1ecd3f0
    -3> 2013-09-17 15:01:06.168143 7f67723d8780  5 asok(0x1ecd220) register_command log flush hook 0x1ecd3f0
    -2> 2013-09-17 15:01:06.168147 7f67723d8780  5 asok(0x1ecd220) register_command log dump hook 0x1ecd3f0
    -1> 2013-09-17 15:01:06.168159 7f67723d8780  5 asok(0x1ecd220) register_command log reopen hook 0x1ecd3f0
     0> 2013-09-17 15:02:12.809282 7f67723d8780 -1 common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f67723d8780 time 2013-09-17 15:02:12.808953
common/WorkQueue.cc: 65: FAILED assert(_threads.empty())

 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 2: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 3: (main()+0xd97) [0x4a9db7]
 4: (__libc_start_main()+0xed) [0x7f67702ee76d]
 5: rest-bench() [0x4ab179]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  99/99 (stderr threshold)
  max_recent       500
  max_new         1000
  log_file
--- end dump of recent events ---
terminate called after throwing an instance of 'ceph::FailedAssertion'
*** Caught signal (Aborted) **
 in thread 7f67723d8780
 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: rest-bench() [0x4bb78a]
 2: (()+0xfcb0) [0x7f6771a87cb0]
 3: (gsignal()+0x35) [0x7f6770303425]
 4: (abort()+0x17b) [0x7f6770306b8b]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f6770c5569d]
 6: (()+0xb5846) [0x7f6770c53846]
 7: (()+0xb5873) [0x7f6770c53873]
 8: (()+0xb596e) [0x7f6770c5396e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x4c399f]
 10: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 11: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 12: (main()+0xd97) [0x4a9db7]
 13: (__libc_start_main()+0xed) [0x7f67702ee76d]
 14: rest-bench() [0x4ab179]
2013-09-17 15:02:12.810025 7f67723d8780 -1 *** Caught signal (Aborted) **
 in thread 7f67723d8780

 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: rest-bench() [0x4bb78a]
 2: (()+0xfcb0) [0x7f6771a87cb0]
 3: (gsignal()+0x35) [0x7f6770303425]
 4: (abort()+0x17b) [0x7f6770306b8b]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f6770c5569d]
 6: (()+0xb5846) [0x7f6770c53846]
 7: (()+0xb5873) [0x7f6770c53873]
 8: (()+0xb596e) [0x7f6770c5396e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x4c399f]
 10: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 11: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 12: (main()+0xd97) [0x4a9db7]
 13: (__libc_start_main()+0xed) [0x7f67702ee76d]
 14: rest-bench() [0x4ab179]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2013-09-17 15:02:12.810025 7f67723d8780 -1 *** Caught signal (Aborted) **
 in thread 7f67723d8780

 ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a)
 1: rest-bench() [0x4bb78a]
 2: (()+0xfcb0) [0x7f6771a87cb0]
 3: (gsignal()+0x35) [0x7f6770303425]
 4: (abort()+0x17b) [0x7f6770306b8b]
 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f6770c5569d]
 6: (()+0xb5846) [0x7f6770c53846]
 7: (()+0xb5873) [0x7f6770c53873]
 8: (()+0xb596e) [0x7f6770c5396e]
 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x4c399f]
 10: (ThreadPool::~ThreadPool()+0x126) [0x4c0ee6]
 11: (RESTDispatcher::~RESTDispatcher()+0xe9) [0x4aee99]
 12: (main()+0xd97) [0x4a9db7]
 13: (__libc_start_main()+0xed) [0x7f67702ee76d]
 14: rest-bench() [0x4ab179]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/ 5 hadoop
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
  -2/-2 (syslog threshold)
  99/99 (stderr threshold)
  max_recent       500
  max_new         1000
  log_file
--- end dump of recent events ---
Aborted (core dumped)