Project

General

Profile

Bug #10955 ยป rest-bench_segfault.txt

Italo Santos, 02/25/2015 10:23 PM

 
root@host:~# rest-bench -t 128 --api-host=hostname --access-key=access_key --secret=secret_key --bucket=bench --uri_style=vhost --no-cleanup write
host=hostname
Maintaining 128 concurrent writes of 4194304 bytes for up to 10 seconds or 0 objects
Object prefix: benchmark_data_cpro24374_14928
sec Cur ops started finished avg MB/s cur MB/s last lat avg lat
0 1 1 0 0 0 - 0
1 128 128 0 0 0 - 0
2 128 129 1 1.97244 2 1.23336 1.23336
3 128 129 1 1.19953 0 - 1.23336
4 128 138 10 9.227 18 4.27861 3.86935
5 127 182 55 41.2308 180 5.25191 4.6642
ERROR: benchmark_data_cpro24374_14928_object124: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object110: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object57: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object113: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object26: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object117: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object56: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object109: ConnectionFailed
6 128 216 88 55.5478 132 1.85818 4.23068
ERROR: benchmark_data_cpro24374_14928_object119: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object71: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object10: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object111: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object17: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object80: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object104: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object122: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object127: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object16: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object65: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object47: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object90: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object121: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object108: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object115: ConnectionFailedERROR: benchmark_data_cpro24374_14928_object81: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object82: ConnectionFailed

ERROR: benchmark_data_cpro24374_14928_object114: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object21: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object79: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object49: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object116: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object73: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object123: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object94: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object52: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object24: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object103: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object118: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object125: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object40: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object38: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object35: ConnectionFailed
ERROR: ERROR: benchmark_data_cpro24374_14928_object105: ConnectionFailed
benchmark_data_cpro24374_14928_object84: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object20: ConnectionFailed
error during benchmark: -5
common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f7b45f2d7c0 time 2015-02-25 18:46:30.393222
common/WorkQueue.cc: 68: FAILED assert(_threads.empty())
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x4cd9fb]
2: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
3: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
4: (main()+0x42e) [0x4b071e]
5: (__libc_start_main()+0xf5) [0x7f7b43debec5]
6: rest-bench() [0x4b1a47]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2015-02-25 18:46:30.394252 7f7b45f2d7c0 -1 common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f7b45f2d7c0 time 2015-02-25 18:46:30.393222
common/WorkQueue.cc: 68: FAILED assert(_threads.empty())

ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x4cd9fb]
2: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
3: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
4: (main()+0x42e) [0x4b071e]
5: (__libc_start_main()+0xf5) [0x7f7b43debec5]
6: rest-bench() [0x4b1a47]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-13> 2015-02-25 18:46:21.767160 7f7b45f2d7c0 5 asok(0x4319850) register_command perfcounters_dump hook 0x4319710
-12> 2015-02-25 18:46:21.767197 7f7b45f2d7c0 5 asok(0x4319850) register_command 1 hook 0x4319710
-11> 2015-02-25 18:46:21.767289 7f7b45f2d7c0 5 asok(0x4319850) register_command perf dump hook 0x4319710
-10> 2015-02-25 18:46:21.767320 7f7b45f2d7c0 5 asok(0x4319850) register_command perfcounters_schema hook 0x4319710
-9> 2015-02-25 18:46:21.767343 7f7b45f2d7c0 5 asok(0x4319850) register_command 2 hook 0x4319710
-8> 2015-02-25 18:46:21.767351 7f7b45f2d7c0 5 asok(0x4319850) register_command perf schema hook 0x4319710
-7> 2015-02-25 18:46:21.767359 7f7b45f2d7c0 5 asok(0x4319850) register_command config show hook 0x4319710
-6> 2015-02-25 18:46:21.767382 7f7b45f2d7c0 5 asok(0x4319850) register_command config set hook 0x4319710
-5> 2015-02-25 18:46:21.767391 7f7b45f2d7c0 5 asok(0x4319850) register_command config get hook 0x4319710
-4> 2015-02-25 18:46:21.767399 7f7b45f2d7c0 5 asok(0x4319850) register_command config diff hook 0x4319710
-3> 2015-02-25 18:46:21.767434 7f7b45f2d7c0 5 asok(0x4319850) register_command log flush hook 0x4319710
-2> 2015-02-25 18:46:21.767458 7f7b45f2d7c0 5 asok(0x4319850) register_command log dump hook 0x4319710
-1> 2015-02-25 18:46:21.767467 7f7b45f2d7c0 5 asok(0x4319850) register_command log reopen hook 0x4319710
0> 2015-02-25 18:46:30.394252 7f7b45f2d7c0 -1 common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f7b45f2d7c0 time 2015-02-25 18:46:30.393222
common/WorkQueue.cc: 68: FAILED assert(_threads.empty())

ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x4cd9fb]
2: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
3: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
4: (main()+0x42e) [0x4b071e]
5: (__libc_start_main()+0xf5) [0x7f7b43debec5]
6: rest-bench() [0x4b1a47]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
-2/-2 (syslog threshold)
99/99 (stderr threshold)
max_recent 500
max_new 1000
log_file
--- end dump of recent events ---
terminate called after throwing an instance of 'ceph::FailedAssertion'
*** Caught signal (Aborted) **
in thread 7f7b45f2d7c0
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: rest-bench() [0x4c240a]
2: (()+0x10340) [0x7f7b45907340]
3: (gsignal()+0x39) [0x7f7b43e00bb9]
4: (abort()+0x148) [0x7f7b43e03fc8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f7b4470c6b5]
6: (()+0x5e836) [0x7f7b4470a836]
7: (()+0x5e863) [0x7f7b4470a863]
8: (()+0x5eaa2) [0x7f7b4470aaa2]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x4cdbe8]
10: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
11: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
12: (main()+0x42e) [0x4b071e]
13: (__libc_start_main()+0xf5) [0x7f7b43debec5]
14: rest-bench() [0x4b1a47]
2015-02-25 18:46:30.398877 7f7b45f2d7c0 -1 *** Caught signal (Aborted) **
in thread 7f7b45f2d7c0

ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: rest-bench() [0x4c240a]
2: (()+0x10340) [0x7f7b45907340]
3: (gsignal()+0x39) [0x7f7b43e00bb9]
4: (abort()+0x148) [0x7f7b43e03fc8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f7b4470c6b5]
6: (()+0x5e836) [0x7f7b4470a836]
7: (()+0x5e863) [0x7f7b4470a863]
8: (()+0x5eaa2) [0x7f7b4470aaa2]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x4cdbe8]
10: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
11: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
12: (main()+0x42e) [0x4b071e]
13: (__libc_start_main()+0xf5) [0x7f7b43debec5]
14: rest-bench() [0x4b1a47]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
0> 2015-02-25 18:46:30.398877 7f7b45f2d7c0 -1 *** Caught signal (Aborted) **
in thread 7f7b45f2d7c0

ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)
1: rest-bench() [0x4c240a]
2: (()+0x10340) [0x7f7b45907340]
3: (gsignal()+0x39) [0x7f7b43e00bb9]
4: (abort()+0x148) [0x7f7b43e03fc8]
5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f7b4470c6b5]
6: (()+0x5e836) [0x7f7b4470a836]
7: (()+0x5e863) [0x7f7b4470a863]
8: (()+0x5eaa2) [0x7f7b4470aaa2]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x278) [0x4cdbe8]
10: (ThreadPool::~ThreadPool()+0x126) [0x4c93a6]
11: (RESTDispatcher::~RESTDispatcher()+0xe1) [0x4b6191]
12: (main()+0x42e) [0x4b071e]
13: (__libc_start_main()+0xf5) [0x7f7b43debec5]
14: rest-bench() [0x4b1a47]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
-2/-2 (syslog threshold)
99/99 (stderr threshold)
max_recent 500
max_new 1000
log_file
--- end dump of recent events ---
Aborted
    (1-1/1)