Project

General

Profile

Bug #10955

rest-bench - segmentation fault with concurrent writes > 64

Added by Italo Santos over 4 years ago. Updated about 4 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Target version:
Start date:
02/25/2015
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

When I try to use rest-bench with more then 64 concurrent writes I face with a unexpected error, aborting the execution generating too many Errors and a segmentation fault (see attachment).

Looking to the radosgw logs (debug=20) and apache logs I can't see any error, which make me believe this could be an rest-bench issue. See the errors below:

root@host:~# rest-bench -t 128 --api-host=hostname --access-key=access_key --secret=secret_key --bucket=bench --uri_style=vhost --no-cleanup write
host=hostname
Maintaining 128 concurrent writes of 4194304 bytes for up to 10 seconds or 0 objects
Object prefix: benchmark_data_cpro24374_14928
sec Cur ops started finished avg MB/s cur MB/s last lat avg lat
0 1 1 0 0 0 - 0
1 128 128 0 0 0 - 0
2 128 129 1 1.97244 2 1.23336 1.23336
3 128 129 1 1.19953 0 - 1.23336
4 128 138 10 9.227 18 4.27861 3.86935
5 127 182 55 41.2308 180 5.25191 4.6642
ERROR: benchmark_data_cpro24374_14928_object124: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object110: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object57: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object113: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object26: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object117: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object56: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object109: ConnectionFailed
6 128 216 88 55.5478 132 1.85818 4.23068
ERROR: benchmark_data_cpro24374_14928_object119: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object71: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object10: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object111: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object17: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object80: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object104: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object122: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object127: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object16: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object65: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object47: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object90: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object121: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object108: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object115: ConnectionFailedERROR: benchmark_data_cpro24374_14928_object81: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object82: ConnectionFailed

ERROR: benchmark_data_cpro24374_14928_object114: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object21: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object79: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object49: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object116: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object73: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object123: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object94: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object52: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object24: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object103: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object118: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object125: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object40: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object38: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object35: ConnectionFailed
ERROR: ERROR: benchmark_data_cpro24374_14928_object105: ConnectionFailed
benchmark_data_cpro24374_14928_object84: ConnectionFailed
ERROR: benchmark_data_cpro24374_14928_object20: ConnectionFailed
error during benchmark: -5
common/WorkQueue.cc: In function 'virtual ThreadPool::~ThreadPool()' thread 7f7b45f2d7c0 time 2015-02-25 18:46:30.393222
common/WorkQueue.cc: 68: FAILED assert(_threads.empty())
ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578)

rest-bench_segfault.txt View (11.4 KB) Italo Santos, 02/25/2015 10:23 PM


Related issues

Duplicates rgw - Bug #3896: rest-bench common/WorkQueue.cc: 54: FAILED assert(_threads.empty()) Resolved 01/22/2013

History

#1 Updated by Kefu Chai about 4 years ago

  • Status changed from New to Duplicate
  • Regression set to No

Also available in: Atom PDF