Bug #16323: terminate called after throwing an instance of 'std::out_of_range' in rgw-hammer-backports - rgw - Ceph

Actions

Copy link

Bug #16323

closed

terminate called after throwing an instance of 'std::out_of_range' in rgw-hammer-backports

Added by Nathan Cutler almost 8 years ago. Updated over 7 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

http://pulpito.ceph.com/smithfarm-2016-06-02_00:42:57-rgw-hammer-backports---basic-smithi/229980/
http://pulpito.ceph.com/smithfarm-2016-06-02_00:42:57-rgw-hammer-backports---basic-smithi/229987/
http://pulpito.ceph.com/smithfarm-2016-06-02_00:42:57-rgw-hammer-backports---basic-smithi/229998/

Failure Reason:

Command failed (s3 tests against rgw) on smithi053 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw'"

2016-06-10T15:21:09.453 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok
2016-06-10T15:21:09.822 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok
2016-06-10T15:21:09.954 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok
2016-06-10T15:21:10.518 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_recreate_not_overriding ... ok
2016-06-10T15:21:11.018 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_create_special_key_names ... ok
2016-06-10T15:21:11.518 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_list_special_prefix ... ok
2016-06-10T15:21:11.947 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_zero_size ... ok
2016-06-10T15:21:12.385 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_same_bucket ... ok
2016-06-10T15:21:12.737 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_to_itself ... ok
2016-06-10T15:21:13.138 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok
2016-06-10T15:21:13.749 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_diff_bucket ... ok
2016-06-10T15:21:14.178 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_not_owned_bucket ... ok
2016-06-10T15:21:14.587 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_object_copy_canned_acl ... ok
2016-06-10T15:21:19.220 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_multipart_upload ... ok
2016-06-10T15:21:24.474 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_multipart_upload_multiple_sizes ... ok
2016-06-10T15:21:25.406 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_multipart_upload_size_too_small ... ok
2016-06-10T15:21:28.599 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_multipart_upload_contents ... ok
2016-06-10T15:21:30.600 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok
2016-06-10T15:21:31.633 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_abort_multipart_upload ... ok
2016-06-10T15:21:33.631 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_list_multipart_upload ... ok
2016-06-10T15:21:33.857 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_100_continue ... ok
2016-06-10T15:21:34.514 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_bucket_acls_changes_persistent ... ok
2016-06-10T15:21:39.842 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_stress_bucket_acls_changes ... ok
2016-06-10T15:21:40.294 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_set_cors ... ok
2016-06-10T15:21:43.674 INFO:tasks.rgw.client.0.smithi053.stdout:terminate called after throwing an instance of 'std::out_of_range'
2016-06-10T15:21:43.677 INFO:tasks.rgw.client.0.smithi053.stdout:  what():  basic_string::compare
2016-06-10T15:21:43.680 INFO:tasks.rgw.client.0.smithi053.stdout:*** Caught signal (Aborted) **
2016-06-10T15:21:43.680 INFO:tasks.rgw.client.0.smithi053.stdout: in thread 319ac700
2016-06-10T15:21:43.717 INFO:tasks.rgw.client.0.smithi053.stdout: ceph version 0.94.7-94-g1ac25be (1ac25bec8ee0fa5c2a32adf0b307ebb4f8d1cdfa)
2016-06-10T15:21:43.717 INFO:tasks.rgw.client.0.smithi053.stdout: 1: radosgw() [0x6ba34a]
2016-06-10T15:21:43.717 INFO:tasks.rgw.client.0.smithi053.stdout: 2: (()+0x10340) [0x80a1340]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 3: (gsignal()+0x39) [0x98b9cc9]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 4: (abort()+0x148) [0x98bd0d8]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x90c3535]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 6: (()+0x5e6d6) [0x90c16d6]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 7: (()+0x5e703) [0x90c1703]
2016-06-10T15:21:43.718 INFO:tasks.rgw.client.0.smithi053.stdout: 8: (()+0x5e922) [0x90c1922]
2016-06-10T15:21:43.719 INFO:tasks.rgw.client.0.smithi053.stdout: 9: (std::__throw_out_of_range(char const*)+0x67) [0x9113427]
2016-06-10T15:21:43.719 INFO:tasks.rgw.client.0.smithi053.stdout: 10: (()+0xb9f0a) [0x911cf0a]
2016-06-10T15:21:43.719 INFO:tasks.rgw.client.0.smithi053.stdout: 11: radosgw() [0x65b6ed]
2016-06-10T15:21:43.719 INFO:tasks.rgw.client.0.smithi053.stdout: 12: (RGWCORSRule::is_origin_present(char const*)+0x38) [0x65bef8]
2016-06-10T15:21:43.719 INFO:tasks.rgw.client.0.smithi053.stdout: 13: (RGWOp::generate_cors_headers(std::string&, std::string&, std::string&, std::string&, unsigned int*)+0x251) [0x626f11]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 14: (dump_access_control(req_state*, RGWOp*)+0x55) [0x524a05]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 15: (end_header(req_state*, RGWOp*, char const*, long, bool)+0x47) [0x524b77]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 16: (RGWListBucket_ObjStore_S3::send_response()+0x3f) [0x538f5f]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 17: radosgw() [0x577520]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 18: radosgw() [0x57812a]
2016-06-10T15:21:43.720 INFO:tasks.rgw.client.0.smithi053.stdout: 19: radosgw() [0x685b4f]
2016-06-10T15:21:43.721 INFO:tasks.rgw.client.0.smithi053.stdout: 20: radosgw() [0x687b1e]
2016-06-10T15:21:43.721 INFO:tasks.rgw.client.0.smithi053.stdout: 21: (()+0x8182) [0x8099182]
2016-06-10T15:21:43.721 INFO:tasks.rgw.client.0.smithi053.stdout: 22: (clone()+0x6d) [0x997d47d]
2016-06-10T15:21:43.730 INFO:tasks.rgw.client.0.smithi053.stdout:2016-06-10 22:21:43.724041 319ac700 -1 *** Caught signal (Aborted) **
2016-06-10T15:21:43.731 INFO:tasks.rgw.client.0.smithi053.stdout: in thread 319ac700
2016-06-10T15:21:43.731 INFO:tasks.rgw.client.0.smithi053.stdout:
2016-06-10T15:21:43.731 INFO:tasks.rgw.client.0.smithi053.stdout: ceph version 0.94.7-94-g1ac25be (1ac25bec8ee0fa5c2a32adf0b307ebb4f8d1cdfa)
2016-06-10T15:21:43.731 INFO:tasks.rgw.client.0.smithi053.stdout: 1: radosgw() [0x6ba34a]
2016-06-10T15:21:43.732 INFO:tasks.rgw.client.0.smithi053.stdout: 2: (()+0x10340) [0x80a1340]
2016-06-10T15:21:43.732 INFO:tasks.rgw.client.0.smithi053.stdout: 3: (gsignal()+0x39) [0x98b9cc9]
2016-06-10T15:21:43.732 INFO:tasks.rgw.client.0.smithi053.stdout: 4: (abort()+0x148) [0x98bd0d8]
2016-06-10T15:21:43.732 INFO:tasks.rgw.client.0.smithi053.stdout: 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x90c3535]
2016-06-10T15:21:43.733 INFO:tasks.rgw.client.0.smithi053.stdout: 6: (()+0x5e6d6) [0x90c16d6]
2016-06-10T15:21:43.733 INFO:tasks.rgw.client.0.smithi053.stdout: 7: (()+0x5e703) [0x90c1703]
2016-06-10T15:21:43.733 INFO:tasks.rgw.client.0.smithi053.stdout: 8: (()+0x5e922) [0x90c1922]
2016-06-10T15:21:43.733 INFO:tasks.rgw.client.0.smithi053.stdout: 9: (std::__throw_out_of_range(char const*)+0x67) [0x9113427]
2016-06-10T15:21:43.734 INFO:tasks.rgw.client.0.smithi053.stdout: 10: (()+0xb9f0a) [0x911cf0a]
2016-06-10T15:21:43.734 INFO:tasks.rgw.client.0.smithi053.stdout: 11: radosgw() [0x65b6ed]
2016-06-10T15:21:43.734 INFO:tasks.rgw.client.0.smithi053.stdout: 12: (RGWCORSRule::is_origin_present(char const*)+0x38) [0x65bef8]
2016-06-10T15:21:43.734 INFO:tasks.rgw.client.0.smithi053.stdout: 13: (RGWOp::generate_cors_headers(std::string&, std::string&, std::string&, std::string&, unsigned int*)+0x251) [0x626f11]
2016-06-10T15:21:43.735 INFO:tasks.rgw.client.0.smithi053.stdout: 14: (dump_access_control(req_state*, RGWOp*)+0x55) [0x524a05]
2016-06-10T15:21:43.735 INFO:tasks.rgw.client.0.smithi053.stdout: 15: (end_header(req_state*, RGWOp*, char const*, long, bool)+0x47) [0x524b77]
2016-06-10T15:21:43.735 INFO:tasks.rgw.client.0.smithi053.stdout: 16: (RGWListBucket_ObjStore_S3::send_response()+0x3f) [0x538f5f]
2016-06-10T15:21:43.735 INFO:tasks.rgw.client.0.smithi053.stdout: 17: radosgw() [0x577520]
2016-06-10T15:21:43.735 INFO:tasks.rgw.client.0.smithi053.stdout: 18: radosgw() [0x57812a]
2016-06-10T15:21:43.736 INFO:tasks.rgw.client.0.smithi053.stdout: 19: radosgw() [0x685b4f]
2016-06-10T15:21:43.736 INFO:tasks.rgw.client.0.smithi053.stdout: 20: radosgw() [0x687b1e]
2016-06-10T15:21:43.736 INFO:tasks.rgw.client.0.smithi053.stdout: 21: (()+0x8182) [0x8099182]
2016-06-10T15:21:43.736 INFO:tasks.rgw.client.0.smithi053.stdout: 22: (clone()+0x6d) [0x997d47d]
2016-06-10T15:21:43.737 INFO:tasks.rgw.client.0.smithi053.stdout: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2016-06-10T15:21:43.737 INFO:tasks.rgw.client.0.smithi053.stdout:
2016-06-10T15:21:44.267 INFO:tasks.rgw.client.0.smithi053.stdout:     0> 2016-06-10 22:21:43.724041 319ac700 -1 *** Caught signal (Aborted) **
2016-06-10T15:21:44.268 INFO:tasks.rgw.client.0.smithi053.stdout: in thread 319ac700
2016-06-10T15:21:44.268 INFO:tasks.rgw.client.0.smithi053.stdout:
2016-06-10T15:21:44.268 INFO:tasks.rgw.client.0.smithi053.stdout: ceph version 0.94.7-94-g1ac25be (1ac25bec8ee0fa5c2a32adf0b307ebb4f8d1cdfa)
2016-06-10T15:21:44.269 INFO:tasks.rgw.client.0.smithi053.stdout: 1: radosgw() [0x6ba34a]
2016-06-10T15:21:44.269 INFO:tasks.rgw.client.0.smithi053.stdout: 2: (()+0x10340) [0x80a1340]
2016-06-10T15:21:44.269 INFO:tasks.rgw.client.0.smithi053.stdout: 3: (gsignal()+0x39) [0x98b9cc9]
2016-06-10T15:21:44.269 INFO:tasks.rgw.client.0.smithi053.stdout: 4: (abort()+0x148) [0x98bd0d8]
2016-06-10T15:21:44.270 INFO:tasks.rgw.client.0.smithi053.stdout: 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x90c3535]
2016-06-10T15:21:44.270 INFO:tasks.rgw.client.0.smithi053.stdout: 6: (()+0x5e6d6) [0x90c16d6]
2016-06-10T15:21:44.270 INFO:tasks.rgw.client.0.smithi053.stdout: 7: (()+0x5e703) [0x90c1703]
2016-06-10T15:21:44.271 INFO:tasks.rgw.client.0.smithi053.stdout: 8: (()+0x5e922) [0x90c1922]
2016-06-10T15:21:44.271 INFO:tasks.rgw.client.0.smithi053.stdout: 9: (std::__throw_out_of_range(char const*)+0x67) [0x9113427]
2016-06-10T15:21:44.271 INFO:tasks.rgw.client.0.smithi053.stdout: 10: (()+0xb9f0a) [0x911cf0a]
2016-06-10T15:21:44.271 INFO:tasks.rgw.client.0.smithi053.stdout: 11: radosgw() [0x65b6ed]
2016-06-10T15:21:44.272 INFO:tasks.rgw.client.0.smithi053.stdout: 12: (RGWCORSRule::is_origin_present(char const*)+0x38) [0x65bef8]
2016-06-10T15:21:44.272 INFO:tasks.rgw.client.0.smithi053.stdout: 13: (RGWOp::generate_cors_headers(std::string&, std::string&, std::string&, std::string&, unsigned int*)+0x251) [0x626f11]
2016-06-10T15:21:44.272 INFO:tasks.rgw.client.0.smithi053.stdout: 14: (dump_access_control(req_state*, RGWOp*)+0x55) [0x524a05]
2016-06-10T15:21:44.272 INFO:tasks.rgw.client.0.smithi053.stdout: 15: (end_header(req_state*, RGWOp*, char const*, long, bool)+0x47) [0x524b77]
2016-06-10T15:21:44.273 INFO:tasks.rgw.client.0.smithi053.stdout: 16: (RGWListBucket_ObjStore_S3::send_response()+0x3f) [0x538f5f]
2016-06-10T15:21:44.273 INFO:tasks.rgw.client.0.smithi053.stdout: 17: radosgw() [0x577520]
2016-06-10T15:21:44.273 INFO:tasks.rgw.client.0.smithi053.stdout: 18: radosgw() [0x57812a]
2016-06-10T15:21:44.274 INFO:tasks.rgw.client.0.smithi053.stdout: 19: radosgw() [0x685b4f]
2016-06-10T15:21:44.274 INFO:tasks.rgw.client.0.smithi053.stdout: 20: radosgw() [0x687b1e]
2016-06-10T15:21:44.274 INFO:tasks.rgw.client.0.smithi053.stdout: 21: (()+0x8182) [0x8099182]
2016-06-10T15:21:44.274 INFO:tasks.rgw.client.0.smithi053.stdout: 22: (clone()+0x6d) [0x997d47d]
2016-06-10T15:21:44.275 INFO:tasks.rgw.client.0.smithi053.stdout: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2016-06-10T15:21:44.275 INFO:tasks.rgw.client.0.smithi053.stdout:
2016-06-10T15:21:47.048 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_cors_origin_response ... ERROR
2016-06-10T15:21:47.107 INFO:tasks.rgw.client.0.smithi053.stderr:daemon-helper: command crashed with signal 9
2016-06-10T15:23:16.055 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_read_1mb ... ERROR
2016-06-10T15:24:09.660 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_read_4mb ... ERROR
2016-06-10T15:24:51.838 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_read_8mb ... ERROR
2016-06-10T15:25:34.918 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_write_1mb ... ERROR
2016-06-10T15:26:21.103 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_write_4mb ... ERROR
2016-06-10T15:27:31.594 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_write_8mb ... ERROR
2016-06-10T15:28:31.061 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_dual_write_1mb ... ERROR
2016-06-10T15:29:44.585 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_dual_write_4mb ... ERROR
2016-06-10T15:30:42.687 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_dual_write_8mb ... ERROR
2016-06-10T15:31:35.657 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_conditional_write_1mb ... ERROR
2016-06-10T15:32:09.727 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ERROR
2016-06-10T15:32:59.947 INFO:teuthology.orchestra.run.smithi053.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ERROR