Project

General

Profile

Bug #3941

s3tests crash on bobtail

Added by Sage Weil about 11 years ago. Updated about 11 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
bobtail
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2013-01-27 01:06:19.720494 7f83b97fa700  1 ====== starting new request req=0x1ce0b20 =====
2013-01-27 01:06:19.720564 7f83b97fa700  2 req 396:0.000069::::initializing
2013-01-27 01:06:19.720622 7f83b97fa700 10 s->object=<NULL> s->bucket=test-client.0-klt9avnbmwr2xdg-114
2013-01-27 01:06:19.720629 7f83b97fa700 20 FCGI_ROLE=RESPONDER
2013-01-27 01:06:19.720630 7f83b97fa700 20 SCRIPT_URL=/test-client.0-klt9avnbmwr2xdg-114
2013-01-27 01:06:19.720632 7f83b97fa700 20 SCRIPT_URI=http://plana47.front.sepia.ceph.com:7280/test-client.0-klt9avnbmwr2xdg-114
2013-01-27 01:06:19.720634 7f83b97fa700 20 HTTP_AUTHORIZATION=
2013-01-27 01:06:19.720635 7f83b97fa700 20 RGW_LOG_LEVEL=20
2013-01-27 01:06:19.720636 7f83b97fa700 20 RGW_PRINT_CONTINUE=yes
2013-01-27 01:06:19.720637 7f83b97fa700 20 RGW_SHOULD_LOG=yes
2013-01-27 01:06:19.720638 7f83b97fa700 20 HTTP_HOST=plana47.front.sepia.ceph.com:7280
2013-01-27 01:06:19.720639 7f83b97fa700 20 CONTENT_LENGTH=0
2013-01-27 01:06:19.720640 7f83b97fa700 20 HTTP_ACCEPT_ENCODING=gzip, deflate, compress
2013-01-27 01:06:19.720641 7f83b97fa700 20 HTTP_ACCEPT=*/*
2013-01-27 01:06:19.720642 7f83b97fa700 20 HTTP_USER_AGENT=python-requests/1.1.0 CPython/2.7.3 Linux/3.6.0-ceph-00144-g0fa6ebc
2013-01-27 01:06:19.720644 7f83b97fa700 20 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
2013-01-27 01:06:19.720645 7f83b97fa700 20 SERVER_SIGNATURE=
2013-01-27 01:06:19.720646 7f83b97fa700 20 SERVER_SOFTWARE=Apache/2.2.22 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141
2013-01-27 01:06:19.720647 7f83b97fa700 20 SERVER_NAME=plana47.front.sepia.ceph.com
2013-01-27 01:06:19.720648 7f83b97fa700 20 SERVER_ADDR=127.0.1.1
2013-01-27 01:06:19.720649 7f83b97fa700 20 SERVER_PORT=7280
2013-01-27 01:06:19.720650 7f83b97fa700 20 REMOTE_ADDR=127.0.0.1
2013-01-27 01:06:19.720651 7f83b97fa700 20 DOCUMENT_ROOT=/tmp/cephtest/apache/htdocs
2013-01-27 01:06:19.720651 7f83b97fa700 20 SERVER_ADMIN=[no address given]
2013-01-27 01:06:19.720653 7f83b97fa700 20 SCRIPT_FILENAME=/tmp/cephtest/apache/htdocs/rgw.fcgi
2013-01-27 01:06:19.720654 7f83b97fa700 20 REMOTE_PORT=35446
2013-01-27 01:06:19.720655 7f83b97fa700 20 GATEWAY_INTERFACE=CGI/1.1
2013-01-27 01:06:19.720656 7f83b97fa700 20 SERVER_PROTOCOL=HTTP/1.1
2013-01-27 01:06:19.720657 7f83b97fa700 20 REQUEST_METHOD=POST
2013-01-27 01:06:19.720658 7f83b97fa700 20 QUERY_STRING=page=test-client.0-klt9avnbmwr2xdg-114&params=&bucket=test-client.0-klt9avnbmwr2xdg-114&key=foo.txt&etag=%2237b51d194a7513e45b56f6524f2d51f2%22
2013-01-27 01:06:19.720659 7f83b97fa700 20 REQUEST_URI=/test-client.0-klt9avnbmwr2xdg-114?bucket=test-client.0-klt9avnbmwr2xdg-114&key=foo.txt&etag=%2237b51d194a7513e45b56f6524f2d51f2%22
2013-01-27 01:06:19.720660 7f83b97fa700 20 SCRIPT_NAME=/test-client.0-klt9avnbmwr2xdg-114
2013-01-27 01:06:19.720663 7f83b97fa700  2 req 396:0.000168:s3:POST /test-client.0-klt9avnbmwr2xdg-114::getting op
2013-01-27 01:06:19.720670 7f83b97fa700  2 req 396:0.000176:s3:POST /test-client.0-klt9avnbmwr2xdg-114:post_obj:authorizing
2013-01-27 01:06:19.720675 7f83b97fa700  2 req 396:0.000181:s3:POST /test-client.0-klt9avnbmwr2xdg-114:post_obj:reading permissions
2013-01-27 01:06:19.720702 7f83b97fa700 20 get_obj_state: rctx=0x7f83580293e0 obj=.rgw:test-client.0-klt9avnbmwr2xdg-114 state=0x7f835804b0e8 s->prefetch_data=0
2013-01-27 01:06:19.720721 7f83b97fa700 10 moving .rgw+test-client.0-klt9avnbmwr2xdg-114 to cache LRU end
2013-01-27 01:06:19.720725 7f83b97fa700 10 cache get: name=.rgw+test-client.0-klt9avnbmwr2xdg-114 : hit
2013-01-27 01:06:19.720737 7f83b97fa700 20 get_obj_state: s->obj_tag was set empty
2013-01-27 01:06:19.720746 7f83b97fa700 10 moving .rgw+test-client.0-klt9avnbmwr2xdg-114 to cache LRU end
2013-01-27 01:06:19.720748 7f83b97fa700 10 cache get: name=.rgw+test-client.0-klt9avnbmwr2xdg-114 : hit
2013-01-27 01:06:19.720774 7f83b97fa700 20 rgw_get_bucket_info: bucket=test-client.0-klt9avnbmwr2xdg-114(@.rgw.buckets[4104.103]) owner foo.client.0
2013-01-27 01:06:19.720794 7f83b97fa700 20 get_obj_state: rctx=0x7f83580293e0 obj=test-client.0-klt9avnbmwr2xdg-114: state=0x7f835804b0e8 s->prefetch_data=0
2013-01-27 01:06:19.720824 7f83b97fa700 15 Read AccessControlPolicy<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>foo.client.0</ID><DisplayName>Mr. foo.client.0</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="ht
tp://www.w3.org/2001/XMLSchema-instance" xsi:type="Group"><URI>http://acs.amazonaws.com/groups/global/AllUsers</URI></Grantee><Permission>READ</Permission></Grant><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>f
oo.client.0</ID><DisplayName>Mr. foo.client.0</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
2013-01-27 01:06:19.720837 7f83b97fa700  2 req 396:0.000343:s3:POST /test-client.0-klt9avnbmwr2xdg-114:post_obj:verifying op permissions
2013-01-27 01:06:19.720842 7f83b97fa700  2 req 396:0.000348:s3:POST /test-client.0-klt9avnbmwr2xdg-114:post_obj:verifying op params
2013-01-27 01:06:19.720846 7f83b97fa700  2 req 396:0.000351:s3:POST /test-client.0-klt9avnbmwr2xdg-114:post_obj:executing
2013-01-27 01:06:19.755437 7f83b97fa700 -1 *** Caught signal (Aborted) **
 in thread 7f83b97fa700

 ceph version 0.56.1-87-ge6bceee (e6bceeedb0b77d23416560bd951326587470aacb)
 1: (ceph::BackTrace::BackTrace(int)+0x2d) [0x596143]
 2: /tmp/cephtest/binary/usr/local/bin/radosgw() [0x5958aa]
 3: (()+0xfcb0) [0x7f841cadacb0]
 4: (gsignal()+0x35) [0x7f841b015445]
 5: (abort()+0x17b) [0x7f841b018bab]
 6: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f841b96369d]
 7: (()+0xb5846) [0x7f841b961846]
 8: (()+0xb5873) [0x7f841b961873]
 9: (()+0xb596e) [0x7f841b96196e]
 10: (std::__throw_logic_error(char const*)+0x57) [0x7f841b90e787]
 11: (char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag)+0xa9) [0x7f841b94c509]
 12: (std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)+0x43) [0x7f841b94c5e3]
 13: (RGWPostObj_ObjStore_S3::get_params()+0xcc) [0x5662d8]
 14: (RGWPostObj::execute()+0x7b) [0x642b4d]
 15: (RGWProcess::handle_request(RGWRequest*)+0x679) [0x58c25b]
 16: (RGWProcess::RGWWQ::_process(RGWRequest*)+0x40) [0x58e92a]
 17: (ThreadPool::WorkQueue<RGWRequest>::_process(RGWRequest*, ThreadPool::TPHandle&)+0x32) [0x592dee]
 18: (ThreadPool::WorkQueue<RGWRequest>::_void_process(void*, ThreadPool::TPHandle&)+0x37) [0x592d89]
 19: (ThreadPool::worker(ThreadPool::WorkThread*)+0x713) [0x5ac667]
 20: (ThreadPool::WorkThread::entry()+0x23) [0x5ae389]
 21: (Thread::_entry_func(void*)+0x23) [0x5a569d]
 22: (()+0x7e9a) [0x7f841cad2e9a]
 23: (clone()+0x6d) [0x7f841b0d14bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

is this somethign like commit:e1da85f286838cdd3a6329840cec748c6a11fd26 ?

ubuntu@teuthology:/a/teuthology-2013-01-26_20:00:03-regression-bobtail-master-basic/6124
(and several other tests in that run)

Associated revisions

Revision 09522e5a (diff)
Added by Yehuda Sadeh about 11 years ago

rgw: fix crash when missing content-type in POST object

Fixes: #3941
This fixes a crash when handling S3 POST request and content type
is not provided.

Signed-off-by: Yehuda Sadeh <>

Revision f41010c4 (diff)
Added by Yehuda Sadeh about 11 years ago

rgw: fix crash when missing content-type in POST object

Fixes: #3941
This fixes a crash when handling S3 POST request and content type
is not provided.

Signed-off-by: Yehuda Sadeh <>

Revision a8964107 (diff)
Added by Yehuda Sadeh about 11 years ago

rgw: fix crash when missing content-type in POST object

Fixes: #3941
This fixes a crash when handling S3 POST request and content type
is not provided.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit f41010c44b3a4489525d25cd35084a168dc5f537)

History

#1 Updated by Ian Colle about 11 years ago

  • Assignee set to Yehuda Sadeh

#2 Updated by caleb miles about 11 years ago

  • Assignee changed from Yehuda Sadeh to caleb miles

#3 Updated by Yehuda Sadeh about 11 years ago

Yeah, similar to that other issue (#3777)

string req_content_type_str = s->env->get("CONTENT_TYPE");

CONTENT_TYPE is not set for some reason in the teuthology environment.
Need to fix this one, also to check why it's not set. I'm also reopening #3777, we need to make sure that all such calls are covered.

#4 Updated by Yehuda Sadeh about 11 years ago

  • Status changed from New to Resolved
  • Backport set to bobtail

Crash fixed, commit:f41010c44b3a4489525d25cd35084a168dc5f537.
Also, pushed a change to s3-tests.git, setting a required version for the 'requests' package. The recent version that is used doesn't handle POST redirect correctly.

Also available in: Atom PDF