Project

General

Profile

Actions
Copy link

Bug #44240

open

rgw: segmentation fault RGWBulkUploadOp_ObjStore_SWIFT::send_response calling RGWFormatter_Plain

Added by J. Eric Ivancich about 4 years ago. Updated about 4 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
Ceph - v15.0.0
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Found during teuthology rgw suite. Link: http://qa-proxy.ceph.com/teuthology/ivancich-2020-02-20_21:35:10-rgw-wip-listing-w-namespaces-distro-basic-smithi/4784951/remote/smithi045/crash/posted/2020-02-20T21%3A55%3A33.035055Z_ca7dc832-ba37-4e57-a0ad-5e2395e7522f/

0> 2020-02-20T21:55:33.029+0000 7f57ebe97700 -1 ** Caught signal (Segmentation fault) *
 in thread 7f57ebe97700 thread_name:radosgw
ceph version 15.1.0-908-ge3780fa (e3780fa87839ba0b157725ed36103e410d324d43) octopus (rc)
 1: (()+0x3ef20) [0x7f59a5d2df20]
 2: (()+0x18e5a1) [0x7f59a5e7d5a1]
 3: (IO_vfprintf()+0x2143) [0x7f59a5d4c4d3]
 4: (_vsnprintf_chk()+0xa9) [0x7f59a5e21169]
 5: (RGWFormatter_Plain::write_data(char const*, ...)+0x102) [0x7f59a6618492]
 6: (RGWFormatter_Plain::dump_value_int(std::basic_string_view<char, std::char_traits<char> >, char const*, ...)+0x192) [0x7f59a6618952]
 7: (RGWBulkUploadOp_ObjStore_SWIFT::send_response()+0x163) [0x7f59a64d4e43]
 8: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, bool)+0x14e2) [0x7f59a64a44a2]
 9: (process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, int*)+0x336c) [0x7f59a64a85cc]
 10: (()+0x347e1d) [0x7f59a6427e1d]
 11: (()+0x348e14) [0x7f59a6428e14]
 12: (()+0x348f48) [0x7f59a6428f48]
 13: (make_fcontext()+0x2f) [0x7f59a6a53bcf]
182> 2020-02-20T21:55:32.665+0000 7f58066cc700  1 ====== starting new request req=0x7f59902c2870 =====
  -181> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s initializing for trans_id = tx00000000000000000002e-005e4f0054-1087-default
  -180> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s getting op 4
  -179> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata verifying requester
  -178> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata normalizing buckets and tenants
  -177> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata init permissions
  -176> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata recalculating target
  -175> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata reading permissions
  -174> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata init op
  -173> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata verifying op mask
  -172> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata verifying op permissions
  -171> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 req 46 0s swift:put_account_metadata Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=b2ebbe5bfb8449a79f8bf9ce9990ee2c, acct_name=tempest-AccountQuotasTest-1146143091, perm_mask=15, is_admin=1) mask=50
  -170> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Searching permissions for uid=b2ebbe5bfb8449a79f8bf9ce9990ee2c
  -169> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Permissions for user not found
  -168> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Searching permissions for uid=b2ebbe5bfb8449a79f8bf9ce9990ee2c$b2ebbe5bfb8449a79f8bf9ce9990ee2c
  -167> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Permissions for user not found
  -166> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Searching permissions for group=1 mask=50
  -165> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Permissions for group not found
  -164> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Searching permissions for group=2 mask=50
  -163> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 Permissions for group not found
  -162> 2020-02-20T21:55:32.665+0000 7f58066cc700  5 req 46 0s swift:put_account_metadata - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=b2ebbe5bfb8449a79f8bf9ce9990ee2c, acct_name=tempest-AccountQuotasTest-1146143091, perm_mask=15, is_admin=1), owner=d9c74da931e74336966620f17ccb8709, perm=0
  -161> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 overriding permissions due to admin operation
  -160> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata verifying op params
  -159> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata pre-executing
  -158> 2020-02-20T21:55:32.665+0000 7f58066cc700  2 req 46 0s swift:put_account_metadata executing
  -157> 2020-02-20T21:55:32.669+0000 7f57ffebf700  2 req 46 0.004000000s swift:put_account_metadata completing
  -156> 2020-02-20T21:55:32.669+0000 7f57ffebf700  5 nothing to log for operation
  -155> 2020-02-20T21:55:32.669+0000 7f57ffebf700  2 req 46 0.004000000s swift:put_account_metadata op status=1902
  -154> 2020-02-20T21:55:32.669+0000 7f57ffebf700  2 req 46 0.004000000s swift:put_account_metadata http status=204
  -153> 2020-02-20T21:55:32.669+0000 7f57ffebf700  1 ====== req done req=0x7f59902c2870 op status=1902 http_status=204 latency=0.004000000s ======
  -152> 2020-02-20T21:55:32.669+0000 7f5802ec5700  1 ====== starting new request req=0x7f59902c2870 =====
  -151> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s initializing for trans_id = tx00000000000000000002f-005e4f0054-1087-default
  -150> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s getting op 1
  -149> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj verifying requester
  -148> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj normalizing buckets and tenants
  -147> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj init permissions
  -146> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj recalculating target
  -145> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj reading permissions
  -144> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj init op
  -143> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj verifying op mask
  -142> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj verifying op permissions
  -141> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 req 47 0s swift:put_obj Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0) mask=50
  -140> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709
  -139> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Found permission: 15
  -138> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709$d9c74da931e74336966620f17ccb8709
  -137> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Permissions for user not found
  -136> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Searching permissions for group=1 mask=50
  -135> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Permissions for group not found
  -134> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Searching permissions for group=2 mask=50
  -133> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 Permissions for group not found
  -132> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 req 47 0s swift:put_obj - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0), owner=d9c74da931e74336966620f17ccb8709, perm=2
  -131> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj verifying op params
  -130> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj pre-executing
  -129> 2020-02-20T21:55:32.669+0000 7f5802ec5700  2 req 47 0s swift:put_obj executing
  -128> 2020-02-20T21:55:32.669+0000 7f5802ec5700  5 req 47 0s swift:put_obj NOTICE: call to do_aws4_auth_completion
  -127> 2020-02-20T21:55:32.673+0000 7f57fbeb7700  2 req 47 0.004000000s swift:put_obj completing
  -126> 2020-02-20T21:55:32.673+0000 7f57fbeb7700  2 req 47 0.004000000s swift:put_obj op status=1900
  -125> 2020-02-20T21:55:32.673+0000 7f57fbeb7700  2 req 47 0.004000000s swift:put_obj http status=201
  -124> 2020-02-20T21:55:32.673+0000 7f57fbeb7700  1 ====== req done req=0x7f59902c2870 op status=1900 http_status=201 latency=0.004000000s ======
  -123> 2020-02-20T21:55:32.673+0000 7f57fdebb700  1 ====== starting new request req=0x7f59902c2870 =====
  -122> 2020-02-20T21:55:32.673+0000 7f57fdebb700  2 req 48 0s initializing for trans_id = tx000000000000000000030-005e4f0054-1087-default
  -121> 2020-02-20T21:55:32.673+0000 7f57fdebb700  2 req 48 0s getting op 4
  -120> 2020-02-20T21:55:32.673+0000 7f57fdebb700  2 req 48 0s swift:put_account_metadata verifying requester
  -119> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata normalizing buckets and tenants
  -118> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata init permissions
  -117> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata recalculating target
  -116> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata reading permissions
  -115> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata init op
  -114> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata verifying op mask
  -113> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata verifying op permissions
  -112> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 req 48 0.004000000s swift:put_account_metadata Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=b2ebbe5bfb8449a79f8bf9ce9990ee2c, acct_name=tempest-AccountQuotasTest-1146143091, perm_mask=15, is_admin=1) mask=50
  -111> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Searching permissions for uid=b2ebbe5bfb8449a79f8bf9ce9990ee2c
  -110> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Permissions for user not found
  -109> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Searching permissions for uid=b2ebbe5bfb8449a79f8bf9ce9990ee2c$b2ebbe5bfb8449a79f8bf9ce9990ee2c
  -108> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Permissions for user not found
  -107> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Searching permissions for group=1 mask=50
  -106> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Permissions for group not found
  -105> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Searching permissions for group=2 mask=50
  -104> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 Permissions for group not found
  -103> 2020-02-20T21:55:32.677+0000 7f57fdebb700  5 req 48 0.004000000s swift:put_account_metadata - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=b2ebbe5bfb8449a79f8bf9ce9990ee2c, acct_name=tempest-AccountQuotasTest-1146143091, perm_mask=15, is_admin=1), owner=d9c74da931e74336966620f17ccb8709, perm=0
  -102> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 overriding permissions due to admin operation
  -101> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata verifying op params
  -100> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata pre-executing
   -99> 2020-02-20T21:55:32.677+0000 7f57fdebb700  2 req 48 0.004000000s swift:put_account_metadata executing
   -98> 2020-02-20T21:55:32.677+0000 7f57f86b0700  2 req 48 0.004000000s swift:put_account_metadata completing
   -97> 2020-02-20T21:55:32.677+0000 7f57f86b0700  5 nothing to log for operation
   -96> 2020-02-20T21:55:32.677+0000 7f57f86b0700  2 req 48 0.004000000s swift:put_account_metadata op status=1902
   -95> 2020-02-20T21:55:32.677+0000 7f57f86b0700  2 req 48 0.004000000s swift:put_account_metadata http status=204
   -94> 2020-02-20T21:55:32.677+0000 7f57f86b0700  1 ====== req done req=0x7f59902c2870 op status=1902 http_status=204 latency=0.004000000s ======
   -93> 2020-02-20T21:55:32.677+0000 7f57faeb5700  1 ====== starting new request req=0x7f59902c2870 =====
   -92> 2020-02-20T21:55:32.677+0000 7f57faeb5700  2 req 49 0s initializing for trans_id = tx000000000000000000031-005e4f0054-1087-default
   -91> 2020-02-20T21:55:32.677+0000 7f57faeb5700  2 req 49 0s getting op 0
   -90> 2020-02-20T21:55:32.677+0000 7f57faeb5700  2 req 49 0s swift:list_bucket verifying requester
   -89> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket normalizing buckets and tenants
   -88> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket init permissions
   -87> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket recalculating target
   -86> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket reading permissions
   -85> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket init op
   -84> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket verifying op mask
   -83> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket verifying op permissions
   -82> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 req 49 0.003999999s swift:list_bucket Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0) mask=49
   -81> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709
   -80> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Found permission: 15
   -79> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709$d9c74da931e74336966620f17ccb8709
   -78> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Permissions for user not found
   -77> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Searching permissions for group=1 mask=49
   -76> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Permissions for group not found
   -75> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Searching permissions for group=2 mask=49
   -74> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 Permissions for group not found
   -73> 2020-02-20T21:55:32.681+0000 7f57faeb5700  5 req 49 0.003999999s swift:list_bucket - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0), owner=d9c74da931e74336966620f17ccb8709, perm=1
   -72> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket verifying op params
   -71> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket pre-executing
   -70> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket executing
   -69> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket completing
   -68> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket op status=0
   -67> 2020-02-20T21:55:32.681+0000 7f57faeb5700  2 req 49 0.003999999s swift:list_bucket http status=200
   -66> 2020-02-20T21:55:32.681+0000 7f57faeb5700  1 ====== req done req=0x7f59902c2870 op status=0 http_status=200 latency=0.003999999s ======
   -65> 2020-02-20T21:55:32.681+0000 7f57f5eab700  1 ====== starting new request req=0x7f59902c2870 =====
   -64> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s initializing for trans_id = tx000000000000000000032-005e4f0054-1087-default
   -63> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s getting op 2
   -62> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj verifying requester
   -61> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj normalizing buckets and tenants
   -60> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj init permissions
   -59> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj recalculating target
   -58> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj reading permissions
   -57> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj init op
   -56> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj verifying op mask
   -55> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj verifying op permissions
   -54> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 req 50 0s swift:delete_obj Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0) mask=50
   -53> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709
   -52> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Found permission: 15
   -51> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Searching permissions for uid=d9c74da931e74336966620f17ccb8709$d9c74da931e74336966620f17ccb8709
   -50> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Permissions for user not found
   -49> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Searching permissions for group=1 mask=50
   -48> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Permissions for group not found
   -47> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Searching permissions for group=2 mask=50
   -46> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 Permissions for group not found
   -45> 2020-02-20T21:55:32.681+0000 7f57f5eab700  5 req 50 0s swift:delete_obj - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(d9c74da931e74336966620f17ccb8709) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=d9c74da931e74336966620f17ccb8709, acct_name=tempest-AccountQuotasTest-1107054290, perm_mask=15, is_admin=0), owner=d9c74da931e74336966620f17ccb8709, perm=2
   -44> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj verifying op params
   -43> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj pre-executing
   -42> 2020-02-20T21:55:32.681+0000 7f57f5eab700  2 req 50 0s swift:delete_obj executing
   -41> 2020-02-20T21:55:32.685+0000 7f57eee9d700  2 req 50 0.004000000s swift:delete_obj completing
   -40> 2020-02-20T21:55:32.685+0000 7f57eee9d700  2 req 50 0.004000000s swift:delete_obj op status=0
   -39> 2020-02-20T21:55:32.685+0000 7f57eee9d700  2 req 50 0.004000000s swift:delete_obj http status=204
   -38> 2020-02-20T21:55:32.685+0000 7f57eee9d700  1 ====== req done req=0x7f59902c2870 op status=0 http_status=204 latency=0.004000000s ======
   -37> 2020-02-20T21:55:33.013+0000 7f581aef5700  0 req 40 2.359999948s swift:bulk_upload validated token: tempest-BulkTest-111146302:tempest-BulkTest-111146302 expires: 1582239330
   -36> 2020-02-20T21:55:33.017+0000 7f581aef5700  0 req 40 2.363999948s swift:bulk_upload NOTICE: couldn't map swift user f176a3367e1b4debb745477e665ef391
   -35> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload normalizing buckets and tenants
   -34> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload init permissions
   -33> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload recalculating target
   -32> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload reading permissions
   -31> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload init op
   -30> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload verifying op mask
   -29> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload verifying op permissions
   -28> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 req 40 2.363999948s swift:bulk_upload Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(f176a3367e1b4debb745477e665ef391) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=f176a3367e1b4debb745477e665ef391, acct_name=tempest-BulkTest-111146302, perm_mask=15, is_admin=0) mask=50
   -27> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Searching permissions for uid=f176a3367e1b4debb745477e665ef391
   -26> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Found permission: 15
   -25> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Searching permissions for uid=f176a3367e1b4debb745477e665ef391$f176a3367e1b4debb745477e665ef391
   -24> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Permissions for user not found
   -23> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Searching permissions for group=1 mask=50
   -22> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Permissions for group not found
   -21> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Searching permissions for group=2 mask=50
   -20> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 Permissions for group not found
   -19> 2020-02-20T21:55:33.017+0000 7f581aef5700  5 req 40 2.363999948s swift:bulk_upload - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(f176a3367e1b4debb745477e665ef391) > rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=f176a3367e1b4debb745477e665ef391, acct_name=tempest-BulkTest-111146302, perm_mask=15, is_admin=0), owner=f176a3367e1b4debb745477e665ef391, perm=2
   -18> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload verifying op params
   -17> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload pre-executing
   -16> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload executing
   -15> 2020-02-20T21:55:33.017+0000 7f581aef5700  2 req 40 2.363999948s swift:bulk_upload handling regular directory
   -14> 2020-02-20T21:55:33.025+0000 7f57ece99700  2 req 40 2.371999948s swift:bulk_upload handling regular file
   -13> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 req 40 2.371999948s swift:bulk_upload Searching permissions for identity=rgw::auth::ThirdPartyAccountApplier(f176a3367e1b4debb745477e665ef391) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=f176a3367e1b4debb745477e665ef391, acct_name=tempest-BulkTest-111146302, perm_mask=15, is_admin=0) mask=50
   -12> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Searching permissions for uid=f176a3367e1b4debb745477e665ef391
   -11> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Found permission: 15
   -10> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Searching permissions for uid=f176a3367e1b4debb745477e665ef391$f176a3367e1b4debb745477e665ef391
    -9> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Permissions for user not found
    -8> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Searching permissions for group=1 mask=50
    -7> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Permissions for group not found
    -6> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Searching permissions for group=2 mask=50
    -5> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 Permissions for group not found
    -4> 2020-02-20T21:55:33.025+0000 7f57ece99700  5 req 40 2.371999948s swift:bulk_upload - Getting permissions done for identity=rgw::auth::ThirdPartyAccountApplier(f176a3367e1b4debb745477e665ef391) -> rgw::auth::SysReqApplier -> rgw::auth::RemoteApplier(acct_user=f176a3367e1b4debb745477e665ef391, acct_name=tempest-BulkTest-111146302, perm_mask=15, is_admin=0), owner=f176a3367e1b4debb745477e665ef391, perm=2
    -3> 2020-02-20T21:55:33.029+0000 7f57ebe97700  2 req 40 2.375999948s swift:bulk_upload an empty block
    -2> 2020-02-20T21:55:33.029+0000 7f57ebe97700  2 req 40 2.375999948s swift:bulk_upload an empty block
    -1> 2020-02-20T21:55:33.029+0000 7f57ebe97700  2 req 40 2.375999948s swift:bulk_upload completing
     0> 2020-02-20T21:55:33.029+0000 7f57ebe97700 -1 ** Caught signal (Segmentation fault) *
 in thread 7f57ebe97700 thread_name:radosgw
Actions
Copy link
 #1

Updated by Matthew Oliver about 4 years ago

I'll give swift bulk upload a whirl and see if I can reproduce locally.

Actions
Copy link
 #2

Updated by Nathan Cutler about 4 years ago

Link to teuthology run Eric is referring to: http://pulpito.ceph.com/ivancich-2020-02-20_21:35:10-rgw-wip-listing-w-namespaces-distro-basic-smithi/

@Eric Chen - do you know if this is reproducible on the octopus branch?

Actions
Copy link
 #3

Updated by J. Eric Ivancich about 4 years ago

Nathan Cutler wrote:

Link to teuthology run Eric is referring to: http://pulpito.ceph.com/ivancich-2020-02-20_21:35:10-rgw-wip-listing-w-namespaces-distro-basic-smithi/

@Eric Chen - do you know if this is reproducible on the octopus branch?

Sorry, I haven't tried, @Nathan Weinberg.

Actions
Copy link
 #4

Updated by Matthew Oliver about 4 years ago

OK, tracked down the branch that is in this issue. It's sitting on ceph-ci. When I pull it, build ceph, run vstart and try and bulk upload using the swift API I do indeed get the segfault. So finally managed to recreate.

I couldn't on master or on octopus, so is definitely related to this branch. Next to find the PR that created it :)

Also now that I've recreated it, will now delve deeper.

Actions
Copy link

Also available in: Atom PDF