Project

General

Profile

Actions
Copy link

Bug #16033

open

A file passed to bulk delete request is not being parsed correctly

Added by Shilpa MJ almost 8 years ago. Updated almost 8 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Load a file with the list of objects to be deleted. For example:
  1. cat del
    container1/segment_aa\ncontainer1/segment_ab

DELETE request with ?bulk-delete query returns:

  1. curl -i -X DELETE "http://rgw:8080/swift/v1?bulk-delete" -H "X-Auth-Token: AUTH_rgwtk0e00000074" --data-binary "'cat ./del'"

HTTP/1.1 200 OK
X-Trans-Id: tx00000000000000000009e-005746a2ec-102e-default
Content-Type: text/plain; charset=utf-8
Content-Length: 87
Date: Thu, 26 May 2016 07:17:00 GMT

Number Deleted: 0
Number Not Found: 1
Response Body:
Response Status: 200 OK
Errors:

2016-05-26 07:17:00.181671 7ff468ff9700 20 QUERY_STRING=bulk-delete
2016-05-26 07:17:00.181672 7ff468ff9700 20 REMOTE_USER=
2016-05-26 07:17:00.181672 7ff468ff9700 20 REQUEST_METHOD=DELETE
2016-05-26 07:17:00.181673 7ff468ff9700 20 REQUEST_URI=/swift/v1
2016-05-26 07:17:00.181673 7ff468ff9700 20 SCRIPT_URI=/swift/v1
2016-05-26 07:17:00.181674 7ff468ff9700 20 SERVER_PORT=8080
2016-05-26 07:17:00.181676 7ff468ff9700 1 ====== starting new request req=0x7ff468ff3710 =====
2016-05-26 07:17:00.181692 7ff468ff9700 2 req 158:0.000016::DELETE /swift/v1::initializing for trans_id = tx00000000000000000009e-005746a2ec-102e-default
2016-05-26 07:17:00.181699 7ff468ff9700 20 subdomain= domain= in_hosted_domain=0 in_hosted_domain_s3website=0
2016-05-26 07:17:00.181725 7ff468ff9700 10 ver=v1 first= req=
2016-05-26 07:17:00.181729 7ff468ff9700 10 handler=29RGWHandler_REST_Service_SWIFT
2016-05-26 07:17:00.181731 7ff468ff9700 2 req 158:0.000055:swift:DELETE /swift/v1::getting op 2
2016-05-26 07:17:00.181733 7ff468ff9700 10 op=28RGWBulkDelete_ObjStore_SWIFT
2016-05-26 07:17:00.181734 7ff468ff9700 2 req 158:0.000059:swift:DELETE /swift/v1:bulk_delete:authorizing
2016-05-26 07:17:00.181750 7ff468ff9700 10 swift_user=testuser:swift
2016-05-26 07:17:00.181763 7ff468ff9700 20 build_token token=0e00000074657374757365723a7377696674a153ccb2dc27f91388e2475705a3812b
2016-05-26 07:17:00.181803 7ff468ff9700 2 req 158:0.000127:swift:DELETE /swift/v1:bulk_delete:normalizing buckets and tenants
2016-05-26 07:17:00.181810 7ff468ff9700 10 s->object=<NULL> s->bucket=
2016-05-26 07:17:00.181813 7ff468ff9700 2 req 158:0.000137:swift:DELETE /swift/v1:bulk_delete:init permissions
2016-05-26 07:17:00.181817 7ff468ff9700 2 req 158:0.000142:swift:DELETE /swift/v1:bulk_delete:recalculating target
2016-05-26 07:17:00.181819 7ff468ff9700 2 req 158:0.000143:swift:DELETE /swift/v1:bulk_delete:reading permissions
2016-05-26 07:17:00.181821 7ff468ff9700 2 req 158:0.000145:swift:DELETE /swift/v1:bulk_delete:init op
2016-05-26 07:17:00.181822 7ff468ff9700 2 req 158:0.000147:swift:DELETE /swift/v1:bulk_delete:verifying op mask
2016-05-26 07:17:00.181824 7ff468ff9700 20 required_mask= 4 user.op_mask=7
2016-05-26 07:17:00.181825 7ff468ff9700 2 req 158:0.000150:swift:DELETE /swift/v1:bulk_delete:verifying op permissions
2016-05-26 07:17:00.181827 7ff468ff9700 2 req 158:0.000151:swift:DELETE /swift/v1:bulk_delete:verifying op params
2016-05-26 07:17:00.181828 7ff468ff9700 2 req 158:0.000153:swift:DELETE /swift/v1:bulk_delete:pre-executing
2016-05-26 07:17:00.181830 7ff468ff9700 2 req 158:0.000155:swift:DELETE /swift/v1:bulk_delete:executing
2016-05-26 07:17:00.182006 7ff468ff9700 20 extracted Bulk Delete entry: 'cat ./del'
2016-05-26 07:17:00.182012 7ff468ff9700 20 in delete_chunk
2016-05-26 07:17:00.182014 7ff468ff9700 20 bulk deleting path: 'cat ./del'
2016-05-26 07:17:00.182027 7ff468ff9700 20 get_system_obj_state: rctx=0x7ff468ff2e50 obj=default.rgw.data.root:'cat . state=0x7ff41c00d4d8 s->prefetch_data=0
2016-05-26 07:17:00.182037 7ff468ff9700 10 cache get: name=default.rgw.data.root+'cat . : type miss (requested=22, cached=0)
58500 ==== osd_op_reply(190478 'cat . [call,getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v7 ==== 210+0+0 (3580360390 0 0) 0x7ff49c0064b0 con 0x7ff59400ca30
2016-05-26 07:17:00.182752 7ff468ff9700 10 cache put: name=default.rgw.data.root+'cat . info.flags=0
2016-05-26 07:17:00.182763 7ff468ff9700 10 moving default.rgw.data.root+'cat . to cache LRU end
2016-05-26 07:17:00.182771 7ff468ff9700 20 cannot find bucket = 'cat .
2016-05-26 07:17:00.182773 7ff468ff9700 2 req 158:0.001097:swift:DELETE /swift/v1:bulk_delete:completing
2016-05-26 07:17:00.182860 7ff468ff9700 2 req 158:0.001184:swift:DELETE /swift/v1:bulk_delete:op status=0
2016-05-26 07:17:00.182867 7ff468ff9700 2 req 158:0.001191:swift:DELETE /swift/v1:bulk_delete:http status=200
2016-05-26 07:17:00.182870 7ff468ff9700 1 ====== req done req=0x7ff468ff3710 op status=0 http_status=200 ======

Actions
Copy link
 #1

Updated by Radoslaw Zarzynski almost 8 years ago

It looks that the client request has wrong body:

2016-05-26 07:17:00.182006 7ff468ff9700 20 extracted Bulk Delete entry: 'cat ./del'
2016-05-26 07:17:00.182012 7ff468ff9700 20 in delete_chunk
2016-05-26 07:17:00.182014 7ff468ff9700 20 bulk deleting path: 'cat ./del'

Following command should work:

curl -i -X DELETE "http://rgw:8080/swift/v1?bulk-delete" -H "X-Auth-Token: AUTH_rgwtk0e00000074" --data-binary "`cat ./del`"

The only difference is: 
`
instead of: 
'

Actions
Copy link
 #2

Updated by Shilpa MJ almost 8 years ago

Radoslaw Zarzynski wrote:

It looks that the client request has wrong body:

[...]

Following command should work:
[...]
The only difference is: [...] instead of: [...]

Thanks that works. Is the contents of the file right?

  1. cat del
    container1/segment_aa\ncontainer1/segment_ab

2016-05-27 06:54:57.379191 7f92757fa700 20 get_obj_state: rctx=0x7f92757f3e50 obj=container1:segment_aa\ncontainer1/segment_ab state=0x7f923800d778 s->prefetch_data=0
2016-05-27 06:54:57.379658 7f92757fa700 20 cannot find entry container1/segment_aa\ncontainer1/segment_ab

Actions
Copy link
 #3

Updated by Pritha Srivastava almost 8 years ago

I was trying to find the maximum number of deletes allowed in a bulk delete operation and found that the following works:

curl -i -X DELETE "http://localhost:8000/swift/v1?bulk-delete" -H "X-Auth-Token: AUTH_rgwtk0b000000746573743a7465737465724255926aadc363a40e5a495709f2f7185fc2a19ef19f951a9f5adf4b25456b8493269788" --data-binary @input.txt

i.e. specifying the input file with an '@'.

And the input file is in the following form:
container1/object1
container1/object2
container1/object3
container1/object4
container1/object5
container1/object6
container1/object7
container1/object8
container1/object9
container1/object10

Actions
Copy link
 #4

Updated by Shilpa MJ almost 8 years ago

Pritha Srivastava wrote:

I was trying to find the maximum number of deletes allowed in a bulk delete operation and found that the following works:

curl -i -X DELETE "http://localhost:8000/swift/v1?bulk-delete" -H "X-Auth-Token: AUTH_rgwtk0b000000746573743a7465737465724255926aadc363a40e5a495709f2f7185fc2a19ef19f951a9f5adf4b25456b8493269788" --data-binary @input.txt

i.e. specifying the input file with an '@'.

And the input file is in the following form:
container1/object1
container1/object2
container1/object3
container1/object4
container1/object5
container1/object6
container1/object7
container1/object8
container1/object9
container1/object10

Thanks Pritha. This works. However, seems like the '\n' separator won't work.

Actions
Copy link

Also available in: Atom PDF