Project

General

Profile

Actions
Copy link

Bug #13240

closed

AssertionError: 31457280 != 30932992: s3tests.functional.test_s3.test_multipart_upload_resend_part

Added by Vasu Kulkarni over 8 years ago. Updated over 8 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Samuel Just
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

quite a few failures due to this in rgw suite, one of the logs : http://qa-proxy.ceph.com/teuthology/teuthology-2015-09-22_23:02:02-rgw-master---basic-multi/1066229/teuthology.log

2015-09-25T00:06:09.427 INFO:teuthology.orchestra.run.burnupi49.stderr:test_realistic.TestFiles.test_random_file_valid ... ok
2015-09-25T00:06:09.427 INFO:teuthology.orchestra.run.burnupi49.stderr:
2015-09-25T00:06:09.427 INFO:teuthology.orchestra.run.burnupi49.stderr:======================================================================
2015-09-25T00:06:09.427 INFO:teuthology.orchestra.run.burnupi49.stderr:FAIL: s3tests.functional.test_s3.test_multipart_upload_resend_part
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:----------------------------------------------------------------------
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:Traceback (most recent call last):
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:  File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:    self.test(*self.arg)
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:  File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4687, in test_multipart_upload_resend_part
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:    _check_upload_multipart_resend(bucket, key, objlen, [0])
2015-09-25T00:06:09.428 INFO:teuthology.orchestra.run.burnupi49.stderr:  File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4668, in _check_upload_multipart_resend
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:    eq(k.size, len(test_string))
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:AssertionError: 31457280 != 30932992
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:-------------------- >> begin captured logging << --------------------
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: path=/test-client.0-yx3f3b9zow79iaz-248/
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: auth_path=/test-client.0-yx3f3b9zow79iaz-248/
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Method: PUT
2015-09-25T00:06:09.429 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Path: /test-client.0-yx3f3b9zow79iaz-248/
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Data:
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Headers: {}
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Host: burnupi49.front.sepia.ceph.com:7280
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Port: 7280
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Params: {}
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Token: None
2015-09-25T00:06:09.430 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: StringToSign:
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:PUT
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:Fri, 25 Sep 2015 07:03:42 GMT
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:/test-client.0-yx3f3b9zow79iaz-248/
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Signature:
2015-09-25T00:06:09.431 INFO:teuthology.orchestra.run.burnupi49.stderr:AWS CDSFNPTYQPURHDQZECOL:31fxvltcU17lA5ZZtXNoPZVThlw=
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Final headers: {'Date': 'Fri, 25 Sep 2015 07:03:42 GMT', 'Content-Length': '0', 'Authorization': u'AWS CDSFNPTYQPURHDQZECOL:31fxvltcU17lA5ZZtXNoPZVThlw=', 'User-Agent': 'Boto/2.38.0 Python/2.7.6 Linux/3.13.0-63-generic'}
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: Response headers: [('date', 'Fri, 25 Sep 2015 07:03:42 GMT'), ('transfer-encoding', 'chunked'), ('x-amz-request-id', 'tx00000000000000000043a-005604f1ce-1016-default'), ('server', 'Apache/2.4.7 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141')]
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: path=/test-client.0-yx3f3b9zow79iaz-248/mymultipart
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: auth_path=/test-client.0-yx3f3b9zow79iaz-248/mymultipart
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: path=/test-client.0-yx3f3b9zow79iaz-248/mymultipart?uploads
2015-09-25T00:06:09.432 INFO:teuthology.orchestra.run.burnupi49.stderr:boto: DEBUG: auth_path=/test-client.0-yx3f3b9zow79iaz-248/mymultipart?uploads
Actions
Copy link
 #1

Updated by Yehuda Sadeh over 8 years ago

Note that it seems that all these type of failures (data length mismatch happen with the ec backend).

Actions
Copy link
 #2

Updated by Yehuda Sadeh over 8 years ago

  • Assignee set to Samuel Just
Actions
Copy link
 #3

Updated by Samuel Just over 8 years ago

Ugh, won't let me link issues across projects. I think this is probably the same bug as http://tracker.ceph.com/issues/12983. Can you retry with ceph/wip-12983?

Actions
Copy link
 #4

Updated by Samuel Just over 8 years ago

  • Status changed from New to Need More Info
  • Assignee changed from Samuel Just to Vasu Kulkarni

(Reassign back to me when it's back in my court!)

Actions
Copy link
 #5

Updated by Vasu Kulkarni over 8 years ago

  • Status changed from Need More Info to New
  • Assignee changed from Vasu Kulkarni to Samuel Just
Actions
Copy link
 #6

Updated by Samuel Just over 8 years ago

  • Status changed from New to 7

Found it, full handling reorders ops with need_promote() set.

Actions
Copy link
 #7

Updated by Samuel Just over 8 years ago

  • Status changed from 7 to Resolved
Actions
Copy link

Also available in: Atom PDF