Actions
Bug #7606
closeddumpling-x parallel ceph-qa-suite errors.
Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
% Done:
0%
Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
The yaml's generated from ceph-qa-suite for dumpling-x are invalid.
I think that the tasks: field in the 4-final-workload directory is incorrect.
Updated by Anonymous about 10 years ago
Two issues fixed in experimental
1. all: for install.upgrades induced coredumps because client was being reinstalled too -- changed to mon.a: and mon.b.
2 -- branch: dumpling needed to be added to workunit for client.1 (defaulted to HEAD).
Updated by Anonymous about 10 years ago
The upgrade test is getting a lot of failures with:
S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.1.conf /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw'
I used pdb.set_trace() to stop the execution of teuthology just prior to this command being run. I then ssh'ed to the remote machine and manually ran this command. I got the following output
.
.
.
s3tests.fuzz.test.test_fuzzer.test_null_child ... ok
s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok
s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok
s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok
s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok
======================================================================
ERROR: s3tests.functional.test_s3.test_multipart_upload_contents
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4140, in test_multipart_upload_contents
test_string=key.get_contents_as_string()
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 1730, in get_contents_as_string
response_headers=response_headers)
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 1603, in get_contents_to_file
response_headers=response_headers)
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 1435, in get_file
query_args=None)
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 1488, in _get_file_internal
for bytes in self:
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 370, in next
self.close()
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py", line 352, in close
self.resp.read()
File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/connection.py", line 413, in read
self._cached_response = httplib.HTTPResponse.read(self)
File "/usr/lib/python2.7/httplib.py", line 548, in read
s = self._safe_read(self.length)
File "/usr/lib/python2.7/httplib.py", line 649, in _safe_read
raise IncompleteRead(''.join(s), amt)
IncompleteRead: IncompleteRead(0 bytes read, 13631328 more expected)
-------------------- >> begin captured logging << --------------------
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /test-client.1-cyd0nuzxbebjuzs-212/
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: mira016.front.sepia.ceph.com:7280
boto: DEBUG: Port: 7280
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: StringToSign:
PUT
Fri, 04 Apr 2014 03:28:27 GMT
/test-client.1-cyd0nuzxbebjuzs-212/
boto: DEBUG: Signature:
AWS ZAWDFJOIPWAVYDIUBBRC:U5qm3MAJpIhCvh0DHbsL/ZpCOmQ=
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploads
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploads
boto: DEBUG: Method: POST
boto: DEBUG: Path: /test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploads
boto: DEBUG: Data:
boto: DEBUG: Headers: {}
boto: DEBUG: Host: mira016.front.sepia.ceph.com:7280
boto: DEBUG: Port: 7280
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: StringToSign:
POST
Fri, 04 Apr 2014 03:28:27 GMT
/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploads
boto: DEBUG: Signature:
AWS ZAWDFJOIPWAVYDIUBBRC:NmOjxKtnWBXPqsXkD0IXLCToHIk=
boto: DEBUG: <?xml version="1.0" encoding="UTF-8"?><InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>test-client.1-cyd0nuzxbebjuzs-212</Bucket><Key>mymultipart</Key><UploadId>2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD</UploadId></InitiateMultipartUploadResult>
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=1
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=1
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=1
boto: DEBUG: Data:
boto: DEBUG: Headers: {'Content-Length': '5242880', 'Content-MD5': 'nsmJ9+x6kTPiAyMMU5b0gA==', 'Content-Type': 'application/octet-stream', 'Expect': '100-Continue', 'User-Agent': 'Boto/2.27.0 Python/2.7.3 Linux/3.8.0-36-generic'}
boto: DEBUG: Host: mira016.front.sepia.ceph.com:7280
boto: DEBUG: Port: 7280
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: StringToSign:
PUT
nsmJ9+x6kTPiAyMMU5b0gA==
application/octet-stream
Fri, 04 Apr 2014 03:28:27 GMT
/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?partNumber=1&uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD
boto: DEBUG: Signature:
AWS ZAWDFJOIPWAVYDIUBBRC:gU5cgYU79Rjv2M0so45U5gy6PI0=
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=2
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=2
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=2
boto: DEBUG: Data:
boto: DEBUG: Headers: {'Content-Length': '5242880', 'Content-MD5': 'nsmJ9+x6kTPiAyMMU5b0gA==', 'Content-Type': 'application/octet-stream', 'Expect': '100-Continue', 'User-Agent': 'Boto/2.27.0 Python/2.7.3 Linux/3.8.0-36-generic'}
boto: DEBUG: Host: mira016.front.sepia.ceph.com:7280
boto: DEBUG: Port: 7280
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: StringToSign:
PUT
nsmJ9+x6kTPiAyMMU5b0gA==
application/octet-stream
Fri, 04 Apr 2014 03:28:27 GMT
/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?partNumber=2&uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD
boto: DEBUG: Signature:
AWS ZAWDFJOIPWAVYDIUBBRC:N1uh3Ptic61oPPPkN6boKAhzuiY=
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart
boto: DEBUG: path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=3
boto: DEBUG: auth_path=/test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=3
boto: DEBUG: Method: PUT
boto: DEBUG: Path: /test-client.1-cyd0nuzxbebjuzs-212/mymultipart?uploadId=2/fAaVaX2OFNIXLm5kjtotBnpmNqM3itD&partNumber=3
boto: DEBUG: Data:
boto: DEBUG: Headers: {'Content-Length': '5242880', 'Content-MD5': 'nsmJ9+x6kTPiAyMMU5b0gA==', 'Content-Type': 'application/octet-stream', 'Expect': '100-Continue', 'User-Agent': 'Boto/2.27.0 Python/2.7.3 Linux/3.8.0-36-generic'}
boto: DEBUG: Host: mira016.front.sepia.ceph.com:7280
boto: DEBUG: Port: 7280
boto: DEBUG: Params: {}
boto: DEBUG: Token: None
boto: DEBUG: StringToSign:
PUT
.
.
.
I remember seeing s3tests.functional.test_s3.test_multipart_upload_contents
mentioned in another redmine report, so this is possibly a duplicate of problems there.
Updated by Anonymous about 10 years ago
One more thing...
A few runs here and on dumpling-emperor-x are failing with:
2014-04-02 23:24:02.522179 osd.3 10.214.138.94:6800/4539 3 : [WRN] map e257 wrongly marked me down" in cluster log
Is this a real problem or a candidate for white-listing?
Updated by Anonymous about 10 years ago
- Status changed from New to Resolved
The dumpling-x parallel suites run pretty well now. (Still some network and disk space issues sometimes).
Actions