Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run - rgw - Ceph

Actions

Copy link

Bug #10354

closed

Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run

Added by Yuri Weinstein over 9 years ago. Updated about 9 years ago.

Status:

Duplicate

Priority:

Urgent

Assignee:

Yehuda Sadeh

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-12-16_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/664018/

2014-12-16T20:02:25.990 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_list_buckets_invalid_auth ... FAIL
2014-12-16T20:02:25.995 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_list_buckets_bad_auth ... FAIL

2014-12-16T20:02:11.838 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifmatch_good ... ok
2014-12-16T20:02:12.018 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifmatch_failed ... FAIL
2014-12-16T20:02:12.263 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok
2014-12-16T20:02:12.341 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... FAIL
2014-12-16T20:02:12.471 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifnonmatch_good ... ok
2014-12-16T20:02:12.591 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifnonmatch_failed ... FAIL
2014-12-16T20:02:12.689 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok
2014-12-16T20:02:12.859 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... FAIL

.....

2014-12-16T20:02:56.142 INFO:teuthology.orchestra.run.plana22.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... FAIL

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Same issue - http://pulpito.ceph.com/teuthology-2014-12-17_17:15:01-upgrade:dumpling-firefly-x:parallel-next-distro-basic-vps/

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Priority changed from Normal to Urgent

Same issue http://pulpito.ceph.com/teuthology-2014-12-17_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Same in Octo - 5 jobs in run http://pulpito.ceph.redhat.com/teuthology-2014-12-16_22:04:02-rgw-dumpling-distro-basic-magna/

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Still failed in run http://pulpito.ceph.com/teuthology-2014-12-21_17:10:01-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Failed in run http://pulpito.ceph.com/teuthology-2014-12-22_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/

Jobs ['672762', '672763']

2014-12-23T00:39:13.918 INFO:teuthology.orchestra.run.plana72.stderr:FAIL: s3tests.functional.test_s3.test_list_buckets_invalid_auth

2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:======================================================================
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:FAIL: s3tests.functional.test_s3.test_list_buckets_bad_auth
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:----------------------------------------------------------------------
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:Traceback (most recent call last):
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:  File "/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:    self.test(*self.arg)
2014-12-23T00:39:13.926 INFO:teuthology.orchestra.run.plana72.stderr:  File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4089, in test_list_buckets_bad_auth
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:    eq(e.error_code, 'SignatureDoesNotMatch')
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:AssertionError: u'AccessDenied' != 'SignatureDoesNotMatch'
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:-------------------- >> begin captured logging << --------------------
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Using access key provided by client.
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Using secret key provided by client.
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: path=/
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: auth_path=/
2014-12-23T00:39:13.927 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Method: GET
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Path: /
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Data:
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Headers: {}
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Host: plana72.front.sepia.ceph.com:7280
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Port: 7280
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Params: {}
2014-12-23T00:39:13.928 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: establishing HTTP connection: kwargs={'port': 7280, 'timeout': 30}
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Token: None
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: StringToSign:
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:GET
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:Tue, 23 Dec 2014 06:54:05 GMT
2014-12-23T00:39:13.929 INFO:teuthology.orchestra.run.plana72.stderr:/
2014-12-23T00:39:13.930 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Signature:
2014-12-23T00:39:13.930 INFO:teuthology.orchestra.run.plana72.stderr:AWS PNOKHIFTTSFFUMPEZWLJ:msb1DDuG2mXmqjzRTPDi47giKZI=
2014-12-23T00:39:13.930 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Final headers: {'Date': 'Tue, 23 Dec 2014 06:54:05 GMT', 'Content-Length': 0, 'Authorization': u'AWS PNOKHIFTTSFFUMPEZWLJ:msb1DDuG2mXmqjzRTPDi47giKZI=', 'User-Agent': 'Boto/2.34.0 Python/2.7.3 Linux/3.13.0-27-generic'}
2014-12-23T00:39:13.930 INFO:teuthology.orchestra.run.plana72.stderr:boto: DEBUG: Response headers: [('date', 'Tue, 23 Dec 2014 06:54:05 GMT'), ('accept-ranges', 'bytes'), ('content-type', 'application/xml'), ('content-length', '78'), ('server', 'Apache/2.2.22 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141')]
2014-12-23T00:39:13.930 INFO:teuthology.orchestra.run.plana72.stderr:--------------------- >> end captured logging << ---------------------

Actions

Copy link

Updated by Yehuda Sadeh over 9 years ago

It's running the master branch of the s3 tests instead of the next branch.

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

This suite did not change AFAIK and was passing as is
http://pulpito.front.sepia.ceph.com/teuthology-2014-12-03_17:10:01-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/

We don't have branch specified here https://github.com/ceph/ceph-qa-suite/blob/next/suites/upgrade/dumpling-firefly-x/parallel/7-final-workload/rgw_s3tests.yaml , so assuming it should run on next

What are you suggesting?

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

We were running teuthology pointing to an old path, thus running something old, Zack modified the path and will see how it runs next time.

Testing now - http://pulpito.front.sepia.ceph.com/teuthology-2014-12-23_10:24:16-upgrade:dumpling-firefly-x:parallel-giant-distro-basic-vps/

Actions

Copy link

Updated by Yuri Weinstein over 9 years ago

Status changed from New to Resolved

All passed

Actions

Copy link

#10

Updated by Yuri Weinstein over 9 years ago

Status changed from Resolved to New

Reopening as the initial problem is still there:

Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-12-25_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi/680229/

Running on next branch
2014-12-26T07:16:30.944 INFO:tasks.s3tests:Using branch 'next' for s3tests

2014-12-26T08:39:07.254 INFO:teuthology.orchestra.run.plana51.stderr:s3tests.functional.test_s3.test_list_buckets_invalid_auth ... FAIL
2014-12-26T08:39:07.259 INFO:teuthology.orchestra.run.plana51.stderr:s3tests.functional.test_s3.test_list_buckets_bad_auth ... FAIL

Actions

Copy link

#11

Updated by Yehuda Sadeh over 9 years ago

Status changed from New to Fix Under Review
Assignee set to Josh Durgin

This is duplicate of 10329, there's a pull request pending. Note though that the pull request is against master, but at this point we need to merge it into next.

Actions

Copy link

#12