Bug #19367: test.functional.tests.* failed in rados/upgrade - Ceph - Ceph

Actions

Copy link

Bug #19367

closed

test.functional.tests.* failed in rados/upgrade

Added by Yuri Weinstein about 7 years ago. Updated almost 7 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Yehuda Sadeh

Category:

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

rados

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Run: http://pulpito.ceph.com/yuriw-2017-03-22_21:32:35-rados-wip-yuri-testing_2017_3_23-distro-basic-smithi/
Job: 933002
Logs: /a/yuriw-2017-03-22_21:32:35-rados-wip-yuri-testing_2017_3_23-distro-basic-smithi/933002/teuthology.log

Sage ran w/debug logs http://pulpito.ceph.com/sage-2017-03-23_11:56:22-rados:upgrade:jewel-x-singleton-wip-osdmap-remap-2---basic-smithi/937435/

2017-03-23T13:34:47.779 INFO:teuthology.orchestra.run.smithi063.stdout:
2017-03-23T13:34:47.779 INFO:teuthology.orchestra.run.smithi063.stdout:Installed /home/ubuntu/cephtest/swift
2017-03-23T13:34:47.781 INFO:teuthology.orchestra.run.smithi063.stdout:Processing dependencies for swift==1.4.4
2017-03-23T13:34:47.781 INFO:teuthology.orchestra.run.smithi063.stdout:Finished processing dependencies for swift==1.4.4
2017-03-23T13:34:47.789 INFO:teuthology.orchestra.run.smithi063:Running: 'python -c \'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.a
rgv[1], "wb"))\' /home/ubuntu/cephtest/archive/testswift.client.0.conf'
2017-03-23T13:34:47.836 INFO:teuthology.orchestra.run.smithi063:Running: "SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.
conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw'" 
2017-03-23T13:34:49.755 INFO:teuthology.orchestra.run.smithi063.stderr:testAccountHead (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:testContainerListing (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.766 INFO:teuthology.orchestra.run.smithi063.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.770 INFO:teuthology.orchestra.run.smithi063.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.774 INFO:teuthology.orchestra.run.smithi063.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.777 INFO:teuthology.orchestra.run.smithi063.stderr:testInvalidUTF8Path (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.781 INFO:teuthology.orchestra.run.smithi063.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.785 INFO:teuthology.orchestra.run.smithi063.stderr:testListingLimit (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ERROR
2017-03-23T13:34:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:testPUT (test.functional.tests.TestAccount) ... ERROR
........................

Actions

Copy link

Updated by Yehuda Sadeh about 7 years ago

Assignee set to Yehuda Sadeh

osds are upgraded, radosgw still on jewel. Problem with listing bucket, as cls_user_bucket structure is not backward compatible. See: https://github.com/yehudasa/ceph/commit/8a8d9c153eae2ee050b1033fcf3dab5626465f62

Actions

Copy link

Updated by Yuri Weinstein about 7 years ago

testing http://pulpito.ceph.com/yuriw-2017-03-23_23:18:20-rados-wip-19367---basic-smithi/

Note to self to enable rgw debug

Rerun with rgw debug log:

  ceph:
    conf:
      mon:
        debug mon: 20
        debug ms: 1
        debug paxos: 20
        mon warn on legacy crush tunables: false
        mon warn on osd down out interval zero: false
      osd:
        debug filestore: 20
        debug journal: 20
        debug ms: 1
        debug osd: 25
      client:
        debug rgw: 20
        debug ms: 1

Logs are in /a/19367

Actions

Copy link