Tools » teuthology

While working on #7933 I wanted to make sure the work I was doing was consistent with other rgw task work, so I looked at some tests that had been run recently. In this one I see 20 exceptions with tracebacks.

The test passed.

Here is one of the exceptions at the beginning

2014-04-14T01:45:41.291 DEBUG:teuthology.orchestra.run:Running [10.214.133.29]: '/home/ubuntu/cephtest/s3-tests/virtualenv/bin/s3tests-test-roundtrip'
2014-04-14T01:45:41.574 INFO:teuthology.orchestra.run.err:[10.214.133.29]: Using random seeds: {'writer': 846882397, 'main': 1883575116, 'names': 3920074108, 'contents': 839267835, 'reader': 2082859414}
2014-04-14T01:45:44.338 INFO:teuthology.orchestra.run.err:[10.214.133.29]: Created bucket: rttest
2014-04-14T01:45:44.339 INFO:teuthology.orchestra.run.err:[10.214.133.29]: Writing 10 objects with 3 workers...
2014-04-14T01:45:44.343 INFO:teuthology.orchestra.run.out:[10.214.133.29]: bucket: rttest
2014-04-14T01:45:44.344 INFO:teuthology.orchestra.run.out:[10.214.133.29]: chunks: null
2014-04-14T01:45:44.344 INFO:teuthology.orchestra.run.out:[10.214.133.29]: duration: 15020
2014-04-14T01:45:44.344 INFO:teuthology.orchestra.run.out:[10.214.133.29]: error: {msg: '', traceback: "Traceback (most recent call last):\n  File \"/home/ubuntu/cephtest/s3-tests/s3tests/roundtrip.py\"\
2014-04-14T01:45:44.344 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 30, in writer\n    key.set_contents_from_file(fp)\n  File \"/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.345 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 1178, in set_contents_from_file\n    fp.seek(0, os.SEEK_END)\n  File \"\
2014-04-14T01:45:44.345 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     /home/ubuntu/cephtest/s3-tests/s3tests/realistic.py\", line 39, in seek\n    assert\
2014-04-14T01:45:44.345 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     \ self.offset == 0\nAssertionError\n"}
2014-04-14T01:45:44.345 INFO:teuthology.orchestra.run.out:[10.214.133.29]: key: swinptxwygyeddzutbjfliv
2014-04-14T01:45:44.346 INFO:teuthology.orchestra.run.out:[10.214.133.29]: start: 1397465144.338893
2014-04-14T01:45:44.346 INFO:teuthology.orchestra.run.out:[10.214.133.29]: type: w

Toward the end:

2014-04-14T01:45:44.471 INFO:teuthology.orchestra.run.out:[10.214.133.29]: bucket: rttest
2014-04-14T01:45:44.471 INFO:teuthology.orchestra.run.out:[10.214.133.29]: chunks: []
2014-04-14T01:45:44.471 INFO:teuthology.orchestra.run.out:[10.214.133.29]: duration: 45953035
2014-04-14T01:45:44.471 INFO:teuthology.orchestra.run.out:[10.214.133.29]: error: {msg: 'S3ResponseError: 404 Not Found
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]: 
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     <?xml version="1.0" encoding="UTF-8"?><Error><Code>NoSuchKey</Code></Error>',
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:   traceback: "Traceback (most recent call last):\n  File \"/home/ubuntu/cephtest/s3-tests/s3tests/roundtrip.py\"\
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 69, in reader\n    key.get_contents_to_file(fp)\n  File \"/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 1603, in get_contents_to_file\n    response_headers=response_headers)\n\
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     \  File \"/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 1435, in get_file\n    query_args=None)\n  File \"/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.472 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 1467, in _get_file_internal\n    override_num_retries=override_num_retries)\n\
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     \  File \"/home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 325, in open\n    override_num_retries=override_num_retries)\n  File \"\
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     /home/ubuntu/cephtest/s3-tests/virtualenv/local/lib/python2.7/site-packages/boto/s3/key.py\"\
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     , line 273, in open_read\n    self.resp.reason, body)\nS3ResponseError: S3ResponseError:\
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]:     \ 404 Not Found\n<?xml version=\"1.0\" encoding=\"UTF-8\"?><Error><Code>NoSuchKey</Code></Error>\n"}
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]: key: lhlmqtohuysjjv
2014-04-14T01:45:44.473 INFO:teuthology.orchestra.run.out:[10.214.133.29]: start: 1397465144.377423
2014-04-14T01:45:44.474 INFO:teuthology.orchestra.run.out:[10.214.133.29]: type: r

And finally:

2014-04-14T01:45:44.560 INFO:teuthology.orchestra.run.err:[10.214.133.29]: Exception KeyError: KeyError(43202032,) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored

http://pulpito.front.sepia.ceph.com/teuthology-2014-04-13_23:00:17-rgw-master-testing-basic-plana/189737/
http://qa-proxy.ceph.com/teuthology/teuthology-2014-04-13_23:00:17-rgw-master-testing-basic-plana/189737/teuthology.log