Project

General

Profile

Actions
Copy link

Bug #6121

closed

key error during readwrite test in upgrade suite

Added by Sage Weil over 10 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2013-08-26T08:14:45.133 INFO:teuthology.orchestra.run.out:[10.214.131.17]: duration: 131042004
2013-08-26T08:14:45.133 INFO:teuthology.orchestra.run.out:[10.214.131.17]: key: kchhbtjzuhbttllalo
2013-08-26T08:14:45.133 INFO:teuthology.orchestra.run.out:[10.214.131.17]: start: 1377529790.083818
2013-08-26T08:14:45.133 INFO:teuthology.orchestra.run.out:[10.214.131.17]: type: r
2013-08-26T08:14:45.133 INFO:teuthology.orchestra.run.out:[10.214.131.17]: worker: 4
2013-08-26T08:14:46.185 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,cvxhpsmyunluwcbq>
2013-08-26T08:14:47.290 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,daipqfhthdishslah>
2013-08-26T08:14:47.385 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,euuxwtacfhjldrkqps>
2013-08-26T08:14:47.468 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,kchhbtjzuhbttllalo>
2013-08-26T08:14:47.547 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,lrwzptrzqbdeqibr>
2013-08-26T08:14:47.598 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,mezluluby>
2013-08-26T08:14:47.706 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,qzrokewndoxd>
2013-08-26T08:14:47.818 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,xkgvphhublih>
2013-08-26T08:14:47.885 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,yvkzmumhvivpcfrbdjy>
2013-08-26T08:14:47.972 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Cleaning bucket <Bucket: rwtest> key <Key: rwtest,zsxwqjzpqzhuys>
2013-08-26T08:14:48.108 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Traceback (most recent call last):
2013-08-26T08:14:48.108 INFO:teuthology.orchestra.run.err:[10.214.131.17]:   File "/home/ubuntu/cephtest/6135/s3-tests/virtualenv/bin/s3tests-test-readwrite", line 9, in <module>
2013-08-26T08:14:48.108 INFO:teuthology.orchestra.run.err:[10.214.131.17]:     load_entry_point('s3tests==0.0.1', 'console_scripts', 's3tests-test-readwrite')()
2013-08-26T08:14:48.108 INFO:teuthology.orchestra.run.err:[10.214.131.17]:   File "/home/ubuntu/cephtest/6135/s3-tests/s3tests/readwrite.py", line 254, in main
2013-08-26T08:14:48.108 INFO:teuthology.orchestra.run.err:[10.214.131.17]:     trace=temp_dict['error']['traceback'])
2013-08-26T08:14:48.109 INFO:teuthology.orchestra.run.err:[10.214.131.17]: KeyError: 'traceback'
2013-08-26T08:14:48.109 INFO:teuthology.orchestra.run.err:[10.214.131.17]: Exception KeyError: KeyError(24270160,) in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored
2013-08-26T08:14:49.267 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/teuthology-next/teuthology/contextutil.py", line 25, in nested
    vars.append(enter())
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/teuthworker/teuthology-next/teuthology/task/s3readwrite.py", line 202, in run_tests
    stdin=conf,
  File "/home/teuthworker/teuthology-next/teuthology/orchestra/cluster.py", line 52, in run
    return [remote.run(**kwargs) for remote in remotes]
  File "/home/teuthworker/teuthology-next/teuthology/orchestra/remote.py", line 47, in run
    r = self._runner(client=self.ssh, **kwargs)
  File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 267, in run
    r.exitstatus = _check_status(r.exitstatus)
  File "/home/teuthworker/teuthology-next/teuthology/orchestra/run.py", line 263, in _check_status
    raise CommandFailedError(command=r.command, exitstatus=status, node=host)
CommandFailedError: Command failed on 10.214.131.17 with status 1: '/home/ubuntu/cephtest/6135/s3-tests/virtualenv/bin/s3tests-test-readwrite'
2013-08-26T08:14:49.299 DEBUG:teuthology.orchestra.run:Running [10.214.131.17]: '/home/ubuntu/cephtest/6135/adjust-ulimits ceph-coverage /home/ubuntu/cephtest/6135/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data'

ubuntu@teuthology:/a/teuthology-2013-08-26_01:30:04-upgrade-next-testing-basic-plana/6135

Actions
Copy link
 #1

Updated by Sage Weil over 10 years ago

  • Project changed from Ceph to rgw
Actions
Copy link
 #2

Updated by Sage Weil over 10 years ago

  • Status changed from 12 to 7
Actions
Copy link
 #3

Updated by Anonymous over 10 years ago

  • Status changed from 7 to Resolved

Fixed a code path where the traceback key was not being set.
The fix has been in for a few days and we've seen no occurrences of this issue.
Commit b4b99d3e23e53bfc1fe3cbdf69d7797f81a3be9e

Actions
Copy link

Also available in: Atom PDF