Bug #9279
closedtest_s3.test_atomic_* tests failed in rgw:multifs-dumpling-testing-basic-vps suite
0%
Description
In the run teuthology-2014-08-28_18:37:07-rgw:multifs-dumpling-testing-basic-vps
the following jobs failed: ['459233', '459237', '459241', '459261', '459265', '459269', '459273', '459277'] due to the same errors.
Failed on centos 6.5 and rhel 6.4
Logs for on in http://qa-proxy.ceph.com/teuthology/teuthology-2014-08-28_18:37:07-rgw:multifs-dumpling-testing-basic-vps/459233/
2014-08-28T19:13:35.651 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_100_continue ... ERROR 2014-08-28T19:13:37.958 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_bucket_acls_changes_persistent ... ok 2014-08-28T19:13:40.873 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_stress_bucket_acls_changes ... ok 2014-08-28T19:13:40.985 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_set_cors ... ok 2014-08-28T19:13:55.232 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_read_1mb ... ok 2014-08-28T19:14:09.651 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_read_4mb ... ok 2014-08-28T19:18:45.467 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_read_8mb ... ERROR 2014-08-28T19:37:18.098 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_write_1mb ... ERROR 2014-08-28T19:45:44.503 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_write_4mb ... ERROR 2014-08-28T19:54:51.255 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_write_8mb ... ERROR 2014-08-28T20:12:55.215 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_dual_write_1mb ... ERROR 2014-08-28T20:21:30.416 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_dual_write_4mb ... ERROR 2014-08-28T20:30:14.331 INFO:teuthology.orchestra.run.vpm174.stderr:s3tests.functional.test_s3.test_atomic_dual_write_8mb ... ERROR
snippet:
2014-08-28T20:31:18.177 INFO:teuthology.orchestra.run.vpm174.stderr:====================================================================== 2014-08-28T20:31:18.177 INFO:teuthology.orchestra.run.vpm174.stderr:ERROR: s3tests.functional.test_s3.test_100_continue 2014-08-28T20:31:18.177 INFO:teuthology.orchestra.run.vpm174.stderr:---------------------------------------------------------------------- 2014-08-28T20:31:18.179 INFO:teuthology.orchestra.run.vpm174.stderr:Traceback (most recent call last): 2014-08-28T20:31:18.179 INFO:teuthology.orchestra.run.vpm174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.6/site-packages/nose/case.py", line 197, in runTest 2014-08-28T20:31:18.179 INFO:teuthology.orchestra.run.vpm174.stderr: self.test(*self.arg) 2014-08-28T20:31:18.179 INFO:teuthology.orchestra.run.vpm174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4238, in test_100_continue 2014-08-28T20:31:18.180 INFO:teuthology.orchestra.run.vpm174.stderr: status = _simple_http_req_100_cont(s3.main.host, s3.main.port, s3.main.is_secure, 'PUT', resource) 2014-08-28T20:31:18.180 INFO:teuthology.orchestra.run.vpm174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests/functional/test_s3.py", line 4223, in _simple_http_req_100_cont 2014-08-28T20:31:18.180 INFO:teuthology.orchestra.run.vpm174.stderr: l = data.split(' ') 2014-08-28T20:31:18.181 INFO:teuthology.orchestra.run.vpm174.stderr:UnboundLocalError: local variable 'data' referenced before assignment 2014-08-28T20:31:18.181 INFO:teuthology.orchestra.run.vpm174.stderr:-------------------- >> begin captured stdout << --------------------- 2014-08-28T20:31:18.181 INFO:teuthology.orchestra.run.vpm174.stderr:got response: timed out 2014-08-28T20:31:18.181 INFO:teuthology.orchestra.run.vpm174.stderr:most likely server doesn't support 100-continue
archive_path: /var/lib/teuthworker/archive/teuthology-2014-08-28_18:37:07-rgw:multifs-dumpling-testing-basic-vps/459233 branch: dumpling description: rgw:multifs/{clusters/fixed-2.yaml distros/centos_6.5.yaml fs/btrfs.yaml tasks/rgw_s3tests.yaml} email: ceph-qa@ceph.com job_id: '459233' kernel: &id001 kdb: true sha1: 4813f47b94a56d861ddd496ea54511d650bb1e32 last_in_suite: false machine_type: vps name: teuthology-2014-08-28_18:37:07-rgw:multifs-dumpling-testing-basic-vps nuke-on-error: true os_type: centos os_version: '6.5' overrides: admin_socket: branch: dumpling ceph: conf: global: osd heartbeat grace: 100 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 20 osd op thread timeout: 60 fs: btrfs log-whitelist: - slow request sha1: c7948afa8232117c8a6e13b17774045a5e1bea6d ceph-deploy: branch: dev: dumpling conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: debug mon: 1 debug ms: 20 debug paxos: 20 osd default pool size: 2 install: ceph: sha1: c7948afa8232117c8a6e13b17774045a5e1bea6d rgw: default_idle_timeout: 1200 s3tests: branch: dumpling idle_timeout: 1200 workunit: sha1: c7948afa8232117c8a6e13b17774045a5e1bea6d owner: scheduled_teuthology@teuthology priority: 1000 roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mds.a - osd.3 - osd.4 - osd.5 - client.0 suite: rgw:multifs suite_branch: wip_test_distros_rgw2 suite_path: /var/lib/teuthworker/src/ceph-qa-suite_wip_test_distros_rgw2 targets: ubuntu@vpm135.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAtiYl0tTc8A9deM9/8bFmwGw4epSTEtl3VbCBxxbdHK2blRQvceDbin4DOeGeche8RLOhY6pyYf3E1Juq7KgVNJFe2EG+T9q5b/8J7nYahGw4Nhumo7Z0alJkoIouU/v5BjZOGHIkBem+3FtvkY5Oz6DDRKPnBlrceA7O2j8FbBI2E6uZ2Q8yv0Du5oMOsag6/rzT6O8LZOnSnY6qNeaw+uj7RnKdnQ5z27y7Ej6raBHBD/shwVV3tJKjBXrLcGlJpAmWEKBDRxEC/fEInUShu+KSAe484f2PdArOhBE0wtUtSv0R9HqmfEkWmeIww3XJLUWr2ejmalRowYXHlVoSkQ== ubuntu@vpm174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAtxy7a4FiK/dgQ2YkFpGrt3f0hvfL2b9mrW7XdaI8AHR4AeHE9De/04ghreQwrCEVLGtiqtDcX1st+r3XBz8GZUhQoJ65hC+jvHzrhysjlAH7OQwqdV5daccObRi0wZ0uzyZcAG5nKaRw9B1NZqcsn8WjQzZn+AvlYJ5gxIVL2rw9CVLszRdX6PFmlUY9OQD+kOzR9epWWFpUMWEaXfIgASUXLWEC67nCrfrLj1rhudiY27thusFFh6m5VCz2wc2nBTfyfIGnRRqgrE+SgC3EeAFuCbamLcP33hdoQ+gS40ZbR+9igf2m5R335qsk20b2mBHReHcrdrYiTk3o3pst9Q== tasks: - internal.lock_machines: - 2 - vps - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.serialize_remote_roles: null - internal.check_conflict: null - internal.check_ceph_data: null - internal.vm_setup: null - kernel: *id001 - internal.base: null - internal.archive: null - internal.coredump: null - internal.sudo: null - internal.syslog: null - internal.timer: null - chef: null - clock.check: null - install: null - ceph: null - rgw: - client.0 - s3tests: client.0: rgw_server: client.0 teuthology_branch: master tube: vps verbose: true worker_log: /var/lib/teuthworker/archive/worker_logs/worker.vps.15681
description: rgw:multifs/{clusters/fixed-2.yaml distros/centos_6.5.yaml fs/btrfs.yaml tasks/rgw_s3tests.yaml} duration: 6005.982199907303 failure_reason: 'Command failed on vpm174 with status 1: "S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a ''!fails_on_rgw''"' flavor: basic mon.a-kernel-sha1: 4813f47b94a56d861ddd496ea54511d650bb1e32 mon.b-kernel-sha1: 4813f47b94a56d861ddd496ea54511d650bb1e32 owner: scheduled_teuthology@teuthology success: false
Updated by Yuri Weinstein over 9 years ago
Could be related to 9169 or idle_timeout (?)
Updated by Yuri Weinstein over 9 years ago
Same problems on http://qa-proxy.ceph.com/teuthology/teuthology-2014-08-28_19:00:01-upgrade:firefly-firefly-testing-basic-vps/
Jobs failed: ['459329', '459331', '459332', '459335', '459337', '459338', '459341', '459343', '459344']
Updated by Sage Weil over 9 years ago
- Project changed from Ceph to rgw
- Priority changed from Normal to Urgent
Updated by Yuri Weinstein over 9 years ago
Same issues in http://pulpito.front.sepia.ceph.com/teuthology-2014-09-02_16:17:37-rgw:verify-dumpling-testing-basic-vps/ run
Jobs:['466721', '466725', '466729', '466749', '466753', '466761', '466765']