Actions
Bug #9771
closedSegmentation fault after upgrade v0.80.5 -> v0.80.6
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
This is new test upgrades from v0.80.4 > v0.80.5 -> v0.80.4>firefly and runs different workloads after each step.
2014-10-14T08:41:01.196 INFO:tasks.workunit.client.0.vpm053.stderr:+ rbd import --new-format --order 20 /tmp/sparse1 2014-10-14T08:41:01.247 INFO:tasks.workunit.client.0.vpm053.stderr:*** Caught signal (Segmentation fault) ** 2014-10-14T08:41:01.248 INFO:tasks.workunit.client.0.vpm053.stderr: in thread 7f630c1df700 2014-10-14T08:41:01.392 INFO:tasks.workunit.client.0.vpm053.stderr:Segmentation fault (core dumped) 2014-10-14T08:41:01.456 INFO:tasks.workunit:Stopping rbd/import_export.sh on client.0...
archive_path: /var/lib/teuthworker/archive/teuthology-2014-10-14_08:06:20-upgrade:firefly:singleton-firefly-distro-basic-vps/548125 branch: firefly description: upgrade:firefly:singleton/{all/versions-steps.yaml distros/ubuntu_12.04.yaml} email: yuri.weinstein@inktank.com job_id: '548125' kernel: &id001 kdb: true sha1: distro last_in_suite: false machine_type: vps name: teuthology-2014-10-14_08:06:20-upgrade:firefly:singleton-firefly-distro-basic-vps nuke-on-error: true os_type: ubuntu os_version: '12.04' overrides: admin_socket: branch: firefly ceph: conf: mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 20 fs: xfs log-whitelist: - slow request - scrub sha1: a6544a3137aeb5066d57a6fd54208438a8cd66b8 ceph-deploy: branch: dev: firefly conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: debug mon: 1 debug ms: 20 debug paxos: 20 osd default pool size: 2 install: ceph: sha1: a6544a3137aeb5066d57a6fd54208438a8cd66b8 s3tests: branch: firefly workunit: sha1: a6544a3137aeb5066d57a6fd54208438a8cd66b8 owner: yuriw priority: 100 roles: - - mon.a - mds.a - osd.0 - osd.1 - osd.2 - - mon.b - mon.c - osd.3 - osd.4 - osd.5 - client.0 - client.1 suite: upgrade:firefly:singleton suite_branch: wip_9758 suite_path: /var/lib/teuthworker/src/ceph-qa-suite_wip_9758 targets: ubuntu@vpm053.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHHLy4mq1Jy8mfNxFtH2Tx0wqxNPAbDPqpxUcI3DKXerfGx58pXfIpxYyvtQRL49odDjaj7rXBSCM7yDaBO7hhyK6FLDKRcewxnv/4x2TH/XqwU1qq07V1qs4O6eX/ZLK40ScBVX/91A6BY75O5kHPLrU54GDnPfjDeR/Vn4jntiWQ4USRbR/zaGBTkxc5kwCjRkDIhFjRBzQb6RCsjh7zDK8cxSuTMMTapqlN3+GpCHkVJ9ayCDhgKI7NLWgTMfYwrOembKUIIdnShVa8PHACMydE4/M3rURis02Vbe8zMNYIjwW8midW1lHt3lRH12YkQpifw++usvRgftQ4lp0P ubuntu@vpm139.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDb60ZXRkWl6fQ9Kg0sez6ScoVWCTl6DbL+9WD0tbR4d6oGGgwLxhRTdO7ZlGJPtypgBIEIsS3kvyRAOh0Ki8AnGvZzh66kFgYszlUzo/4UuLZZN7rJOVIkAR89GADq8XKaVaF7H9rzFIF9ixoRpo4ylLiCj6d6sEzCFDb94CXCfMwfDt8+f+GcRn4zvFJpIRB9r5mTvb1SIcYMjyB2cLApWVgTFRaPN/k8FlmhpGKcvJgjffULFwbSRYxje0UqWbOU2Vu3TeaaKI5yV++m8sLwDIOuk0VwX7GCx2x6m+VaQiRwIVuyquwg7+s3olbAuNGqZLfyi1mCkV+qweqcikWN tasks: - internal.lock_machines: - 2 - vps - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.push_inventory: null - internal.serialize_remote_roles: null - internal.check_conflict: null - internal.check_ceph_data: null - internal.vm_setup: null - kernel: *id001 - internal.base: null - internal.archive: null - internal.coredump: null - internal.sudo: null - internal.syslog: null - internal.timer: null - chef: null - clock.check: null - install: tag: v0.80.4 - print: '**** done v0.80.4 install' - ceph: fs: xfs - print: '**** done ceph xfs' - parallel: - workload - print: '**** done workload' - parallel: - workload1 - upgrade-sequence1 - print: '**** done parallel 1' - parallel: - workload2 - upgrade-sequence2 - print: '**** done parallel 2' - parallel: - workload_firefly - upgrade-sequence_firefly - print: '**** done parallel firefly branch' teuthology_branch: master tube: vps upgrade-sequence1: sequential: - install.upgrade: mon.a: tag: v0.80.5 mon.b: tag: v0.80.5 - print: '**** done v0.80.5 install.upgrade' - ceph.restart: - mon.a - sleep: duration: 60 - ceph.restart: - mon.b - sleep: duration: 60 - ceph.restart: - mon.c - sleep: duration: 60 - ceph.restart: - mds.a - sleep: duration: 60 - ceph.restart: - osd.0 - sleep: duration: 30 - ceph.restart: - osd.1 - sleep: duration: 30 - ceph.restart: - osd.2 - sleep: duration: 30 - ceph.restart: - osd.3 - sleep: duration: 30 - ceph.restart: - osd.4 - sleep: duration: 30 - ceph.restart: - osd.5 - sleep: duration: 30 - print: '**** done ceph.restart all 1' upgrade-sequence2: sequential: - install.upgrade: mon.a: tag: v0.80.6 mon.b: tag: v0.80.6 - print: '**** done v0.80.6 install.upgrade' - ceph.restart: - osd.0 - sleep: duration: 30 - ceph.restart: - osd.1 - sleep: duration: 30 - ceph.restart: - osd.2 - sleep: duration: 30 - ceph.restart: - osd.3 - sleep: duration: 30 - ceph.restart: - osd.4 - sleep: duration: 30 - ceph.restart: - osd.5 - sleep: duration: 60 - ceph.restart: - mon.a - sleep: duration: 60 - ceph.restart: - mon.b - sleep: duration: 60 - ceph.restart: - mon.c - sleep: duration: 60 - ceph.restart: - mds.a - sleep: duration: 60 - print: '**** done ceph.restart all 2' upgrade-sequence_firefly: sequential: - install.upgrade: mon.a: branch: firefly mon.b: branch: firefly - print: '**** done branch: firefly install.upgrade' - ceph.restart: - osd.0 - sleep: duration: 30 - ceph.restart: - osd.1 - sleep: duration: 30 - ceph.restart: - osd.2 - sleep: duration: 30 - ceph.restart: - osd.3 - sleep: duration: 30 - ceph.restart: - osd.4 - sleep: duration: 30 - ceph.restart: - osd.5 - sleep: duration: 60 - ceph.restart: - mon.a - sleep: duration: 60 - ceph.restart: - mon.b - sleep: duration: 60 - ceph.restart: - mon.c - sleep: duration: 60 - ceph.restart: - mds.a - sleep: duration: 60 - print: '**** done ceph.restart all firefly current branch' verbose: true worker_log: /var/lib/teuthworker/archive/worker_logs/worker.vps.3000 workload: workunit: clients: client.0: - suites/blogbench.sh workload1: sequential: - workunit: clients: client.0: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - workunit: clients: client.0: - cls/test_cls_rbd.sh workload2: sequential: - workunit: clients: client.0: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - workunit: clients: client.0: - cls/test_cls_rbd.sh workload_firefly: sequential: - rgw: - client.0 - s3tests: client.0: force-branch: firefly-original rgw_server: client.0
description: upgrade:firefly:singleton/{all/versions-steps.yaml distros/ubuntu_12.04.yaml} duration: 1542.9695949554443 failure_reason: 'Command failed on vpm053 with status 139: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=a6544a3137aeb5066d57a6fd54208438a8cd66b8 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" RBD_CREATE_ARGS=--new-format adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/rbd/import_export.sh''' flavor: basic owner: yuriw success: false
Updated by Jason Dillaman over 9 years ago
- Subject changed from Segmentation fault after upgrade v0.80.4 -> v0.80.5 to Segmentation fault after upgrade v0.80.5 -> v0.80.6
Updated by Jason Dillaman over 9 years ago
Commit b75f85a2 added new elements to the Thread class, breaking ABI. In this (and several other upgrade tests from v0.80.5), if librados has been updated while librbd is still in the process of being upgraded, code within librbd that uses the Finisher class will incorrectly expect the old Thread class ABI where priority isn't present but execute new Thread class ABI where priority is added. As a result, the Finisher's this pointer is now located at the same memory address as the priority values, resulting in a crash.
Actions