Project

General

Profile

Bug #9416

ods crash in upgrade:dumpling-dumpling-distro-basic-vps run

Added by Yuri Weinstein over 9 years ago. Updated over 9 years ago.

Status:
Duplicate
Priority:
High
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-09-09_18:30:01-upgrade:dumpling-dumpling-distro-basic-vps/475538/

Coredump in */475538/remote/vpm176/log$ceph-osd.1.log.gz:

ceph-osd.1.log.gz:     0> 2014-09-10 03:15:26.214087 7f3dfca45700 -1 *** Caught signal (Aborted) **
ceph-osd.1.log.gz: in thread 7f3dfca45700
ceph-osd.1.log.gz:
ceph-osd.1.log.gz: ceph version 0.67.10-10-gf7498c4 (f7498c47ed90db71e50a3d4ff9ce13626e863719)
ceph-osd.1.log.gz: 1: ceph-osd() [0x7fd46a]
ceph-osd.1.log.gz: 2: (()+0xfcb0) [0x7f3e001f0cb0]
ceph-osd.1.log.gz: 3: (gsignal()+0x35) [0x7f3dfe41a0d5]
ceph-osd.1.log.gz: 4: (abort()+0x17b) [0x7f3dfe41d83b]
ceph-osd.1.log.gz: 5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7f3dfed6c69d]
ceph-osd.1.log.gz: 6: (()+0xb5846) [0x7f3dfed6a846]
ceph-osd.1.log.gz: 7: (()+0xb5873) [0x7f3dfed6a873]
ceph-osd.1.log.gz: 8: (()+0xb596e) [0x7f3dfed6a96e]
ceph-osd.1.log.gz: 9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1df) [0x8c696f]
ceph-osd.1.log.gz: 10: (ceph::HeartbeatMap::_check(ceph::heartbeat_handle_d*, char const*, long)+0x107) [0x816fd7]
ceph-osd.1.log.gz: 11: (ceph::HeartbeatMap::is_healthy()+0xa7) [0x817987]
ceph-osd.1.log.gz: 12: (ceph::HeartbeatMap::check_touch_file()+0x23) [0x817f33]
ceph-osd.1.log.gz: 13: (CephContextServiceThread::entry()+0x55) [0x8d4225]
ceph-osd.1.log.gz: 14: (()+0x7e9a) [0x7f3e001e8e9a]
ceph-osd.1.log.gz: 15: (clone()+0x6d) [0x7f3dfe4d831d]
archive_path: /var/lib/teuthworker/archive/teuthology-2014-09-09_18:30:01-upgrade:dumpling-dumpling-distro-basic-vps/475538
branch: dumpling
description: upgrade:dumpling/fs/{0-cluster/start.yaml 1-dumpling-install/v0.67.10.yaml
  2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-osd-mon-mds.yaml 4-final/osdthrash.yaml}
email: ceph-qa@ceph.com
job_id: '475538'
kernel: &id001
  kdb: true
  sha1: distro
last_in_suite: false
machine_type: vps
name: teuthology-2014-09-09_18:30:01-upgrade:dumpling-dumpling-distro-basic-vps
nuke-on-error: true
os_type: ubuntu
overrides:
  admin_socket:
    branch: dumpling
  ceph:
    conf:
      global:
        osd heartbeat grace: 100
      mon:
        debug mon: 20
        debug ms: 1
        debug paxos: 20
      osd:
        debug filestore: 20
        debug journal: 20
        debug ms: 1
        debug osd: 20
    fs: xfs
    log-whitelist:
    - slow request
    - scrub
    - wrongly marked me down
    - objects unfound and apparently lost
    - log bound mismatch
    sha1: f7498c47ed90db71e50a3d4ff9ce13626e863719
  ceph-deploy:
    branch:
      dev: dumpling
    conf:
      client:
        log file: /var/log/ceph/ceph-$name.$pid.log
      mon:
        debug mon: 1
        debug ms: 20
        debug paxos: 20
        osd default pool size: 2
  install:
    ceph:
      sha1: f7498c47ed90db71e50a3d4ff9ce13626e863719
  rgw:
    default_idle_timeout: 1200
  s3tests:
    branch: dumpling
    idle_timeout: 1200
  workunit:
    sha1: f7498c47ed90db71e50a3d4ff9ce13626e863719
owner: scheduled_teuthology@teuthology
priority: 1000
roles:
- - mon.a
  - mds.a
  - osd.0
  - osd.1
  - osd.2
- - mon.b
  - mon.c
  - osd.3
  - osd.4
  - osd.5
  - client.0
suite: upgrade:dumpling
suite_branch: dumpling
suite_path: /var/lib/teuthworker/src/ceph-qa-suite_dumpling
targets:
  ubuntu@vpm093.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYWvGDv4pIaJ0MOszI+luN3xmX7eSP/S3gh0Sbnf8JSbyke94FnpT/xjAIhkSffxAjdn90APu43GXxu9xUF/jVH/zo+sdSwCxl3ibuzWLVdlqB3xaglm55h+qqYe0ODjbEXYmwqGkeVtW4l7Lv1aPQs0/1gTejyUfW/Eti0wjAevYn6kYFUMMjSWZU4iRNUAiU2286vk8MbQ1SRTrdzM3+D/iODIO+a0fpOuEX4Oq77nVh5Btna9iJD6XF+RAxLCo+bdgN45IV4z6Xfua3Gk39IRTXySO9tF6PuV4QPpGd59nM48l99aDpZ+BfJji00LMqHFEwxarGGLRZuo7b/eDj
  ubuntu@vpm176.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0YLAuHqewh4e1OxhigjJBCAs1COkR8nQKFiPWW88okSQxoXkbqvs70rVaFd3rJAFwCjhwNhsaK18Yn4ZULV5X0bRdbRskzx6EMwa5WuXWubzD7OQ9HQLM59dZn53GSQc4chbua0c8qG4fqDGe09sla9xY33gbcHUJHl51rBnrR7R7jJMOmO2nED4cIR9+gkr24BKJyd0PcHnsL4m9ehvcAWDrJ7ml4b/1cqtkFMqg8cY3f8Bo+DrPazjKLKbXi5/nKoxSRbjoukn67U05d/aJ76CwJcZ7xGinKhnP+N9ZRcc9xFG8aOO/Wm6Uk/b3/H7ffPxzllXx+UD67dnxvIuT
tasks:
- internal.lock_machines:
  - 2
  - vps
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.serialize_remote_roles: null
- internal.check_conflict: null
- internal.check_ceph_data: null
- internal.vm_setup: null
- kernel: *id001
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.sudo: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock.check: null
- install:
    tag: v0.67.10
- ceph: null
- install.upgrade:
    all:
      branch: dumpling
- parallel:
  - workload
  - upgrade-sequence
- thrashosds:
    chance_pgnum_grow: 1
    chance_pgpnum_fix: 1
    thrash_primary_affinity: false
    timeout: 1200
- ceph-fuse: null
- workunit:
    clients:
      all:
      - suites/iogen.sh
teuthology_branch: master
tube: vps
upgrade-sequence:
  sequential:
  - ceph.restart:
    - osd.0
  - sleep:
      duration: 30
  - ceph.restart:
    - osd.1
  - sleep:
      duration: 30
  - ceph.restart:
    - osd.2
  - sleep:
      duration: 30
  - ceph.restart:
    - osd.3
  - sleep:
      duration: 30
  - ceph.restart:
    - osd.4
  - sleep:
      duration: 30
  - ceph.restart:
    - osd.5
  - sleep:
      duration: 60
  - ceph.restart:
    - mon.a
  - sleep:
      duration: 60
  - ceph.restart:
    - mon.b
  - sleep:
      duration: 60
  - ceph.restart:
    - mon.c
  - sleep:
      duration: 60
  - ceph.restart:
    - mds.a
verbose: true
worker_log: /var/lib/teuthworker/archive/worker_logs/worker.vps.16799
workload:
  workunit:
    clients:
      all:
      - suites/blogbench.sh
description: upgrade:dumpling/fs/{0-cluster/start.yaml 1-dumpling-install/v0.67.10.yaml
  2-workload/blogbench.yaml 3-upgrade-sequence/upgrade-osd-mon-mds.yaml 4-final/osdthrash.yaml}
duration: 2015.0952730178833
failure_reason: timed out waiting for admin_socket to appear after osd.1 restart
flavor: basic
owner: scheduled_teuthology@teuthology
success: false

Related issues

Duplicates Ceph - Bug #6756: journal full hang on startup Resolved 11/12/2013

History

#1 Updated by Ian Colle over 9 years ago

  • Project changed from devops to Ceph

#2 Updated by Tamilarasi muthamizhan over 9 years ago

  • Priority changed from Normal to High

#3 Updated by Samuel Just over 9 years ago

  • Assignee set to Samuel Just

#4 Updated by Samuel Just over 9 years ago

  • Status changed from New to Duplicate

Also available in: Atom PDF