Bug #2675: osd: segfault during log trim - Ceph - Ceph

Bug #2675

osd: segfault during log trim

Added by Sage Weil about 11 years ago. Updated about 11 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Category:

OSD

Target version:

% Done:

Source:

Q/A

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

2012-06-29 12:46:15.163193 7f499fe7f700 -1 *** Caught signal (Segmentation fault) **
 in thread 7f499fe7f700

 ceph version 0.47.3-618-gd7c18c1 (commit:d7c18c137dace9b9a754608faf95b1804209768f)
 1: /tmp/cephtest/binary/usr/local/bin/ceph-osd() [0x70818a]
 2: (()+0xfcb0) [0x7f49b0dfecb0]
 3: (PG::IndexedLog::trim(ObjectStore::Transaction&, eversion_t)+0x3b) [0x621c7b]
 4: (PG::trim(ObjectStore::Transaction&, eversion_t)+0x1a1) [0x622361]
 5: (PG::append_log(std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&, eversion_t, ObjectStore::Transaction&)+0x775) [0x622b85]
 6: (ReplicatedPG::sub_op_modify(std::tr1::shared_ptr<OpRequest>)+0x710) [0x54fe70]
 7: (ReplicatedPG::do_sub_op(std::tr1::shared_ptr<OpRequest>)+0xff) [0x5640af]
 8: (PG::do_request(std::tr1::shared_ptr<OpRequest>)+0x9f) [0x61e82f]
 9: (OSD::dequeue_op(PG*)+0x238) [0x5d78a8]
 10: (ThreadPool::worker()+0x605) [0x7b6ba5]
 11: (ThreadPool::WorkThread::entry()+0xd) [0x5edd5d]
 12: (()+0x7e9a) [0x7f49b0df6e9a]
 13: (clone()+0x6d) [0x7f49af3ab4bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_07:00:02-marginal-master-testing-basic/3432$ cat config.yaml 
kernel: &id001
  branch: testing
  kdb: true
nuke-on-error: true
overrides:
  ceph:
    branch: master
    fs: btrfs
    log-whitelist:
    - slow request
roles:
- - mon.a
  - mon.c
  - osd.0
  - osd.1
  - osd.2
- - mon.b
  - mds.a
  - osd.3
  - osd.4
  - osd.5
- - client.0
targets:
  ubuntu@plana49.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsodFx35LkTvKGlOrQT7Zt/AuvpUOoNz4sM8ovIxIjn/AJliZfup2KAhO/VRUpcndtJMy0eAp/v6wMy9bP0tqTNbeZ3q+zzzLuwnoCOQ0relggohTE7lzLDVLb/MHCQlxpLifwUrwQpVdiUfJ2B5mcGyMr2Lku4TcC3BrgPKuaXLkGHZl0aaShLfr6PbTvDjqI+IDT4E4iKd65KhzLuzDqAIwuZyoifiG+5KYYsoIZmCHidTZYHdf6utnBzP9jTQHroZR5kE/qbVnoL1tT9vvXWLcG7y2PB3UMmIOJmRXlys2/My5iRSC+1Wd9nBxcwE3BgWHknGQhrwkMl0uC+vH3
  ubuntu@plana52.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9kswBp2g5ZV1Qrvlee8MvUOCNdubQFqUBr5WSsmFBODqEuiitWbhuBu2Ucz0lBMf41DpMKLeYDN0lIC94GZmGaiCN+Ak9Ia05d/uRvesT2nDgHB3Z9J/zEFlY8RVxL3xhD+hq4u8dbASlqqoMDiBP+7efZMxt4Ndnzr/yOxge3KenxyQImBUS+OV+BqnfCOHf6BqM33U1leXz2kng7ocxoE91DAMslKD/2DPRSYEhfucUJZk6IYevr/g0JVhbfvjSlZzwUEfTyVmPeqNyls/U+azhKlvQbqpb+ttc02RNydQ1YgOgHFCaqd9Vm8XjUU6vYGlkFHZ+BMJuEwA9AH/D
  ubuntu@plana55.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXY+KFAAzWoJq5vLwy6PJHxNeqz3fHCisJDAbtdrnjhhxVyUQtQLlhIPqiQHi6PADNYNUS/4um0TNmDFYxJLJU9SxqmBQ3QTM9F56YQa9F/+98o4LyPLS5TXqq+nCDbU1vhMbpu0mv2MDZ9BVZAgdT/yYgYGErIQz2MnaCAbgp0SRSZOxq0/3KgMz4W0KxkagiNglZV3RvarYASdqZheYeQYtnIyEw+Hk/ZLHoxUirBthAuCu5RvYYTDptQDuOR0tjRaMS81kapD5VZhFbetSxJ9rJ21oepmLSY+0UoIufZS4CNJ/sP2HDDc1Pw1mjJhqClScxTOP1yUnNWhW1d0sP
tasks:
- internal.lock_machines: 3
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.check_conflict: null
- kernel: *id001
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock: null
- ceph:
    conf:
      client:
        debug client: 1/20
        debug ms: 0/10
- ceph-fuse: null
- workunit:
    clients:
      all:
      - suites/dbench.sh

History

#1 Updated by Sage Weil about 11 years ago

also:

     0> 2012-06-29 12:38:03.057956 7f03c255a700 -1 *** Caught signal (Segmentation fault) **
 in thread 7f03c255a700

 ceph version 0.47.3-618-gd7c18c1 (commit:d7c18c137dace9b9a754608faf95b1804209768f)
 1: /tmp/cephtest/binary/usr/local/bin/ceph-osd() [0x70818a]
 2: (()+0xfcb0) [0x7f03d34d9cb0]
 3: (PG::IndexedLog::trim(ObjectStore::Transaction&, eversion_t)+0x3b) [0x621c7b]
 4: (PG::trim(ObjectStore::Transaction&, eversion_t)+0x1a1) [0x622361]
 5: (PG::append_log(std::vector<pg_log_entry_t, std::allocator<pg_log_entry_t> >&, eversion_t, ObjectStore::Transaction&)+0x775) [0x622b85]
 6: (ReplicatedPG::sub_op_modify(std::tr1::shared_ptr<OpRequest>)+0x710) [0x54fe70]
 7: (ReplicatedPG::do_sub_op(std::tr1::shared_ptr<OpRequest>)+0xff) [0x5640af]
 8: (PG::do_request(std::tr1::shared_ptr<OpRequest>)+0x9f) [0x61e82f]
 9: (OSD::dequeue_op(PG*)+0x238) [0x5d78a8]
 10: (ThreadPool::worker()+0x605) [0x7b6ba5]
 11: (ThreadPool::WorkThread::entry()+0xd) [0x5edd5d]
 12: (()+0x7e9a) [0x7f03d34d1e9a]
 13: (clone()+0x6d) [0x7f03d1a864bd]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- end dump of recent events ---

by

ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_07:00:02-marginal-master-testing-basic/3431$ cat config.yaml 
kernel: &id001
  branch: testing
  kdb: true
nuke-on-error: true
overrides:
  ceph:
    branch: master
    fs: btrfs
    log-whitelist:
    - slow request
roles:
- - mon.a
  - mon.c
  - osd.0
  - osd.1
  - osd.2
- - mon.b
  - mds.a
  - osd.3
  - osd.4
  - osd.5
- - client.0
targets:
  ubuntu@plana13.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBCv+O3jMwlV+Cu2/+fryD4y6zHcLSJYHD47MVt+7oCMuYLDRHIeezoege4H10/DVy8KjEjMrft8KtFKPMHbfvVvgVYeb8qEF5w7GfFxMz1ox2ThT1heEPtrpBqdF9p2lb6aS+S2tC5noTyb0qoVci6nUK3A3cl+LTh+n+skHviMlJok3tyqz6Ye/j011i4pfiPNbuwR7WKUuQs8hhoDy0pztzbhTZE+KZ42LKM34t9hB1NFT1uVRvfiCMaq4e+SdcwuZRlqws+LG/KpJ/5wsmBhjhBMbDXzks8PDBpnmiKJ5cPPYDmJ4QtAUEdjb6B8awli/R3EGYl0D8RolY9sR1
  ubuntu@plana50.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVJ+lkgUdkr27WFzrmwSQU22m+pFIiqzhfcO4Hinu8A8uyP4FIephrEcq4Rrt4hp14Syb1pxXisV6UKwAZKikDoD1Wl0LSro4TzOs6HuMEhfvzdnISvyzE3f2w0cj1zE61rHFYfPNF14b9fkE3wBf2Vb4i6ReaN2/Yd12J/xO52tJH1lPxgsFoAIRMjdQMbfVwPU6kK9SY4ngt9iLjge6gZ0O9Jwe2vrgD6+LNoMY9qvNjgRvQdCTi85OQwitU0ZMZdGC0cQ/oNbKd+yW92rW9Wu6dcyKSisesRcm7lbtS6X2uUup+u3vWze7coT+Py3TdNW6nGpIg4muyvqHfSinz
  ubuntu@plana51.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLTsxC+nR+xtTXMbOtazCh7MOzgBKjX/oCLMP16k0AtH8Ui92tlqsfNxcHczUol0DNzxCITgrhF8FTvgM3EgkbOUVAxGj+xLqfxsdlf58nTVXbm/pOGYnvOI8CvA4DgISHDbkzuFH4FKtR8qNTTFVmtEXaZ+jpSvn7vrYuI/Uu9XZOQh73phYW8zvVB1x8770czM0Gy2wgxdNguKy6L/Q9ShsLcFfm8Uvxf6aXb3qmuxwGhqYsMlNl0X3AjoOwmow74rodlcMvQP/pAQdjMZfe1lBPqsjmU518BE5eo7zV3O9iF6ahOrm8igOu9bfki0G52R22pA3hE9BPKPfzA0hL
tasks:
- internal.lock_machines: 3
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.check_conflict: null
- kernel: *id001
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock: null
- ceph: null
- kclient: null
- workunit:
    clients:
      all:
      - suites/ffsb.sh

#2 Updated by Sage Weil about 11 years ago

and ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_19:00:12-regression-master-testing-gcov/3437

#3 Updated by Sage Weil about 11 years ago

and ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_19:00:12-regression-master-testing-gcov/3435

#4 Updated by Tamilarasi muthamizhan about 11 years ago

and ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_19:00:12-regression-master-testing-gcov/3441

and ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_19:00:12-regression-master-testing-gcov/3443

#5 Updated by Tamilarasi muthamizhan about 11 years ago

and ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2012-06-28_19:00:12-regression-master-testing-gcov/3450

#6 Updated by Sage Weil about 11 years ago

Status changed from 12 to Resolved

Also available in: Atom PDF

Project

General

Profile

Ceph

Issues

Tags

Custom queries