Project

General

Profile

Actions

Bug #23290

open

"/test/osd/RadosModel.h: 854: FAILED assert(0)" in upgrade:hammer-x-jewel-distro-basic-smithi

Added by Yuri Weinstein about 6 years ago. Updated about 6 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
upgrade/hammer-x
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/teuthology-2018-03-09_01:15:03-upgrade:hammer-x-jewel-distro-basic-smithi/
Jobs: '2267116', '2267120'
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2018-03-09_01:15:03-upgrade:hammer-x-jewel-distro-basic-smithi/2267116/teuthology.log

2018-03-09T03:49:14.975 INFO:tasks.ceph.mon.c.smithi060.stderr:2018-03-09 03:49:14.973796 7f5e12649700 -1 mon.c@2(peon).osd e588 update_from_paxos full map CRC mismatch, resetting to canonical
2018-03-09T03:49:14.992 INFO:tasks.rados.rados.0.smithi098.stdout:291:  finishing write tid 1 to smithi09811607-13 0123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299
2018-03-09T03:49:14.993 INFO:tasks.rados.rados.0.smithi098.stderr:Error: finished tid 1 when last_acked_tid was 6
2018-03-09T03:49:15.005 INFO:tasks.thrashosds.thrasher:in_osds:  [1, 2, 3, 0, 5] out_osds:  [4] dead_osds:  [3] live_osds:  [0, 2, 4, 1, 5]
2018-03-09T03:49:15.005 INFO:tasks.thrashosds.thrasher:choose_action: min_in 3 min_out 0 min_live 2 min_dead 0
2018-03-09T03:49:15.005 INFO:tasks.thrashosds.thrasher:inject_pause on 4
2018-03-09T03:49:15.005 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3
2018-03-09T03:49:15.006 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False
2018-03-09T03:49:15.006 INFO:teuthology.orchestra.run.smithi143:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3'
2018-03-09T03:49:15.009 INFO:tasks.rados.rados.0.smithi098.stderr:./test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(TestOp::CallbackInfo*)' thread 7f8f03fff700 time 2018-03-09 03:49:14.992791
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr:./test/osd/RadosModel.h: 854: FAILED assert(0)
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: ceph version 0.94.10-85-ga8e54ce (a8e54cee69fc2fdc8df27f35ebe1b56444f43317)
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x4e147b]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x46e) [0x4c8bfe]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 3: (write_callback(void*, void*)+0x19) [0x4d8a79]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 4: (()+0x9526d) [0x7f8f1d11c26d]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 5: (()+0x704c9) [0x7f8f1d0f74c9]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 6: (()+0x1351e8) [0x7f8f1d1bc1e8]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 7: (()+0x8184) [0x7f8f1cc5b184]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: 8: (clone()+0x6d) [0x7f8f1b5dc03d]
2018-03-09T03:49:15.010 INFO:tasks.rados.rados.0.smithi098.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

Related issues 1 (0 open1 closed)

Has duplicate RADOS - Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade testDuplicate04/09/2018

Actions
Actions #1

Updated by Greg Farnum about 6 years ago

  • Project changed from Ceph to RADOS

Is that the "the disk errored out" bug?

Actions #2

Updated by Greg Farnum about 6 years ago

  • Priority changed from Urgent to Normal
Actions #3

Updated by Kefu Chai almost 6 years ago

  • Related to Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade test added
Actions #4

Updated by Kefu Chai almost 6 years ago

  • Related to deleted (Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade test)
Actions #5

Updated by Kefu Chai almost 6 years ago

  • Has duplicate Bug #23598: hammer->jewel: ceph_test_rados crashes during radosbench task in jewel rados upgrade test added
Actions

Also available in: Atom PDF