Project

General

Profile

Actions

Bug #10165

closed

ceph_test_rados got short read

Added by Yuri Weinstein over 9 years ago. Updated over 9 years ago.

Status:
Duplicate
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Runs:
http://pulpito.front.sepia.ceph.com/teuthology-2014-11-20_17:05:01-upgrade:giant-x-next-distro-basic-vps/
Jobs: ['611379', '611383', '611386', '611388', '611390']
http://pulpito.front.sepia.ceph.com/teuthology-2014-11-20_17:05:01-upgrade:giant-x-next-distro-basic-multi/
Jobs: ['611356', '611358']

Logs for one are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-11-20_17:05:01-upgrade:giant-x-next-distro-basic-vps/611379/

2014-11-20T21:38:22.130 INFO:tasks.rados.rados.0.vpm200.stderr:./test/osd/RadosModel.h: 1077: FAILED assert(0)
2014-11-20T21:38:22.182 INFO:tasks.rados.rados.0.vpm200.stderr: ceph version 0.87-34-g37ffccb (37ffccbd57e7e441e0eb1499e5c173aa9c375d35)
2014-11-20T21:38:22.182 INFO:tasks.rados.rados.0.vpm200.stderr: 1: (ReadOp::_finish(TestOp::CallbackInfo*)+0x17d7) [0x422ca7]
2014-11-20T21:38:22.182 INFO:tasks.rados.rados.0.vpm200.stderr: 2: (librados::C_AioComplete::finish(int)+0x18) [0x7f34f28d5898]
2014-11-20T21:38:22.182 INFO:tasks.rados.rados.0.vpm200.stderr: 3: (Context::complete(int)+0x9) [0x7f34f28af899]
2014-11-20T21:38:22.183 INFO:tasks.rados.rados.0.vpm200.stderr: 4: (Finisher::finisher_thread_entry()+0x188) [0x7f34f296b7b8]
2014-11-20T21:38:22.183 INFO:tasks.rados.rados.0.vpm200.stderr: 5: (()+0x79d1) [0x7f34f24b09d1]
2014-11-20T21:38:22.183 INFO:tasks.rados.rados.0.vpm200.stderr: 6: (clone()+0x6d) [0x7f34f1ce186d]


Related issues 1 (0 open1 closed)

Is duplicate of Ceph - Bug #10166: fiemap or FileStore::do_sparse_copy_range bug: osd/ReplicatedPG.cc: 8706: FAILED assert((data_included.empty() && data.length() == 0) || (!data_included.empty() && data.length() > 0))ResolvedHaomai Wang11/21/2014

Actions
Actions #1

Updated by Sage Weil over 9 years ago

  • Subject changed from "Assertion: ./test/osd/RadosModel.h" in upgrade:giant-x-next-distro-basic-vps run to ceph_test_rados got short read

2014-11-20T21:38:21.790 INFO:tasks.rados.rados.0.vpm200.stdout:only read 3829760 out of size 3832037

Actions #2

Updated by Yuri Weinstein over 9 years ago

Same problem in run http://pulpito.front.sepia.ceph.com/teuthology-2014-11-23_09:54:59-powercycle-giant-distro-basic-multi/

Jobs: ['615448', '615476', '615477']

Assertion: ./test/osd/RadosModel.h: 1077: FAILED assert(0)
ceph version 0.87-34-g37ffccb (37ffccbd57e7e441e0eb1499e5c173aa9c375d35)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0x7f340e1c32fb]
 2: ceph_test_rados() [0x421111]
 3: (librados::C_AioComplete::finish(int)+0x1d) [0x7f340e1254fd]
 4: (Context::complete(int)+0x9) [0x7f340e100fd9]
 5: (Finisher::finisher_thread_entry()+0x158) [0x7f340e1c23e8]
 6: (()+0x8182) [0x7f340dcff182]
 7: (clone()+0x6d) [0x7f340d51238d]
Actions #3

Updated by Yuri Weinstein over 9 years ago

Repeated issue in run http://pulpito.ceph.com/teuthology-2014-11-22_17:05:01-upgrade:giant-x-next-distro-basic-multi/

Jobs ['615149', '615151']

2014-11-22T18:32:06.190 INFO:tasks.rados.rados.0.plana09.stdout:3744: rollback oid 3 current snap is 485
2014-11-22T18:32:06.190 INFO:tasks.rados.rados.0.plana09.stdout:rollback oid 3 to 380
2014-11-22T18:32:06.191 INFO:tasks.rados.rados.0.plana09.stdout:3742:  expect (ObjNum 1191 snap 425 seq_num 1191)
2014-11-22T18:32:06.266 INFO:teuthology.task.swift:Configuring testswift...
2014-11-22T18:32:06.266 INFO:teuthology.task.swift:client=client.0
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:config={'clients': {'client.0': {'rgw_server': 'client.0'}}, 'testswift_conf': {'client.0': ConfigObj({'func_test': {'auth_port': 7280, 'auth_prefix': '/auth/', 'auth_ssl': 'no', 'account': 'foo.client.0', 'username': 'foo', 'email': 'foo.client.0+test@test.test', 'display_name': 'Mr. foo.client.0 foo', 'password': 'OgozC0daOewhOcfNjei75dka7swfcwBHt5C8UlxFPwV9EVAmwDtF2Q==', 'account2': 'bar.client.0', 'username2': 'bar', 'email2': 'bar.client.0+test@test.test', 'display_name2': 'Mr. bar.client.0 bar', 'password2': 'RJpa1NGEuXcnThh00Jg2tFhO+4ASv2J8VAPOgmMdIUl9kBNtosxw3w=='}})}}
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:roles: ['mon.a', 'mon.b', 'mon.c', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6']
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:target: ubuntu@plana13.front.sepia.ceph.com
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:roles: ['osd.7', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'osd.12', 'osd.13']
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:target: ubuntu@plana42.front.sepia.ceph.com
2014-11-22T18:32:06.267 INFO:teuthology.task.swift:roles: ['client.0']
2014-11-22T18:32:06.268 INFO:teuthology.task.swift:target: ubuntu@plana09.front.sepia.ceph.com
2014-11-22T18:32:06.268 INFO:teuthology.task.swift:client.0
2014-11-22T18:32:06.268 INFO:teuthology.orchestra.run.plana09:Running: 'cd /home/ubuntu/cephtest/swift && ./bootstrap'
2014-11-22T18:32:06.714 INFO:teuthology.orchestra.run.plana09.stdout:The --no-site-packages flag is deprecated; it is now the default behavior.
2014-11-22T18:32:06.714 INFO:teuthology.orchestra.run.plana09.stdout:New python executable in virtualenv/bin/python
2014-11-22T18:32:06.941 INFO:teuthology.orchestra.run.plana13:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph pg dump --format=json'
2014-11-22T18:32:07.256 INFO:teuthology.orchestra.run.plana13.stderr:dumped all in format json
2014-11-22T18:32:07.441 INFO:teuthology.orchestra.run.plana13:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
2014-11-22T18:32:07.444 INFO:tasks.rados.rados.0.plana09.stdout:incorrect buffer at pos 1322908
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr:3742: oid 7 contents (ObjNum 1191 snap 425 seq_num 1191) corrupt
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr:./test/osd/RadosModel.h: In function 'virtual void ReadOp::_finish(TestOp::CallbackInfo*)' thread 7f54e57fa700 time 2014-11-22 18:32:07.383754
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr:./test/osd/RadosModel.h: 1077: FAILED assert(0)
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr: ceph version 0.87-34-g37ffccb (37ffccbd57e7e441e0eb1499e5c173aa9c375d35)
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x7f) [0x7f54f19f8a0f]
2014-11-22T18:32:07.445 INFO:tasks.rados.rados.0.plana09.stderr: 2: (ReadOp::_finish(TestOp::CallbackInfo*)+0x14de) [0x41c27e]
2014-11-22T18:32:07.446 INFO:tasks.rados.rados.0.plana09.stderr: 3: (librados::C_AioComplete::finish(int)+0x1d) [0x7f54f19593cd]
2014-11-22T18:32:07.446 INFO:tasks.rados.rados.0.plana09.stderr: 4: (Context::complete(int)+0x9) [0x7f54f19358e9]
2014-11-22T18:32:07.446 INFO:tasks.rados.rados.0.plana09.stderr: 5: (Finisher::finisher_thread_entry()+0x160) [0x7f54f19f7c00]
2014-11-22T18:32:07.446 INFO:tasks.rados.rados.0.plana09.stderr: 6: (()+0x7e9a) [0x7f54f1597e9a]
2014-11-22T18:32:07.447 INFO:tasks.rados.rados.0.plana09.stderr: 7: (clone()+0x6d) [0x7f54f0dae3fd]

Actions #4

Updated by Samuel Just over 9 years ago

I think this also is due to enabling fiemap in the nightlies, teuthology commit:

0f97481ce44e0487ac6cffa051a05590f907a081

Actions #5

Updated by Samuel Just over 9 years ago

  • Status changed from New to Duplicate
Actions

Also available in: Atom PDF