Actions
Bug #47326
closedout of order i/o
Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2020-09-06T10:23:43.906 INFO:tasks.rados.rados.0.smithi060.stdout:251: writing smithi06020638-251 from 601750 to 1137296 tid 1
..
2020-09-06T10:23:43.912 INFO:tasks.rados.rados.0.smithi060.stdout:251: writing smithi06020638-251 from 1929910 to 2201750 tid 2
..
2020-09-06T10:23:44.686 INFO:tasks.rados.rados.0.smithi060.stdout:251: finishing write tid 4 to smithi06020638-251
..
2020-09-06T10:23:45.846 INFO:tasks.rados.rados.0.smithi060.stdout:update_object_version oid 320 v 0 (ObjNum 319 snap 0 seq_num 319) dirty exists
2020-09-06T10:23:45.846 INFO:tasks.rados.rados.0.smithi060.stdout:320: left oid 320 (ObjNum 319 snap 0 seq_num 319)
2020-09-06T10:23:45.847 INFO:tasks.rados.rados.0.smithi060.stdout:251: finishing write tid 1 to smithi06020638-251
2020-09-06T10:23:45.847 INFO:tasks.rados.rados.0.smithi060.stderr:Error: finished tid 1 when last_acked_tid was 4
2020-09-06T10:23:45.847 INFO:tasks.rados.rados.0.smithi060.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST
/centos8/MACHINE_SIZE/gigantic/release/16.0.0-5172-g4bfd9ba125/rpm/el8/BUILD/ceph-16.0.0-5172-g4bfd9ba125/src/test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(Te
stOp::CallbackInfo*)' thread 7f758affd700 time 2020-09-06T10:23:45.849805+0000
2020-09-06T10:23:45.847 INFO:tasks.rados.rados.0.smithi060.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST
/centos8/MACHINE_SIZE/gigantic/release/16.0.0-5172-g4bfd9ba125/rpm/el8/BUILD/ceph-16.0.0-5172-g4bfd9ba125/src/test/osd/RadosModel.h: 902: ceph_abort_msg("abort() called")
2020-09-06T10:23:45.847 INFO:tasks.rados.rados.0.smithi060.stderr: ceph version 16.0.0-5172-g4bfd9ba125 (4bfd9ba1258d00846dcce0465fb67610aa19332f) pacific (dev)
2020-09-06T10:23:45.848 INFO:tasks.rados.rados.0.smithi060.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > const&)+0xe0) [0x7f759e82f3ce]
2020-09-06T10:23:45.848 INFO:tasks.rados.rados.0.smithi060.stderr: 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x55f587b85e62]
2020-09-06T10:23:45.848 INFO:tasks.rados.rados.0.smithi060.stderr: 3: (write_callback(void*, void*)+0x1e) [0x55f587ba5a8e]
/a//kchai-2020-09-06_10:10:34-crimson-rados:thrash-master-distro-basic-smithi/5411388/
Updated by Kefu Chai over 3 years ago
2020-09-17T08:00:17.173 INFO:tasks.rados.rados.0.smithi205.stdout:635: write initial oid 635
2020-09-17T08:00:17.173 INFO:tasks.rados.rados.0.smithi205.stdout:635: seq_num 634 ranges {655638=565165,1900172=421542}
2020-09-17T08:00:17.178 INFO:tasks.rados.rados.0.smithi205.stdout:635: writing smithi20527322-635 from 655638 to 1220803 tid 1
2020-09-17T08:00:17.184 INFO:tasks.rados.rados.0.smithi205.stdout:635: writing smithi20527322-635 from 1900172 to 2321714 tid 2
...
2020-09-17T08:00:47.710 INFO:tasks.rados.rados.0.smithi205.stdout:635: finishing write tid 4 to smithi20527322-635
...
2020-09-17T08:00:49.432 INFO:tasks.rados.rados.0.smithi205.stdout:635: finishing write tid 1 to smithi20527322-635
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr:Error: finished tid 1 when last_acked_tid was 4
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.0.0-5569-g36d67cf5/rpm/el8/BUILD/ceph-16.0.0-5569-g36d67cf5/src/test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(TestOp::CallbackInfo*)' thread 7f4b6ffff700 time 2020-09-17T08:00:48.241799+0000
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/16.0.0-5569-g36d67cf5/rpm/el8/BUILD/ceph-16.0.0-5569-g36d67cf5/src/test/osd/RadosModel.h: 902: ceph_abort_msg("abort() called")
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr: ceph version 16.0.0-5569-g36d67cf5 (36d67cf59e365a1802bd6cbd0c4cc744d7183ec8) pacific (dev)
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe0) [0x7f4b83b117b0]
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr: 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x6e2) [0x55774b515e62]
2020-09-17T08:00:49.433 INFO:tasks.rados.rados.0.smithi205.stderr: 3: (write_callback(void*, void*)+0x1e) [0x55774b535ade]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 4: (()+0xb0d57) [0x7f4b8d11ad57]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 5: (()+0xcf4f5) [0x7f4b8d1394f5]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 6: (()+0xcee62) [0x7f4b8d138e62]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 7: (()+0xd504a) [0x7f4b8d13f04a]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 8: (()+0xc2b23) [0x7f4b8241fb23]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 9: (()+0x82de) [0x7f4b82f692de]
2020-09-17T08:00:49.434 INFO:tasks.rados.rados.0.smithi205.stderr: 10: (clone()+0x43) [0x7f4b81afc133]
/a/kchai-2020-09-17_07:42:16-crimson-rados:thrash-master-distro-basic-smithi/5443448
Updated by Kefu Chai over 3 years ago
we should keep the order of pending requests including the one still being processed when re-enqueue them. Xuehan is working on a sequencer for client requests.
Updated by hg liu about 3 years ago
Kefu Chai wrote:
[...]
/a//kchai-2020-09-06_10:10:34-crimson-rados:thrash-master-distro-basic-smithi/5411388/
test
Actions