Ceph » crimson

Found while testing wip-sjust-testing-2023-01-12-223717 (9ffcb1a0ee407ff1fc15150c7147dce6acc92e84 based on main commit e4b159899495702b54065dd8c4fdf5862d38f26d)

2023-01-13T10:18:06.258 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":1018,"stamp":"2023-01-13T10:18:05.571201+0000","last_osdmap_epoch":0,">
2023-01-13T10:18:06.261 INFO:tasks.ceph.ceph_manager.ceph:recovered!
2023-01-13T10:18:06.261 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.00
2023-01-13T10:18:06.262 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dum>
2023-01-13T10:18:06.390 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:06.382+0000 7f93e7665700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:06.392 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:06.389+0000 7f93e7665700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:06.490 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tel>
2023-01-13T10:18:06.580 INFO:teuthology.orchestra.run.smithi101.stdout:
2023-01-13T10:18:06.583 INFO:teuthology.orchestra.run.smithi101.stderr:dumped all
2023-01-13T10:18:06.595 INFO:teuthology.orchestra.run.smithi101.stdout:{"pg_ready":true,"pg_map":{"version":1018,"stamp":"2023-01-13T10:18:05.571201+0000","last_osdmap_epoch":0,">
2023-01-13T10:18:06.617 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:06.613+0000 7f551dfcf700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:06.620 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:06.617+0000 7f551dfcf700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:08.692 DEBUG:teuthology.orchestra.run.smithi101:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tel>
2023-01-13T10:18:08.822 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:08.819+0000 7f628b995700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:08.824 INFO:teuthology.orchestra.run.smithi101.stderr:2023-01-13T10:18:08.823+0000 7f628b995700 -1 WARNING: the following dangerous and experimental features are>
2023-01-13T10:18:10.137 INFO:tasks.rados.rados.0.smithi101.stdout:825:  finishing write tid 3 to smithi101103696-825
2023-01-13T10:18:10.138 INFO:tasks.rados.rados.0.smithi101.stderr:Error: finished tid 3 when last_acked_tid was 5
2023-01-13T10:18:10.138 INFO:tasks.rados.rados.0.smithi101.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/>
2023-01-13T10:18:10.139 INFO:tasks.rados.rados.0.smithi101.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/>
2023-01-13T10:18:10.139 INFO:tasks.rados.rados.0.smithi101.stderr: ceph version 18.0.0-1783-g9ffcb1a0 (9ffcb1a0ee407ff1fc15150c7147dce6acc92e84) reef (dev)
2023-01-13T10:18:10.139 INFO:tasks.rados.rados.0.smithi101.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, >
2023-01-13T10:18:10.139 INFO:tasks.rados.rados.0.smithi101.stderr: 2: (WriteOp::_finish(TestOp::CallbackInfo*)+0x1b0) [0x5650f81bc0ca]
2023-01-13T10:18:10.140 INFO:tasks.rados.rados.0.smithi101.stderr: 3: (TestOp::finish(TestOp::CallbackInfo*)+0xe) [0x5650f81d417c]
2023-01-13T10:18:10.140 INFO:tasks.rados.rados.0.smithi101.stderr: 4: (write_callback(void*, void*)+0x1c) [0x5650f81d41b7]
2023-01-13T10:18:10.140 INFO:tasks.rados.rados.0.smithi101.stderr: 5: /lib64/librados.so.2(+0xb92f4) [0x7fcaa1ec22f4]
2023-01-13T10:18:10.140 INFO:tasks.rados.rados.0.smithi101.stderr: 6: /lib64/librados.so.2(+0xd244d) [0x7fcaa1edb44d]
2023-01-13T10:18:10.140 INFO:tasks.rados.rados.0.smithi101.stderr: 7: /lib64/librados.so.2(+0xd20a0) [0x7fcaa1edb0a0]
2023-01-13T10:18:10.141 INFO:tasks.rados.rados.0.smithi101.stderr: 8: /lib64/librados.so.2(+0xd34ec) [0x7fcaa1edc4ec]
2023-01-13T10:18:10.141 INFO:tasks.rados.rados.0.smithi101.stderr: 9: /lib64/librados.so.2(+0xd5b94) [0x7fcaa1edeb94]
2023-01-13T10:18:10.141 INFO:tasks.rados.rados.0.smithi101.stderr: 10: /lib64/libstdc++.so.6(+0xc2ba3) [0x7fca9f3a5ba3]
2023-01-13T10:18:10.141 INFO:tasks.rados.rados.0.smithi101.stderr: 11: /lib64/libpthread.so.0(+0x81ca) [0x7fca9ff6a1ca]
2023-01-13T10:18:10.141 INFO:tasks.rados.rados.0.smithi101.stderr: 12: clone()

Seems to be a recent development, I think it's unrelated to the PR being tested. I'm going to bisect it once the lab works again.