https://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2014-08-11T07:23:47ZCeph Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391532014-08-11T07:23:47ZJohn Sprayjcspray@gmail.com
<ul></ul><p><a class="external" href="http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411119/teuthology.log">http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411119/teuthology.log</a><br /><a class="external" href="http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411120/teuthology.log">http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411120/teuthology.log</a><br /><a class="external" href="http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411159/teuthology.log">http://qa-proxy.ceph.com/teuthology/john-2014-08-10_02:14:59-rados-wip-mds-contexts-testing-basic-plana/411159/teuthology.log</a></p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391542014-08-11T07:25:44ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/39154/diff?detail_id=39937">diff</a>)</li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391602014-08-11T09:29:23ZJohn Sprayjcspray@gmail.com
<ul></ul><p>I understand this a little better now: the operations in this WriteOp are 1,2 (writes), 4 (setxattr), 5 (read). So the read is apparently completing before any of the writes.</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391612014-08-11T09:57:02ZJohn Sprayjcspray@gmail.com
<ul></ul><p>trying to reproduce locally with objecter logging turned up and ``ms inject socket failures`` enabled as it is in the failing cases.</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391642014-08-11T11:40:18ZJohn Sprayjcspray@gmail.com
<ul></ul><p>Got an even more specific backtrace</p>
<pre>
2014-08-11 19:35:36,654.654 INFO:teuthology.task.rados.rados.0.mira114.stdout:update_object_version oid 2506 v 1285 (ObjNum 2505 snap 0 seq_num 2505) dirty exists
2014-08-11 19:35:36,654.654 INFO:teuthology.task.rados.rados.0.mira114.stderr:Error: racing read on 2506 returned version 0 rather than version 1285
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr:./test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(TestOp::CallbackInfo*)' thread 7fec1b7fe700 time 2014-08-11 11:34:44.425107
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr:./test/osd/RadosModel.h: 847: FAILED assert(0 == "racing read got wrong version")
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: ceph version 0.83-704-g1f44a2e (1f44a2ed1780ff8c04d5d040ccc694f31ab8f509)
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 1: ceph_test_rados() [0x41a18b]
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 2: (write_callback(void*, void*)+0x21) [0x4279b1]
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 3: (librados::C_AioSafe::finish(int)+0x1d) [0x7fec2f994d7d]
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 4: (Context::complete(int)+0x9) [0x7fec2f972229]
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 5: (Finisher::finisher_thread_entry()+0x1c0) [0x7fec2fa2f120]
2014-08-11 19:35:36,655.655 INFO:teuthology.task.rados.rados.0.mira114.stderr: 6: (()+0x7e9a) [0x7fec2f5d5e9a]
2014-08-11 19:35:36,656.656 INFO:teuthology.task.rados.rados.0.mira114.stderr: 7: (clone()+0x6d) [0x7fec2edec3fd]
2014-08-11 19:35:36,656.656 INFO:teuthology.task.rados.rados.0.mira114.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
</pre> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=391672014-08-11T12:24:23ZJohn Sprayjcspray@gmail.com
<ul></ul><p>The bug is happening when a new OSD map is received in the middle of the series of transactions. The read transaction is getting redirected to another OSD, while the write transactions are responded to by the original OSD. This results in the writes and reads flowing via different OSD sessions (and threads) in the objecter.</p>
<p>Given that this was triggered by changes to the client side code, I presume the way this is meant to work is that it's the client's responsibility to sort out this ordering, even though in fact it is receiving the responses out of order (responses are ordered correctly from each individual OSD's POV but because they're interleaved from two OSDs they arrive in undefined order at the client).</p>
<p>Here's the exception:<br /><pre>
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: finishing write tid 4 to mira11419081-2468
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stdout:update_object_version oid 2468 v 1973 (ObjNum 2467 snap 0 seq_num 2467) dirty exists
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stderr:Error: racing read on 2468 returned version 0 rather than version 1973
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stderr:./test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(TestOp::CallbackInfo*)' thread 7f0779ffb700 time 2014-08-11 11:50:50.737857
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stderr:./test/osd/RadosModel.h: 847: FAILED assert(0 == "racing read got wrong version")
2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stderr: ceph version 0.83-704-g1f44a2e (1f44a2ed1780ff8c04d5d040ccc694f31ab8f509)
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 1: ceph_test_rados() [0x41a18b]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 2: (write_callback(void*, void*)+0x21) [0x4279b1]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 3: (librados::C_AioSafe::finish(int)+0x1d) [0x7f0786470d7d]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 4: (Context::complete(int)+0x9) [0x7f078644e229]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 5: (Finisher::finisher_thread_entry()+0x1c0) [0x7f078650b120]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 6: (()+0x7e9a) [0x7f07860b1e9a]
2014-08-11T19:51:42.987 INFO:teuthology.task.rados.rados.0.mira114.stderr: 7: (clone()+0x6d) [0x7f07858c83fd]
</pre></p>
<p>This is the log from the ceph_test_rados process for object mira11419081-2468:<br /><pre>
9064_teuthology.log:2014-08-11T19:51:34.695 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: writing mira11419081-2468 from 729525 to 1364279 tid 1
9064_teuthology.log:2014-08-11T19:51:34.695 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: writing mira11419081-2468 from 1861328 to 2329525 tid 2
9064_teuthology.log:2014-08-11T19:51:37.708 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: finishing write tid 5 to mira11419081-2468
9064_teuthology.log:2014-08-11T19:51:42.982 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: finishing write tid 1 to mira11419081-2468
9064_teuthology.log:2014-08-11T19:51:42.985 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: finishing write tid 2 to mira11419081-2468
9064_teuthology.log:2014-08-11T19:51:42.986 INFO:teuthology.task.rados.rados.0.mira114.stdout:2468: finishing write tid 4 to mira11419081-2468
</pre></p>
<pre>
9064_ceph-client.0.19081.log:2014-08-11 11:50:42.427728 7f0787408780 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @2 [write 729525~634754] tid 0 osd.2
9064_ceph-client.0.19081.log:2014-08-11 11:50:42.444512 7f0787408780 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @2 [write 1861328~468197] tid 0 osd.2
9064_ceph-client.0.19081.log:2014-08-11 11:50:42.444572 7f0787408780 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @2 [setxattr _header (56),truncate 2329525] tid 0 osd.2
9064_ceph-client.0.19081.log:2014-08-11 11:50:42.444598 7f0787408780 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @2 [read 0~1] tid 0 osd.2
</pre>
<p>This is the log from the client log with debug_objecter=20 for object mira11419081-2468:<br /><pre>
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.451847 7f07817bc700 20 client.4119.objecter 14622 2.beafe6c5 osd.2 mira11419081-2468 [write 729525~634754]
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.451855 7f07817bc700 20 client.4119.objecter 14623 2.beafe6c5 osd.2 mira11419081-2468 [write 1861328~468197]
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.451863 7f07817bc700 20 client.4119.objecter 14624 2.beafe6c5 osd.2 mira11419081-2468 [setxattr _header (56),truncate 2329525]
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.451872 7f07817bc700 20 client.4119.objecter 14625 2.beafe6c5 osd.2 mira11419081-2468 [read 0~1]
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.460331 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14625 mira11419081-2468 [read 0~1] v0'0 uv0 ack = -2 ((2) No such file or directory) redirect: { object , locator{@1} }) v6
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.460405 7f07801b4700 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @1 [read 0~1] tid 0 osd.5
9064_ceph-client.0.19081.log:2014-08-11 11:50:45.463611 7f07803b6700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14793 mira11419081-2468 [read 0~1] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6
9064_ceph-client.0.19081.log:2014-08-11 11:50:50.733817 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14622 mira11419081-2468 [write 729525~634754] v21'1179 uv1971 ondisk = 0) v6
9064_ceph-client.0.19081.log:2014-08-11 11:50:50.737458 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14623 mira11419081-2468 [write 1861328~468197] v21'1180 uv1972 ondisk = 0) v6
9064_ceph-client.0.19081.log:2014-08-11 11:50:50.737724 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14624 mira11419081-2468 [setxattr (56),truncate 2329525] v21'1181 uv1973 ondisk = 0) v6
</pre></p>
<p>The ops initially issued are:<br /><pre>
2014-08-11 11:50:42.427747 7f0787408780 15 client.4119.objecter _send_op 14622 to osd.2
2014-08-11 11:50:42.444529 7f0787408780 15 client.4119.objecter _send_op 14623 to osd.2
2014-08-11 11:50:42.444576 7f0787408780 15 client.4119.objecter _send_op 14624 to osd.2
2014-08-11 11:50:42.444601 7f0787408780 15 client.4119.objecter _send_op 14625 to osd.2
</pre></p>
<p>Then we get a new OSD map:<br /><pre>
2014-08-11 11:50:45.450774 7f07817bc700 3 client.4119.objecter handle_osd_map decoding incremental epoch 21
</pre></p>
<p>The read transaction (14625) is redirected via osd.5:<br /><pre>
2014-08-11 11:50:45.460331 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14625 mira11419081-2468 [read 0~1] v0'0 uv0 ack = -2 ((2) No such file or directory) redirect: { object , locator{@1} }) v6
2014-08-11 11:50:45.460361 7f07801b4700 7 client.4119.objecter handle_osd_op_reply 14625 ack v 0'0 uv 0 in 2.beafe6c5 attempt 0
2014-08-11 11:50:45.460369 7f07801b4700 5 client.4119.objecter got redirect reply; redirecting
2014-08-11 11:50:45.460375 7f07801b4700 10 client.4119.objecter _op_submit op 0x21c66a0
2014-08-11 11:50:45.460392 7f07801b4700 10 client.4119.objecter _calc_target pgid 1.beafe6c5 acting [5,0]
2014-08-11 11:50:45.460401 7f07801b4700 20 client.4119.objecter note: not requesting commit
2014-08-11 11:50:45.460405 7f07801b4700 10 client.4119.objecter _op_submit oid mira11419081-2468 @1 @1 [read 0~1] tid 0 osd.5
2014-08-11 11:50:45.460419 7f07801b4700 15 client.4119.objecter _send_op 14793 to osd.5
2014-08-11 11:50:45.463611 7f07803b6700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14793 mira11419081-2468 [read 0~1] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6
2014-08-11 11:50:45.463634 7f07803b6700 7 client.4119.objecter handle_osd_op_reply 14793 ack v 0'0 uv 0 in 1.beafe6c5 attempt 1
2014-08-11 11:50:45.463651 7f07803b6700 15 client.4119.objecter handle_osd_op_reply completed tid 14793
2014-08-11 11:50:45.463656 7f07803b6700 15 client.4119.objecter finish_op 14793
</pre></p>
<p>Whereas a write transaction (e.g. 14622) is not:<br /><pre>
2014-08-11 11:50:50.733817 7f07801b4700 10 client.4119.objecter ms_dispatch 0x1d40440 osd_op_reply(14622 mira11419081-2468 [write 729525~634754] v21'1179 uv1971 ondisk = 0) v6
2014-08-11 11:50:50.733825 7f07801b4700 7 client.4119.objecter handle_osd_op_reply 14622 ondisk v 21'1179 uv 1971 in 2.beafe6c5 attempt 0
2014-08-11 11:50:50.733834 7f07801b4700 15 client.4119.objecter handle_osd_op_reply completed tid 14622
2014-08-11 11:50:50.733835 7f07801b4700 15 client.4119.objecter finish_op 14622
</pre></p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=392102014-08-12T05:17:29ZJohn Sprayjcspray@gmail.com
<ul></ul><p>This just reproduced on master 78dc4df, so looks like it's not wip-objecter specific.</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=392112014-08-12T05:17:39ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Subject</strong> changed from <i>(wip-objecter) RadosModel assertion failure</i> to <i>RadosModel assertion failure</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=392872014-08-12T13:26:56ZSamuel Justsjust@redhat.com
<ul><li><strong>Assignee</strong> set to <i>Samuel Just</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Urgent</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393022014-08-12T14:00:06ZJohn Sprayjcspray@gmail.com
<ul><li><strong>File</strong> <a href="/attachments/download/1338/config.yaml">config.yaml</a> <a class="icon-only icon-magnifier" title="View" href="/attachments/1338/config.yaml">View</a> added</li><li><strong>Assignee</strong> deleted (<del><i>Samuel Just</i></del>)</li><li><strong>Priority</strong> changed from <i>Urgent</i> to <i>Normal</i></li></ul><p>I've now seen this in a case where the client wasn't in the process of handling a new OSD map (but the server was), so I think this is probably an issue in the OSD sending redirects for old-epoch requests that it shouldn't be responding to.</p>
<p>There are logs from a failure on master in ~/jcsp/9064 on the teuthology server (this was run by hand hence the unconventional location).</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393032014-08-12T14:00:28ZJohn Sprayjcspray@gmail.com
<ul><li><strong>Assignee</strong> set to <i>Samuel Just</i></li><li><strong>Priority</strong> changed from <i>Normal</i> to <i>Urgent</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393442014-08-12T15:25:39ZSamuel Justsjust@redhat.com
<ul></ul><p>Got it: 0ed3adc1e0a74bf9548d1d956aece11f019afee0</p>
<p>We're redirecting RW ordered reads due to the second read promote thing.</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393462014-08-12T15:26:26ZSamuel Justsjust@redhat.com
<ul></ul><p>wip-9064</p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393472014-08-12T15:26:38ZSamuel Justsjust@redhat.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>7</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393492014-08-12T15:48:22ZSage Weilsage@newdream.net
<ul><li><strong>Status</strong> changed from <i>7</i> to <i>Pending Backport</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=393502014-08-12T15:48:31ZSamuel Justsjust@redhat.com
<ul><li><strong>Status</strong> changed from <i>Pending Backport</i> to <i>Resolved</i></li></ul> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=541132015-06-24T21:02:29ZLoïc Dacharyloic@dachary.org
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Pending Backport</i></li><li><strong>Backport</strong> set to <i>firefly</i></li><li><strong>Regression</strong> set to <i>No</i></li></ul><pre>
2015-06-24 10:13:34,513.513 INFO:tasks.rados.rados.0.entercloudsuite012.stdout:695: finishing write tid 1 to entercloudsuite01212574-27
2015-06-24 10:13:34,513.513 INFO:tasks.rados.rados.0.entercloudsuite012.stderr:Error: finished tid 1 when last_acked_tid was 5
2015-06-24 10:13:34,514.514 INFO:tasks.rados.rados.0.entercloudsuite012.stderr:./test/osd/RadosModel.h: In function 'virtual void WriteOp::_finish(TestOp::CallbackInfo*)' thread 7fc4e67fc700 time 201\
5-06-24 20:00:44.189448
2015-06-24 10:13:34,514.514 INFO:tasks.rados.rados.0.entercloudsuite012.stderr:./test/osd/RadosModel.h: 828: FAILED assert(0)
2015-06-24 10:13:34,515.515 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: ceph version 0.80.10-2-g1623e6c (1623e6c6517ff6958b343493fce6a8e50a52a042)
2015-06-24 10:13:34,519.519 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 1: (WriteOp::_finish(TestOp::CallbackInfo*)+0x2e0) [0x417220]
2015-06-24 10:13:34,519.519 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 2: (write_callback(void*, void*)+0x19) [0x4274b9]
2015-06-24 10:13:34,519.519 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 3: (librados::C_AioSafe::finish(int)+0x1d) [0x7fc4f1a502bd]
2015-06-24 10:13:34,519.519 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 4: (Context::complete(int)+0x9) [0x7fc4f1a2d089]
2015-06-24 10:13:34,519.519 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 5: (Finisher::finisher_thread_entry()+0x1b8) [0x7fc4f1adeb68]
2015-06-24 10:13:34,520.520 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 6: (()+0x8182) [0x7fc4f1675182]
2015-06-24 10:13:34,520.520 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: 7: (clone()+0x6d) [0x7fc4f0e8847d]
2015-06-24 10:13:34,520.520 INFO:tasks.rados.rados.0.entercloudsuite012.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2015-06-24 10:13:34,520.520 INFO:tasks.rados.rados.0.entercloudsuite012.stderr:terminate called after throwing an instance of 'ceph::FailedAssertion'
</pre>
<p>While running <strong>upgrade:firefly-x/parallel/{0-cluster/start.yaml 1-firefly-install/firefly.yaml 2-workload/{ec-rados-parallel.yaml rados_api.yaml rados_loadgenbig.yaml test_rbd_api.yaml test_rbd_python.yaml} 3-upgrade-sequence/upgrade-all.yaml 4-final-workload/{rados-snaps-few-objects.yaml rados_loadgenmix.yaml rados_mon_thrash.yaml rbd_cls.yaml rbd_import_export.yaml rgw_swift.yaml} distros/ubuntu_14.04.yaml}</strong></p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=547372015-07-10T21:14:30ZLoïc Dacharyloic@dachary.org
<ul></ul><p>The commit fixing this issue <a class="external" href="http://tracker.ceph.com/projects/ceph/repository/revisions/24aeca9c439474b079d23600d8d42c0af5d9ce5c">http://tracker.ceph.com/projects/ceph/repository/revisions/24aeca9c439474b079d23600d8d42c0af5d9ce5c</a> refers to 0ed3adc which was introduced by <a class="external" href="http://tracker.ceph.com/issues/8674#note-12">http://tracker.ceph.com/issues/8674#note-12</a></p> Ceph - Bug #9064: RadosModel assertion failurehttps://tracker.ceph.com/issues/9064?journal_id=557962015-07-28T18:07:59ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Status</strong> changed from <i>Pending Backport</i> to <i>Resolved</i></li></ul>