Bug #13432
"Mutex.cc: 105: FAILED assert(r == 0)" in rbd run
0%
Description
Run: http://pulpito.ceph.com/teuthology-2015-10-05_23:00:07-rbd-master---basic-multi/
Job: 1091223
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2015-10-05_23:00:07-rbd-master---basic-multi/1091223/teuthology.log
2015-10-08T13:36:15.216 INFO:teuthology.orchestra.run.plana09.stdout:rbd engine: RBD version: 0.1.10 2015-10-08T13:36:15.350 INFO:teuthology.orchestra.run.plana09.stdout:rbd engine: RBD version: 0.1.10 2015-10-08T13:42:49.619 INFO:teuthology.orchestra.run.plana09.stderr:common/Mutex.cc: In function 'void Mutex::Lock(bool)' thread 7f9c057ea700 time 2015-10-08 13:42:49.592973 2015-10-08T13:42:49.620 INFO:teuthology.orchestra.run.plana09.stderr:common/Mutex.cc: 105: FAILED assert(r == 0) 2015-10-08T13:42:49.634 INFO:teuthology.orchestra.run.plana09.stderr: ceph version 9.0.3-2146-gd1e6976 (d1e697628931775e7fcdbb79a76ddd8e7dedffd5) 2015-10-08T13:42:49.634 INFO:teuthology.orchestra.run.plana09.stderr: 1: (()+0x18ebcb) [0x7f9c68dfdbcb] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 2: (()+0x144dcc) [0x7f9c68db3dcc] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 3: (()+0x5c4c0) [0x7f9c68ccb4c0] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 4: /home/ubuntu/rbd-fio-test/fio-fio-2.2.9/fio() [0x462904] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 5: (()+0x5c149) [0x7f9c68ccb149] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 6: (()+0x5d145) [0x7f9c68ccc145] 2015-10-08T13:42:49.635 INFO:teuthology.orchestra.run.plana09.stderr: 7: (()+0x5d64d) [0x7f9c68ccc64d] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 8: (()+0x5b899) [0x7f9c68cca899] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 9: (()+0x5de37) [0x7f9c68ccce37] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 10: (()+0x5bd18) [0x7f9c68ccad18] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 11: (()+0x81e62) [0x7f9c68cf0e62] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 12: (()+0x180126) [0x7f9c68def126] 2015-10-08T13:42:49.636 INFO:teuthology.orchestra.run.plana09.stderr: 13: (()+0x181010) [0x7f9c68df0010] 2015-10-08T13:42:49.637 INFO:teuthology.orchestra.run.plana09.stderr: 14: (()+0x8182) [0x7f9c65f39182] 2015-10-08T13:42:49.637 INFO:teuthology.orchestra.run.plana09.stderr: 15: (clone()+0x6d) [0x7f9c65a6247d] 2015-10-08T13:42:49.637 INFO:teuthology.orchestra.run.plana09.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. 2015-10-08T13:42:49.647 INFO:teuthology.orchestra.run.plana09.stderr:terminate called after throwing an instance of 'ceph::FailedAssertion' 2015-10-08T13:42:49.721 INFO:tasks.rbd_fio:Cleaning up fio install
History
#1 Updated by Yuri Weinstein over 8 years ago
- Subject changed from "Mutex.cc: 105: FAILED assert(r == 0)" in to "Mutex.cc: 105: FAILED assert(r == 0)" in rbd run
#2 Updated by Vasu Kulkarni over 8 years ago
- Project changed from Ceph to rbd
Is this a issue with rbd fio engine? If So I can open a ticket at fio github to get it fixed.
#3 Updated by Vasu Kulkarni over 8 years ago
yeah this is from ceph common/mutex.cc not fio
#4 Updated by Jason Dillaman over 8 years ago
- Status changed from New to In Progress
- Assignee set to Jason Dillaman
#5 Updated by Jason Dillaman over 8 years ago
Race condition within fio rbd engine where a 'fio_rbd_iou' object is reused prior to retrieving the return value from the completion.
#6 Updated by Jason Dillaman over 8 years ago
- Status changed from In Progress to 15
#7 Updated by Yuri Weinstein over 8 years ago
Also in run: http://pulpito.ceph.com/teuthology-2015-10-27_23:00:01-rbd-infernalis-distro-basic-multi/
Jobs: ['1129560', '1129605']
#8 Updated by Tamilarasi muthamizhan over 8 years ago
the fix is pending upstream for fio.
not a blocker for ceph. once fio bug is fixed, the teuthology test has to be modified to pull the latest fio.
ps: this is a new test that was added for infernalis.
#9 Updated by Jason Dillaman over 8 years ago
- Status changed from 15 to Resolved
Fixed included in fio v2.2.11 release