Bug #20236
open
bluestore: ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2 failure
Added by Sage Weil almost 7 years ago.
Updated almost 4 years ago.
Backport:
nautilus,octopus
Description
2017-06-09T03:37:28.314 INFO:teuthology.orchestra.run.smithi068.stderr:seeding object 0
2017-06-09T03:37:28.314 INFO:teuthology.orchestra.run.smithi068.stderr:Op 0
2017-06-09T03:37:28.314 INFO:teuthology.orchestra.run.smithi068.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-06-09T03:37:41.290 INFO:teuthology.orchestra.run.smithi068.stderr:Op 200
2017-06-09T03:37:41.291 INFO:teuthology.orchestra.run.smithi068.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-06-09T03:37:55.988 INFO:teuthology.orchestra.run.smithi068.stderr:Op 400
2017-06-09T03:37:55.989 INFO:teuthology.orchestra.run.smithi068.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-06-09T03:38:12.470 INFO:teuthology.orchestra.run.smithi068.stderr:Op 600
2017-06-09T03:38:12.470 INFO:teuthology.orchestra.run.smithi068.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-06-09T03:38:31.486 INFO:teuthology.orchestra.run.smithi068.stderr:Op 800
2017-06-09T03:38:31.486 INFO:teuthology.orchestra.run.smithi068.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-06-09T03:38:52.763 INFO:teuthology.orchestra.run.smithi068.stdout:/build/ceph-12.0.2-2533-g080b00b/src/test/objectstore/store_test.cc:5729: Failure
2017-06-09T03:38:52.764 INFO:teuthology.orchestra.run.smithi068.stdout: Expected: res_stat.allocated
2017-06-09T03:38:52.764 INFO:teuthology.orchestra.run.smithi068.stdout: Which is: 4259840
2017-06-09T03:38:52.764 INFO:teuthology.orchestra.run.smithi068.stdout:To be equal to: max_object
2017-06-09T03:38:52.764 INFO:teuthology.orchestra.run.smithi068.stdout: Which is: 4194304
/a/yuriw-2017-06-08_19:40:51-rados-wip-yuri-testing2_2017_7_10-distro-basic-smithi/1271584
- Project changed from Ceph to RADOS
- Component(RADOS) BlueStore added
- Status changed from 12 to Need More Info
- Priority changed from Immediate to Urgent
hmm, passed 1000 iterations for me. Not sure what this might be.. Igor?
- Priority changed from Urgent to High
- Priority changed from High to Normal
- Project changed from RADOS to bluestore
/a//kchai-2018-01-11_06:11:31-rados-wip-kefu-testing-2018-01-11-1036-distro-basic-mira/2058374/teuthology.log
2018-01-11T10:36:08.666 INFO:teuthology.orchestra.run.mira057.stdout:/build/ceph-13.0.0-4698-g955c807/src/test/objectstore/store_test.cc:5828: Failure
2018-01-11T10:36:08.666 INFO:teuthology.orchestra.run.mira057.stdout: Expected: res_stat.allocated
2018-01-11T10:36:08.666 INFO:teuthology.orchestra.run.mira057.stdout: Which is: 4259840
2018-01-11T10:36:08.666 INFO:teuthology.orchestra.run.mira057.stdout:To be equal to: max_object
2018-01-11T10:36:08.666 INFO:teuthology.orchestra.run.mira057.stdout: Which is: 4194304
2018-01-11T10:36:09.742 INFO:teuthology.orchestra.run.mira057.stdout:==> rm -r bluestore.test_temp_dir
2018-01-11T10:36:09.761 INFO:teuthology.orchestra.run.mira057.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2, where GetParam() = "bluestore" (11688 ms)
exactly the same failure.
- Status changed from Need More Info to In Progress
- Assignee set to Igor Fedotov
It looks like I have some insight on the root cause, got that much more frequently once having some blobs at DB only (which is my another task intended to optimize space overhead while keeping tiny objects).
So the general idea is that under some extent/blob layout BlueStore is unable to reuse existing allocated pextent within a blob and hence creates a new blob. Which leads to space usage higher than expected.
- Has duplicate Bug #36422: ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
I haven't seen this in a while.. have you?
Sage Weil wrote:
I haven't seen this in a while.. have you?
Me too. Just gave this another try for both master and tiny write PR which I mentioned in comment #7.
Both are fine.
So closing for now.
- Status changed from In Progress to Can't reproduce
- Status changed from Can't reproduce to 12
/a/sage-2019-04-24_22:14:24-rados-wip-sage2-testing-2019-04-24-0829-distro-basic-smithi/3890129
2019-04-25T03:45:42.427 INFO:teuthology.orchestra.run.smithi083.stderr:Op 800
2019-04-25T03:45:42.428 INFO:teuthology.orchestra.run.smithi083.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2019-04-25T03:45:45.922 INFO:teuthology.orchestra.run.smithi083.stdout:/build/ceph-15.0.0-654-g97653e1/src/test/objectstore/store_test.cc:6267: Failure
2019-04-25T03:45:45.922 INFO:teuthology.orchestra.run.smithi083.stdout:Expected: (res_stat.allocated) <= (max_object_size), actual: 4259840 vs 4194304
2019-04-25T03:45:46.436 INFO:teuthology.orchestra.run.smithi083.stdout:==> rm -r bluestore.test_temp_dir
2019-04-25T03:45:46.508 INFO:teuthology.orchestra.run.smithi083.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2, where GetParam() = "bluestore" (9806 ms)
- Priority changed from Normal to Low
- Priority changed from Low to Normal
This appeared in nautilus
/a/yuriw-2019-11-12_15:30:19-rados-wip-yuri5-testing-2019-11-11-1520-nautilus-distro-basic-smithi/4499742/
- Priority changed from Normal to High
/a/yuriw-2019-11-08_20:50:44-rados-wip-yuri3-testing-2019-11-08-1221-nautilus-distro-basic-smithi/4485359/
/a/nojha-2019-11-22_22:10:13-rados-wip-31657-nautilus-distro-basic-smithi/4534580/
- Status changed from 12 to New
- Subject changed from bluestore: ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2 failure to nautilus: bluestore: ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2 failure
- Subject changed from nautilus: bluestore: ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2 failure to bluestore: ObjectStore/StoreTestSpecificAUSize.Many4KWritesNoCSumTest/2 failure
/a/yuriw-2020-04-28_21:58:13-rados-wip-yuri-testing-2020-04-24-1941-master-distro-basic-smithi/4995289
- Priority changed from High to Low
- Backport changed from nautilus to nautilus,octopus
Not critical from user standpoint.
Also available in: Atom
PDF