Bug #21312
closed
occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure
Added by Sage Weil over 6 years ago.
Updated almost 5 years ago.
Backport:
luminous,mimic,nautilus
Description
2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:seeding object 0
2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:Op 0
2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-09-08T01:24:21.989 INFO:teuthology.orchestra.run.smithi057.stderr:Op 200
2017-09-08T01:24:21.989 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-09-08T01:24:36.791 INFO:teuthology.orchestra.run.smithi057.stderr:Op 400
2017-09-08T01:24:36.791 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-09-08T01:24:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:Op 600
2017-09-08T01:24:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-09-08T01:25:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:Op 800
2017-09-08T01:25:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1
2017-09-08T01:25:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/build/ceph-12.2.0-80-gded7118/src/test/objectstore/store_test.cc:5714: Failure
2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Expected: res_stat.allocated
2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Which is: 4259840
2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:To be equal to: max_object
2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Which is: 4194304
/a/yuriw-2017-09-07_19:30:56-rados-wip-yuri-testing4-2017-09-07-1811-distro-basic-smithi/1607591
Files
/a/sage-2017-09-10_02:44:16-rados-wip-yuri-testing4-2017-09-08-1631-distro-basic-smithi/1614995
http://pulpito.ceph.com/yuriw-2018-01-26_18:13:44-rados-wip_yuri_master_1.26.18-distro-basic-smithi/2112995/
2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout:/build/ceph-13.0.0-5148-g7fe6917/src/test/objectstore/store_test.cc:5813: Failure
2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: Expected: res_stat.allocated
2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: Which is: 4259840
2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout:To be equal to: max_object
2018-01-27T01:37:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: Which is: 4194304
2018-01-27T01:37:26.453 INFO:teuthology.orchestra.run.smithi182.stdout:==> rm -r bluestore.test_temp_dir
2018-01-27T01:37:26.469 INFO:teuthology.orchestra.run.smithi182.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2, where GetParam() = "bluestore" (17468 ms)
- Category set to Correctness/Safety
- Component(RADOS) BlueStore added
actual_allocated_size - expected_allocated_size = 4259840 - 4194304 = 0x10000
- Status changed from 12 to 7
- Assignee set to Igor Fedotov
- Status changed from 7 to 12
2018-02-11T15:41:30.493 INFO:teuthology.orchestra.run.mira112.stdout:/build/ceph-13.0.1-1854-gca7168c/src/test/objectstore/store_test.cc:5826: Failure
2018-02-11T15:41:30.493 INFO:teuthology.orchestra.run.mira112.stdout:Expected: (res_stat.allocated) <= (max_object_size), actual: 4259840 vs 4194304
2018-02-11T15:41:31.495 INFO:teuthology.orchestra.run.mira112.stdout:==> rm -r bluestore.test_temp_dir
2018-02-11T15:41:31.513 INFO:teuthology.orchestra.run.mira112.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2, where GetParam() = "bluestore" (13327 ms)
/a//kchai-2018-02-11_04:16:47-rados-wip-kefu-testing-2018-02-11-0959-distro-basic-mira/2181880
and the tested branch contains https://github.com/ceph/ceph/pull/20230
- Status changed from 12 to Fix Under Review
- Pull request ID set to 27444
The root cause seem to be the lack of proper fault_range() call in do_write_small(). It might happen that write request that should reuse existing blob is unable to locate such a blob as the latter is present in unloaded shard.
This happens when write request in question has loffset lower than the one for the the extent containing blob-to-reuse. fault_range() bounds don't include desired extent and hence corresponding shard might remain unloaded.
See attached log snippet with some inline comments for example
- Status changed from Fix Under Review to Pending Backport
- Backport changed from luminous to luminous,mimic,nautilus
- Project changed from RADOS to bluestore
- Category deleted (
Correctness/Safety)
- Copied to Backport #39254: luminous: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
- Copied to Backport #39255: mimic: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
- Copied to Backport #39256: nautilus: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF