Bug #21312
closedoccaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure
0%
Description
2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:seeding object 0 2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:Op 0 2017-09-08T01:24:09.540 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1 2017-09-08T01:24:21.989 INFO:teuthology.orchestra.run.smithi057.stderr:Op 200 2017-09-08T01:24:21.989 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1 2017-09-08T01:24:36.791 INFO:teuthology.orchestra.run.smithi057.stderr:Op 400 2017-09-08T01:24:36.791 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1 2017-09-08T01:24:54.123 INFO:teuthology.orchestra.run.smithi057.stderr:Op 600 2017-09-08T01:24:54.124 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1 2017-09-08T01:25:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:Op 800 2017-09-08T01:25:14.965 INFO:teuthology.orchestra.run.smithi057.stderr:available_objects: 0 in_flight_objects: 1 total objects: 1 in_flight 1 2017-09-08T01:25:40.504 INFO:teuthology.orchestra.run.smithi057.stdout:/build/ceph-12.2.0-80-gded7118/src/test/objectstore/store_test.cc:5714: Failure 2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Expected: res_stat.allocated 2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Which is: 4259840 2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout:To be equal to: max_object 2017-09-08T01:25:40.505 INFO:teuthology.orchestra.run.smithi057.stdout: Which is: 4194304
/a/yuriw-2017-09-07_19:30:56-rados-wip-yuri-testing4-2017-09-07-1811-distro-basic-smithi/1607591
Files
Updated by Sage Weil over 6 years ago
/a/sage-2017-09-10_02:44:16-rados-wip-yuri-testing4-2017-09-08-1631-distro-basic-smithi/1614995
Updated by Kefu Chai over 6 years ago
2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout:/build/ceph-13.0.0-5148-g7fe6917/src/test/objectstore/store_test.cc:5813: Failure 2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: Expected: res_stat.allocated 2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout: Which is: 4259840 2018-01-27T01:37:23.890 INFO:teuthology.orchestra.run.smithi182.stdout:To be equal to: max_object 2018-01-27T01:37:23.891 INFO:teuthology.orchestra.run.smithi182.stdout: Which is: 4194304 2018-01-27T01:37:26.453 INFO:teuthology.orchestra.run.smithi182.stdout:==> rm -r bluestore.test_temp_dir 2018-01-27T01:37:26.469 INFO:teuthology.orchestra.run.smithi182.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2, where GetParam() = "bluestore" (17468 ms)
Updated by Kefu Chai over 6 years ago
- Category set to Correctness/Safety
- Component(RADOS) BlueStore added
Updated by Kefu Chai over 6 years ago
actual_allocated_size - expected_allocated_size = 4259840 - 4194304 = 0x10000
Updated by Igor Fedotov about 6 years ago
Updated by Nathan Cutler about 6 years ago
- Status changed from 12 to 7
- Assignee set to Igor Fedotov
Updated by Kefu Chai about 6 years ago
- Status changed from 7 to 12
2018-02-11T15:41:30.493 INFO:teuthology.orchestra.run.mira112.stdout:/build/ceph-13.0.1-1854-gca7168c/src/test/objectstore/store_test.cc:5826: Failure 2018-02-11T15:41:30.493 INFO:teuthology.orchestra.run.mira112.stdout:Expected: (res_stat.allocated) <= (max_object_size), actual: 4259840 vs 4194304 2018-02-11T15:41:31.495 INFO:teuthology.orchestra.run.mira112.stdout:==> rm -r bluestore.test_temp_dir 2018-02-11T15:41:31.513 INFO:teuthology.orchestra.run.mira112.stdout:[ FAILED ] ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2, where GetParam() = "bluestore" (13327 ms)
/a//kchai-2018-02-11_04:16:47-rados-wip-kefu-testing-2018-02-11-0959-distro-basic-mira/2181880
and the tested branch contains https://github.com/ceph/ceph/pull/20230
Updated by Igor Fedotov about 5 years ago
- Status changed from 12 to Fix Under Review
- Pull request ID set to 27444
The root cause seem to be the lack of proper fault_range() call in do_write_small(). It might happen that write request that should reuse existing blob is unable to locate such a blob as the latter is present in unloaded shard.
This happens when write request in question has loffset lower than the one for the the extent containing blob-to-reuse. fault_range() bounds don't include desired extent and hence corresponding shard might remain unloaded.
See attached log snippet with some inline comments for example
Updated by Igor Fedotov about 5 years ago
Updated by Kefu Chai about 5 years ago
- Status changed from Fix Under Review to Pending Backport
- Backport changed from luminous to luminous,mimic,nautilus
Updated by Igor Fedotov about 5 years ago
- Project changed from RADOS to bluestore
- Category deleted (
Correctness/Safety)
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #39254: luminous: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #39255: mimic: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
Updated by Nathan Cutler about 5 years ago
- Copied to Backport #39256: nautilus: occaionsal ObjectStore/StoreTestSpecificAUSize.Many4KWritesTest/2 failure added
Updated by Igor Fedotov almost 5 years ago
- Status changed from Pending Backport to Resolved