Bug #325
writes starve reads
0%
Description
From: Henry C Chang <henry_c_chang@tcloudcomputing.com>
To: ceph-devel@vger.kernel.org
Subject: writes-starving-reads problem
Hi,
I ran into "writes-starving-reads" problem on a machine with 48G RAM.
I have two processes running concurrently:
Process 1 copies an 8 GB file from local disk to CEPH.
Process 2 copies an 8 GB file from CEPH to local disk.
Process 1 finished quickly since the data were cached in memory first.
Process 2 was blocked until all the cached data have been written to OSD.
(Maybe not all the cached data, but I waited for a long time.)
I don't have this problem on a machine with 8G RAM.
Has anyone experienced this before or any idea why?
Thanks,
Henry
History
#1 Updated by Sage Weil over 13 years ago
- Subject changed from write starve reads to writes starve reads
#2 Updated by Sage Weil over 13 years ago
- Target version set to v0.22
#3 Updated by Sage Weil over 13 years ago
- Target version changed from v0.22 to v0.21.4
#4 Updated by Greg Farnum over 13 years ago
Hmm, I think I'm seeing this problem or an alternative instance of it, running a multi-mds cluster locally. bonnie++ (in the kclient, uml) was taking a long while to finish cleaning up so I wanted to list the bonnie directory and see how much stuff was left. But the ls is just hanging.
This looks to be a problem with the client -- there are no requests in-flight for the ls, just for bonnie cleanup.
And mounting the system via cfuse I was able to get a listing, although it was a little slow to respond.
This isn't very clean, though -- cfuse has crashed several times trying to repeat and after accidentally canceling my ls and starting again I have seen a getattr request on the root that took a while -- but then when it left the uncompleted-requests listing the ls didn't finish.
#5 Updated by Sage Weil over 13 years ago
- Target version changed from v0.21.4 to v0.23
#6 Updated by Sage Weil over 13 years ago
- Target version changed from v0.23 to 12
#7 Updated by Sage Weil almost 13 years ago
- Target version changed from 12 to v0.27
- translation missing: en.field_position deleted (
344) - translation missing: en.field_position set to 326
#8 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_story_points changed from 0 to 5
- translation missing: en.field_position deleted (
326) - translation missing: en.field_position set to 326
#9 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
327) - translation missing: en.field_position set to 341
#10 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
337) - translation missing: en.field_position set to 597
#11 Updated by Sage Weil almost 13 years ago
- Target version changed from v0.27 to v0.28
- translation missing: en.field_position deleted (
600) - translation missing: en.field_position set to 2
#12 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
16) - translation missing: en.field_position set to 629
#13 Updated by Greg Farnum almost 13 years ago
- Target version changed from v0.28 to v0.29
- translation missing: en.field_position deleted (
627) - translation missing: en.field_position set to 1
#14 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
3) - translation missing: en.field_position set to 2
#15 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
12) - translation missing: en.field_position set to 1
- translation missing: en.field_position changed from 1 to 657
#16 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
660) - translation missing: en.field_position set to 1
- translation missing: en.field_position changed from 1 to 663
#17 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
663) - translation missing: en.field_position set to 661
#18 Updated by Sage Weil almost 13 years ago
- Target version changed from v0.29 to v0.30
- translation missing: en.field_position deleted (
665) - translation missing: en.field_position set to 16
#19 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
26) - translation missing: en.field_position set to 682
#20 Updated by Sage Weil almost 13 years ago
- Target version changed from v0.30 to v0.31
- translation missing: en.field_position deleted (
684) - translation missing: en.field_position set to 6
#21 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
12) - translation missing: en.field_position set to 699
#22 Updated by Sage Weil almost 13 years ago
- translation missing: en.field_position deleted (
701) - translation missing: en.field_position set to 696
#23 Updated by Henry Chang almost 13 years ago
This problem seems to be fixed with the latest kclient and server. Cannot reproduce it now.
#24 Updated by Sage Weil over 12 years ago
- Status changed from New to Can't reproduce