Bug #17621
Hadoop does a bad job closing files and we end up holding too many caps
Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, Hadoop/Java
Labels (FS):
Java/Hadoop
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
(Just scribbling this in the tracker for a record.)
We've had several reports on the mailing list of Hadoop clients persistently holding too many caps. This seems to be related to the Hadoop infrastructure doing a bad job of closing files (possibly because it's relying on finalizers instead of explicitly calling close?). We should find a way to work around it in our bindings since Hadoop's job files are pretty safe for us to drop and re-open on-demand.
History
#1 Updated by Patrick Donnelly about 5 years ago
- Status changed from New to Rejected
Java/Hadoop testing is no longer a priority.
#2 Updated by Patrick Donnelly about 5 years ago
- Category deleted (
48) - Labels (FS) Java/Hadoop added